Jul 07 22:01:23.754724 ip-10-0-131-55 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 07 22:01:23.754735 ip-10-0-131-55 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 07 22:01:23.754743 ip-10-0-131-55 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 07 22:01:23.754969 ip-10-0-131-55 systemd[1]: Failed to start Kubernetes Kubelet. Jul 07 22:01:33.999190 ip-10-0-131-55 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 07 22:01:33.999210 ip-10-0-131-55 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 0fef0d97d90f493fac06f9dcded69ca8 -- Jul 07 22:03:36.338190 ip-10-0-131-55 systemd[1]: Starting Kubernetes Kubelet... Jul 07 22:03:38.027802 ip-10-0-131-55 kubenswrapper[2545]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 07 22:03:38.027802 ip-10-0-131-55 kubenswrapper[2545]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 07 22:03:38.027802 ip-10-0-131-55 kubenswrapper[2545]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 07 22:03:38.027802 ip-10-0-131-55 kubenswrapper[2545]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 07 22:03:38.027802 ip-10-0-131-55 kubenswrapper[2545]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 07 22:03:38.028485 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.027875 2545 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 07 22:03:38.032708 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.032692 2545 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 22:03:38.032741 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.032710 2545 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 22:03:38.032741 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.032715 2545 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 22:03:38.032741 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.032718 2545 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 22:03:38.032741 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.032721 2545 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 22:03:38.032741 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.032724 2545 feature_gate.go:328] unrecognized feature gate: Example Jul 07 22:03:38.032741 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.032727 2545 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 22:03:38.032741 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.032730 2545 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 22:03:38.032741 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.032732 2545 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 22:03:38.032741 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.032735 2545 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 22:03:38.032741 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.032738 2545 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 22:03:38.032741 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.032741 2545 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 22:03:38.032741 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.032746 2545 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 22:03:38.033016 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.032749 2545 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 22:03:38.033016 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.032752 2545 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 22:03:38.033016 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.032754 2545 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 22:03:38.033016 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.032757 2545 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 22:03:38.033016 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.032760 2545 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 22:03:38.033016 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.032763 2545 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 22:03:38.033016 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.032765 2545 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 22:03:38.033016 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.032768 2545 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 22:03:38.033016 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.032771 2545 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 22:03:38.033016 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.032774 2545 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 22:03:38.033016 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.032777 2545 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 22:03:38.033016 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.032780 2545 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 22:03:38.033016 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.032783 2545 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 22:03:38.033016 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.032786 2545 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 22:03:38.033016 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.032788 2545 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 22:03:38.033016 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.032791 2545 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 22:03:38.033016 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.032793 2545 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 22:03:38.033016 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.032796 2545 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 22:03:38.033016 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.032798 2545 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 22:03:38.033016 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.032801 2545 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 22:03:38.033499 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.032804 2545 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 22:03:38.033499 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.032807 2545 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 22:03:38.033499 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.032809 2545 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 22:03:38.033499 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.032812 2545 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 22:03:38.033499 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.032815 2545 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 22:03:38.033499 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.032818 2545 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 22:03:38.033499 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.032820 2545 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 22:03:38.033499 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.032823 2545 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 22:03:38.033499 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.032825 2545 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 22:03:38.033499 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.032828 2545 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 22:03:38.033499 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.032830 2545 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 22:03:38.033499 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.032833 2545 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 22:03:38.033499 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.032835 2545 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 22:03:38.033499 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.032838 2545 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 22:03:38.033499 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.032841 2545 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 22:03:38.033499 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.032844 2545 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 22:03:38.033499 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.032847 2545 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 22:03:38.033499 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.032849 2545 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 22:03:38.033499 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.032852 2545 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 22:03:38.033499 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.032855 2545 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 22:03:38.033974 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.032858 2545 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 22:03:38.033974 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.032860 2545 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 22:03:38.033974 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.032863 2545 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 22:03:38.033974 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.032867 2545 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 22:03:38.033974 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.032871 2545 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 22:03:38.033974 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.032874 2545 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 22:03:38.033974 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.032877 2545 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 22:03:38.033974 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.032880 2545 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 22:03:38.033974 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.032883 2545 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 22:03:38.033974 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.032885 2545 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 22:03:38.033974 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.032888 2545 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 22:03:38.033974 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.032890 2545 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 22:03:38.033974 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.032893 2545 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 22:03:38.033974 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.032896 2545 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 22:03:38.033974 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.032898 2545 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 22:03:38.033974 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.032901 2545 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 22:03:38.033974 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.032904 2545 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 22:03:38.033974 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.032906 2545 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 22:03:38.033974 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.032909 2545 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 22:03:38.034465 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.032912 2545 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 22:03:38.034465 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.032915 2545 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 22:03:38.034465 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.032918 2545 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 22:03:38.034465 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.032920 2545 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 22:03:38.034465 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.032923 2545 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 22:03:38.034465 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.032926 2545 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 22:03:38.034465 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.032929 2545 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 22:03:38.034465 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.032934 2545 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 22:03:38.034465 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.032937 2545 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 22:03:38.034465 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.032940 2545 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 22:03:38.034465 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.032943 2545 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 22:03:38.034465 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.032946 2545 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 22:03:38.034465 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.032948 2545 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 22:03:38.034465 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.032951 2545 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 22:03:38.034465 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.032953 2545 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 22:03:38.034465 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.032956 2545 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 22:03:38.034465 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.032959 2545 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 22:03:38.034465 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.032962 2545 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 22:03:38.034465 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.032965 2545 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 22:03:38.034465 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.032967 2545 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 22:03:38.034948 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.032970 2545 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 22:03:38.034948 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.032972 2545 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 22:03:38.034948 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.032975 2545 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 22:03:38.034948 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.032977 2545 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 22:03:38.034948 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.032980 2545 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 22:03:38.034948 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.032983 2545 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 22:03:38.034948 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.032986 2545 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 22:03:38.034948 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.033707 2545 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 22:03:38.034948 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.033714 2545 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 22:03:38.034948 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.033717 2545 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 22:03:38.034948 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.033720 2545 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 22:03:38.034948 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.033725 2545 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 22:03:38.034948 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.033729 2545 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 22:03:38.034948 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.033732 2545 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 22:03:38.034948 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.033735 2545 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 22:03:38.034948 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.033738 2545 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 22:03:38.034948 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.033742 2545 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 22:03:38.034948 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.033744 2545 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 22:03:38.034948 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.033749 2545 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 22:03:38.035405 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.033752 2545 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 22:03:38.035405 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.033755 2545 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 22:03:38.035405 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.033757 2545 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 22:03:38.035405 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.033761 2545 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 22:03:38.035405 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.033763 2545 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 22:03:38.035405 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.033766 2545 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 22:03:38.035405 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.033768 2545 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 22:03:38.035405 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.033771 2545 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 22:03:38.035405 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.033774 2545 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 22:03:38.035405 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.033776 2545 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 22:03:38.035405 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.033779 2545 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 22:03:38.035405 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.033781 2545 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 22:03:38.035405 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.033784 2545 feature_gate.go:328] unrecognized feature gate: Example Jul 07 22:03:38.035405 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.033786 2545 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 22:03:38.035405 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.033789 2545 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 22:03:38.035405 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.033791 2545 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 22:03:38.035405 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.033794 2545 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 22:03:38.035405 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.033797 2545 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 22:03:38.035405 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.033800 2545 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 22:03:38.035405 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.033802 2545 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 22:03:38.035927 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.033805 2545 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 22:03:38.035927 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.033808 2545 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 22:03:38.035927 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.033810 2545 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 22:03:38.035927 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.033813 2545 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 22:03:38.035927 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.033817 2545 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 22:03:38.035927 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.033819 2545 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 22:03:38.035927 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.033822 2545 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 22:03:38.035927 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.033824 2545 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 22:03:38.035927 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.033827 2545 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 22:03:38.035927 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.033830 2545 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 22:03:38.035927 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.033832 2545 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 22:03:38.035927 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.033835 2545 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 22:03:38.035927 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.033840 2545 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 22:03:38.035927 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.033843 2545 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 22:03:38.035927 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.033846 2545 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 22:03:38.035927 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.033849 2545 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 22:03:38.035927 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.033851 2545 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 22:03:38.035927 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.033854 2545 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 22:03:38.035927 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.033856 2545 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 22:03:38.035927 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.033859 2545 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 22:03:38.036444 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.033861 2545 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 22:03:38.036444 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.033864 2545 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 22:03:38.036444 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.033880 2545 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 22:03:38.036444 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.033884 2545 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 22:03:38.036444 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.033887 2545 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 22:03:38.036444 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.033890 2545 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 22:03:38.036444 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.033894 2545 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 22:03:38.036444 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.033896 2545 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 22:03:38.036444 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.033899 2545 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 22:03:38.036444 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.033902 2545 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 22:03:38.036444 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.033905 2545 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 22:03:38.036444 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.033908 2545 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 22:03:38.036444 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.033911 2545 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 22:03:38.036444 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.033913 2545 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 22:03:38.036444 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.033916 2545 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 22:03:38.036444 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.033918 2545 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 22:03:38.036444 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.033921 2545 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 22:03:38.036444 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.033923 2545 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 22:03:38.036444 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.033926 2545 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 22:03:38.036444 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.033928 2545 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 22:03:38.036921 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.033931 2545 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 22:03:38.036921 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.033934 2545 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 22:03:38.036921 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.033936 2545 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 22:03:38.036921 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.033939 2545 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 22:03:38.036921 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.033942 2545 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 22:03:38.036921 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.033945 2545 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 22:03:38.036921 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.033948 2545 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 22:03:38.036921 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.033950 2545 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 22:03:38.036921 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.033953 2545 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 22:03:38.036921 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.033956 2545 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 22:03:38.036921 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.033958 2545 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 22:03:38.036921 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.033961 2545 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 22:03:38.036921 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.033963 2545 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 22:03:38.036921 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.033966 2545 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 22:03:38.036921 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.033969 2545 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 22:03:38.036921 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.033972 2545 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 22:03:38.036921 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.033974 2545 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 22:03:38.036921 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.033977 2545 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 22:03:38.036921 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.033981 2545 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 22:03:38.037386 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.033985 2545 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 22:03:38.037386 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.033988 2545 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 22:03:38.037386 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.033991 2545 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 22:03:38.037386 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.033994 2545 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 22:03:38.037386 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.033997 2545 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 22:03:38.037386 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.034000 2545 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 22:03:38.037386 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.034003 2545 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 22:03:38.037386 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.034005 2545 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 22:03:38.037386 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.034398 2545 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 22:03:38.037386 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.034403 2545 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 22:03:38.037386 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.034406 2545 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 22:03:38.037386 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.034408 2545 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 22:03:38.037386 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.034413 2545 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 22:03:38.037386 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.034417 2545 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 22:03:38.037386 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.034420 2545 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 22:03:38.037386 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.034422 2545 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 22:03:38.037386 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.034425 2545 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 22:03:38.037386 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.034428 2545 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 22:03:38.037386 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.034432 2545 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 22:03:38.037386 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.034434 2545 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 22:03:38.037856 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.034437 2545 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 22:03:38.037856 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.034440 2545 feature_gate.go:328] unrecognized feature gate: Example Jul 07 22:03:38.037856 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.034442 2545 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 22:03:38.037856 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.034445 2545 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 22:03:38.037856 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.034447 2545 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 22:03:38.037856 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.034450 2545 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 22:03:38.037856 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.034453 2545 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 22:03:38.037856 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.034455 2545 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 22:03:38.037856 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.034457 2545 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 22:03:38.037856 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.034460 2545 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 22:03:38.037856 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.034463 2545 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 22:03:38.037856 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.034465 2545 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 22:03:38.037856 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.034468 2545 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 22:03:38.037856 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.034471 2545 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 22:03:38.037856 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.034474 2545 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 22:03:38.037856 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.034477 2545 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 22:03:38.037856 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.034480 2545 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 22:03:38.037856 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.034482 2545 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 22:03:38.037856 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.034485 2545 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 22:03:38.037856 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.034488 2545 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 22:03:38.038356 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.034490 2545 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 22:03:38.038356 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.034494 2545 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 22:03:38.038356 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.034497 2545 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 22:03:38.038356 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.034501 2545 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 22:03:38.038356 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.034504 2545 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 22:03:38.038356 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.034507 2545 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 22:03:38.038356 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.034511 2545 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 22:03:38.038356 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.034515 2545 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 22:03:38.038356 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.034518 2545 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 22:03:38.038356 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.034521 2545 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 22:03:38.038356 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.034524 2545 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 22:03:38.038356 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.034527 2545 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 22:03:38.038356 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.034530 2545 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 22:03:38.038356 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.034532 2545 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 22:03:38.038356 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.034535 2545 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 22:03:38.038356 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.034538 2545 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 22:03:38.038356 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.034540 2545 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 22:03:38.038356 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.034543 2545 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 22:03:38.038356 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.034545 2545 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 22:03:38.038810 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.034548 2545 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 22:03:38.038810 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.034551 2545 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 22:03:38.038810 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.034553 2545 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 22:03:38.038810 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.034556 2545 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 22:03:38.038810 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.034558 2545 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 22:03:38.038810 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.034561 2545 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 22:03:38.038810 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.034563 2545 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 22:03:38.038810 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.034565 2545 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 22:03:38.038810 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.034568 2545 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 22:03:38.038810 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.034570 2545 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 22:03:38.038810 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.034573 2545 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 22:03:38.038810 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.034575 2545 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 22:03:38.038810 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.034579 2545 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 22:03:38.038810 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.034581 2545 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 22:03:38.038810 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.034584 2545 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 22:03:38.038810 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.034586 2545 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 22:03:38.038810 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.034589 2545 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 22:03:38.038810 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.034591 2545 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 22:03:38.038810 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.034594 2545 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 22:03:38.039265 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.034597 2545 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 22:03:38.039265 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.034599 2545 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 22:03:38.039265 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.034602 2545 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 22:03:38.039265 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.034604 2545 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 22:03:38.039265 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.034610 2545 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 22:03:38.039265 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.034612 2545 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 22:03:38.039265 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.034615 2545 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 22:03:38.039265 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.034618 2545 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 22:03:38.039265 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.034621 2545 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 22:03:38.039265 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.034623 2545 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 22:03:38.039265 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.034626 2545 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 22:03:38.039265 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.034628 2545 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 22:03:38.039265 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.034631 2545 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 22:03:38.039265 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.034633 2545 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 22:03:38.039265 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.034636 2545 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 22:03:38.039265 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.034638 2545 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 22:03:38.039265 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.034641 2545 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 22:03:38.039265 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.034643 2545 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 22:03:38.039265 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.034646 2545 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 22:03:38.039265 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.034648 2545 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 22:03:38.039731 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.034651 2545 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 22:03:38.039731 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.034654 2545 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 22:03:38.039731 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.034656 2545 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 22:03:38.039731 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.034659 2545 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 22:03:38.039731 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.034661 2545 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 22:03:38.039731 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.034664 2545 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 22:03:38.039731 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.034667 2545 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 22:03:38.039731 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.034669 2545 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 22:03:38.039731 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.034672 2545 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 22:03:38.039731 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.034742 2545 flags.go:64] FLAG: --address="0.0.0.0" Jul 07 22:03:38.039731 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.034753 2545 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 07 22:03:38.039731 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.034767 2545 flags.go:64] FLAG: --anonymous-auth="true" Jul 07 22:03:38.039731 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.034781 2545 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 07 22:03:38.039731 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.034785 2545 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 07 22:03:38.039731 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.034788 2545 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 07 22:03:38.039731 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.034792 2545 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 07 22:03:38.039731 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.034798 2545 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 07 22:03:38.039731 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.034802 2545 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 07 22:03:38.039731 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.034805 2545 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 07 22:03:38.039731 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.034809 2545 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 07 22:03:38.039731 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.034812 2545 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 07 22:03:38.040268 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.034815 2545 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 07 22:03:38.040268 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.034818 2545 flags.go:64] FLAG: --cgroup-root="" Jul 07 22:03:38.040268 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.034821 2545 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 07 22:03:38.040268 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.034824 2545 flags.go:64] FLAG: --client-ca-file="" Jul 07 22:03:38.040268 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.034827 2545 flags.go:64] FLAG: --cloud-provider="external" Jul 07 22:03:38.040268 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.034830 2545 flags.go:64] FLAG: --cluster-dns="[]" Jul 07 22:03:38.040268 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.034834 2545 flags.go:64] FLAG: --cluster-domain="" Jul 07 22:03:38.040268 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.034837 2545 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 07 22:03:38.040268 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.034840 2545 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 07 22:03:38.040268 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.034843 2545 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 07 22:03:38.040268 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.034847 2545 flags.go:64] FLAG: --container-log-max-files="5" Jul 07 22:03:38.040268 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.034851 2545 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 07 22:03:38.040268 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.034853 2545 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 07 22:03:38.040268 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.034856 2545 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 07 22:03:38.040268 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.034860 2545 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 07 22:03:38.040268 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.034863 2545 flags.go:64] FLAG: --contention-profiling="false" Jul 07 22:03:38.040268 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.034866 2545 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 07 22:03:38.040268 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.034869 2545 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 07 22:03:38.040268 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.034872 2545 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 07 22:03:38.040268 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.034875 2545 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 07 22:03:38.040268 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.034879 2545 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 07 22:03:38.040268 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.034882 2545 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 07 22:03:38.040268 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.034885 2545 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 07 22:03:38.040268 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.034887 2545 flags.go:64] FLAG: --enable-load-reader="false" Jul 07 22:03:38.040268 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.034890 2545 flags.go:64] FLAG: --enable-server="true" Jul 07 22:03:38.040849 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.034893 2545 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 07 22:03:38.040849 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.034898 2545 flags.go:64] FLAG: --event-burst="100" Jul 07 22:03:38.040849 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.034902 2545 flags.go:64] FLAG: --event-qps="50" Jul 07 22:03:38.040849 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.034905 2545 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 07 22:03:38.040849 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.034908 2545 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 07 22:03:38.040849 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.034911 2545 flags.go:64] FLAG: --eviction-hard="" Jul 07 22:03:38.040849 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.034915 2545 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 07 22:03:38.040849 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.034918 2545 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 07 22:03:38.040849 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.034921 2545 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 07 22:03:38.040849 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.034924 2545 flags.go:64] FLAG: --eviction-soft="" Jul 07 22:03:38.040849 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.034927 2545 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 07 22:03:38.040849 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.034930 2545 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 07 22:03:38.040849 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.034932 2545 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 07 22:03:38.040849 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.034935 2545 flags.go:64] FLAG: --experimental-mounter-path="" Jul 07 22:03:38.040849 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.034938 2545 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 07 22:03:38.040849 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.034941 2545 flags.go:64] FLAG: --fail-swap-on="true" Jul 07 22:03:38.040849 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.034943 2545 flags.go:64] FLAG: --feature-gates="" Jul 07 22:03:38.040849 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.034947 2545 flags.go:64] FLAG: --file-check-frequency="20s" Jul 07 22:03:38.040849 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.034950 2545 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 07 22:03:38.040849 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.034953 2545 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 07 22:03:38.040849 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.034956 2545 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 07 22:03:38.040849 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.034960 2545 flags.go:64] FLAG: --healthz-port="10248" Jul 07 22:03:38.040849 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.034963 2545 flags.go:64] FLAG: --help="false" Jul 07 22:03:38.040849 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.034966 2545 flags.go:64] FLAG: --hostname-override="ip-10-0-131-55.ec2.internal" Jul 07 22:03:38.040849 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.034970 2545 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 07 22:03:38.041528 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.034973 2545 flags.go:64] FLAG: --http-check-frequency="20s" Jul 07 22:03:38.041528 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.034976 2545 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 07 22:03:38.041528 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.034980 2545 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 07 22:03:38.041528 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.034983 2545 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 07 22:03:38.041528 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.034986 2545 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 07 22:03:38.041528 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.034988 2545 flags.go:64] FLAG: --image-service-endpoint="" Jul 07 22:03:38.041528 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.034991 2545 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 07 22:03:38.041528 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.034994 2545 flags.go:64] FLAG: --kube-api-burst="100" Jul 07 22:03:38.041528 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.034997 2545 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 07 22:03:38.041528 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.035001 2545 flags.go:64] FLAG: --kube-api-qps="50" Jul 07 22:03:38.041528 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.035003 2545 flags.go:64] FLAG: --kube-reserved="" Jul 07 22:03:38.041528 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.035006 2545 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 07 22:03:38.041528 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.035009 2545 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 07 22:03:38.041528 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.035012 2545 flags.go:64] FLAG: --kubelet-cgroups="" Jul 07 22:03:38.041528 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.035015 2545 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 07 22:03:38.041528 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.035018 2545 flags.go:64] FLAG: --lock-file="" Jul 07 22:03:38.041528 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.035021 2545 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 07 22:03:38.041528 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.035024 2545 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 07 22:03:38.041528 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.035026 2545 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 07 22:03:38.041528 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.035032 2545 flags.go:64] FLAG: --log-json-split-stream="false" Jul 07 22:03:38.041528 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.035034 2545 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 07 22:03:38.041528 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.035037 2545 flags.go:64] FLAG: --log-text-split-stream="false" Jul 07 22:03:38.041528 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.035040 2545 flags.go:64] FLAG: --logging-format="text" Jul 07 22:03:38.042092 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.035043 2545 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 07 22:03:38.042092 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.035046 2545 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 07 22:03:38.042092 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.035049 2545 flags.go:64] FLAG: --manifest-url="" Jul 07 22:03:38.042092 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.035051 2545 flags.go:64] FLAG: --manifest-url-header="" Jul 07 22:03:38.042092 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.035061 2545 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 07 22:03:38.042092 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.035064 2545 flags.go:64] FLAG: --max-open-files="1000000" Jul 07 22:03:38.042092 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.035071 2545 flags.go:64] FLAG: --max-pods="110" Jul 07 22:03:38.042092 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.035074 2545 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 07 22:03:38.042092 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.035092 2545 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 07 22:03:38.042092 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.035095 2545 flags.go:64] FLAG: --memory-manager-policy="None" Jul 07 22:03:38.042092 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.035098 2545 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 07 22:03:38.042092 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.035101 2545 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 07 22:03:38.042092 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.035104 2545 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 07 22:03:38.042092 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.035107 2545 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 07 22:03:38.042092 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.035114 2545 flags.go:64] FLAG: --node-status-max-images="50" Jul 07 22:03:38.042092 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.035117 2545 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 07 22:03:38.042092 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.035120 2545 flags.go:64] FLAG: --oom-score-adj="-999" Jul 07 22:03:38.042092 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.035123 2545 flags.go:64] FLAG: --pod-cidr="" Jul 07 22:03:38.042092 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.035126 2545 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 07 22:03:38.042092 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.035131 2545 flags.go:64] FLAG: --pod-manifest-path="" Jul 07 22:03:38.042092 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.035134 2545 flags.go:64] FLAG: --pod-max-pids="-1" Jul 07 22:03:38.042092 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.035137 2545 flags.go:64] FLAG: --pods-per-core="0" Jul 07 22:03:38.042092 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.035140 2545 flags.go:64] FLAG: --port="10250" Jul 07 22:03:38.042092 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.035143 2545 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 07 22:03:38.042696 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.035147 2545 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0aff6074362dfdfcd" Jul 07 22:03:38.042696 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.035150 2545 flags.go:64] FLAG: --qos-reserved="" Jul 07 22:03:38.042696 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.035153 2545 flags.go:64] FLAG: --read-only-port="10255" Jul 07 22:03:38.042696 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.035156 2545 flags.go:64] FLAG: --register-node="true" Jul 07 22:03:38.042696 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.035159 2545 flags.go:64] FLAG: --register-with-taints="" Jul 07 22:03:38.042696 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.035166 2545 flags.go:64] FLAG: --registry-burst="10" Jul 07 22:03:38.042696 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.035170 2545 flags.go:64] FLAG: --registry-qps="5" Jul 07 22:03:38.042696 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.035172 2545 flags.go:64] FLAG: --reserved-cpus="" Jul 07 22:03:38.042696 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.035175 2545 flags.go:64] FLAG: --reserved-memory="" Jul 07 22:03:38.042696 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.035179 2545 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 07 22:03:38.042696 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.035182 2545 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 07 22:03:38.042696 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.035185 2545 flags.go:64] FLAG: --rotate-certificates="false" Jul 07 22:03:38.042696 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.035188 2545 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 07 22:03:38.042696 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.035191 2545 flags.go:64] FLAG: --runonce="false" Jul 07 22:03:38.042696 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.035194 2545 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 07 22:03:38.042696 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.035197 2545 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 07 22:03:38.042696 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.035200 2545 flags.go:64] FLAG: --seccomp-default="false" Jul 07 22:03:38.042696 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.035205 2545 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 07 22:03:38.042696 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.035208 2545 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 07 22:03:38.042696 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.035212 2545 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 07 22:03:38.042696 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.035215 2545 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 07 22:03:38.042696 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.035218 2545 flags.go:64] FLAG: --storage-driver-password="root" Jul 07 22:03:38.042696 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.035220 2545 flags.go:64] FLAG: --storage-driver-secure="false" Jul 07 22:03:38.042696 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.035223 2545 flags.go:64] FLAG: --storage-driver-table="stats" Jul 07 22:03:38.042696 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.035238 2545 flags.go:64] FLAG: --storage-driver-user="root" Jul 07 22:03:38.043316 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.035242 2545 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 07 22:03:38.043316 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.035246 2545 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 07 22:03:38.043316 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.035249 2545 flags.go:64] FLAG: --system-cgroups="" Jul 07 22:03:38.043316 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.035252 2545 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 07 22:03:38.043316 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.035257 2545 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 07 22:03:38.043316 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.035260 2545 flags.go:64] FLAG: --tls-cert-file="" Jul 07 22:03:38.043316 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.035263 2545 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 07 22:03:38.043316 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.035267 2545 flags.go:64] FLAG: --tls-min-version="" Jul 07 22:03:38.043316 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.035270 2545 flags.go:64] FLAG: --tls-private-key-file="" Jul 07 22:03:38.043316 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.035273 2545 flags.go:64] FLAG: --topology-manager-policy="none" Jul 07 22:03:38.043316 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.035276 2545 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 07 22:03:38.043316 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.035278 2545 flags.go:64] FLAG: --topology-manager-scope="container" Jul 07 22:03:38.043316 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.035281 2545 flags.go:64] FLAG: --v="2" Jul 07 22:03:38.043316 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.035285 2545 flags.go:64] FLAG: --version="false" Jul 07 22:03:38.043316 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.035289 2545 flags.go:64] FLAG: --vmodule="" Jul 07 22:03:38.043316 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.035292 2545 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 07 22:03:38.043316 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.035296 2545 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 07 22:03:38.043316 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.035417 2545 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 22:03:38.043316 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.035421 2545 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 22:03:38.043316 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.035426 2545 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 22:03:38.043316 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.035429 2545 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 22:03:38.043316 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.035433 2545 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 22:03:38.043316 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.035436 2545 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 22:03:38.043870 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.035438 2545 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 22:03:38.043870 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.035443 2545 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 22:03:38.043870 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.035446 2545 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 22:03:38.043870 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.035449 2545 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 22:03:38.043870 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.035451 2545 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 22:03:38.043870 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.035454 2545 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 22:03:38.043870 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.035457 2545 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 22:03:38.043870 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.035459 2545 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 22:03:38.043870 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.035462 2545 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 22:03:38.043870 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.035465 2545 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 22:03:38.043870 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.035468 2545 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 22:03:38.043870 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.035470 2545 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 22:03:38.043870 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.035473 2545 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 22:03:38.043870 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.035476 2545 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 22:03:38.043870 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.035478 2545 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 22:03:38.043870 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.035481 2545 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 22:03:38.043870 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.035484 2545 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 22:03:38.043870 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.035486 2545 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 22:03:38.043870 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.035489 2545 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 22:03:38.044351 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.035492 2545 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 22:03:38.044351 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.035495 2545 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 22:03:38.044351 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.035497 2545 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 22:03:38.044351 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.035500 2545 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 22:03:38.044351 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.035502 2545 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 22:03:38.044351 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.035505 2545 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 22:03:38.044351 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.035508 2545 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 22:03:38.044351 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.035511 2545 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 22:03:38.044351 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.035513 2545 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 22:03:38.044351 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.035516 2545 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 22:03:38.044351 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.035520 2545 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 22:03:38.044351 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.035522 2545 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 22:03:38.044351 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.035525 2545 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 22:03:38.044351 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.035527 2545 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 22:03:38.044351 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.035532 2545 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 22:03:38.044351 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.035534 2545 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 22:03:38.044351 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.035537 2545 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 22:03:38.044351 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.035540 2545 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 22:03:38.044351 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.035542 2545 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 22:03:38.044351 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.035545 2545 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 22:03:38.044831 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.035547 2545 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 22:03:38.044831 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.035550 2545 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 22:03:38.044831 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.035553 2545 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 22:03:38.044831 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.035555 2545 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 22:03:38.044831 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.035559 2545 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 22:03:38.044831 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.035562 2545 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 22:03:38.044831 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.035565 2545 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 22:03:38.044831 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.035568 2545 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 22:03:38.044831 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.035571 2545 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 22:03:38.044831 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.035574 2545 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 22:03:38.044831 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.035577 2545 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 22:03:38.044831 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.035580 2545 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 22:03:38.044831 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.035583 2545 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 22:03:38.044831 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.035586 2545 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 22:03:38.044831 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.035588 2545 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 22:03:38.044831 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.035591 2545 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 22:03:38.044831 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.035593 2545 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 22:03:38.044831 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.035596 2545 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 22:03:38.044831 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.035599 2545 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 22:03:38.044831 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.035601 2545 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 22:03:38.045371 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.035604 2545 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 22:03:38.045371 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.035606 2545 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 22:03:38.045371 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.035612 2545 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 22:03:38.045371 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.035614 2545 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 22:03:38.045371 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.035617 2545 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 22:03:38.045371 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.035619 2545 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 22:03:38.045371 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.035623 2545 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 22:03:38.045371 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.035626 2545 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 22:03:38.045371 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.035629 2545 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 22:03:38.045371 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.035633 2545 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 22:03:38.045371 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.035636 2545 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 22:03:38.045371 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.035639 2545 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 22:03:38.045371 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.035642 2545 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 22:03:38.045371 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.035644 2545 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 22:03:38.045371 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.035647 2545 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 22:03:38.045371 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.035649 2545 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 22:03:38.045371 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.035652 2545 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 22:03:38.045371 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.035654 2545 feature_gate.go:328] unrecognized feature gate: Example Jul 07 22:03:38.045371 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.035657 2545 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 22:03:38.045371 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.035659 2545 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 22:03:38.045927 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.035662 2545 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 22:03:38.045927 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.035665 2545 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 22:03:38.045927 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.035668 2545 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 22:03:38.045927 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.035670 2545 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 22:03:38.045927 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.035673 2545 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 22:03:38.045927 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.035676 2545 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 22:03:38.045927 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.035678 2545 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 22:03:38.045927 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.035680 2545 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 22:03:38.045927 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.035683 2545 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 22:03:38.045927 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.035685 2545 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 22:03:38.045927 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.035688 2545 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 22:03:38.045927 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.035691 2545 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 22:03:38.045927 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.035693 2545 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 22:03:38.045927 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.035696 2545 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 22:03:38.045927 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.036644 2545 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 07 22:03:38.046393 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.043783 2545 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 07 22:03:38.046393 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.043796 2545 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 07 22:03:38.046393 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.044636 2545 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 07 22:03:38.046393 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.044646 2545 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 07 22:03:38.046393 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.044704 2545 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 22:03:38.046393 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.044709 2545 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 22:03:38.046393 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.044712 2545 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 22:03:38.046393 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.044715 2545 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 22:03:38.046393 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.044718 2545 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 22:03:38.046393 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.044722 2545 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 22:03:38.046393 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.044724 2545 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 22:03:38.046393 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.044727 2545 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 22:03:38.046393 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.044730 2545 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 22:03:38.046393 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.044732 2545 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 22:03:38.046393 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.044735 2545 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 22:03:38.046393 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.044737 2545 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 22:03:38.046393 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.044742 2545 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 22:03:38.046393 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.044746 2545 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 22:03:38.046393 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.044749 2545 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 22:03:38.046839 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.044752 2545 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 22:03:38.046839 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.044755 2545 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 22:03:38.046839 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.044757 2545 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 22:03:38.046839 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.044760 2545 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 22:03:38.046839 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.044763 2545 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 22:03:38.046839 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.044767 2545 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 22:03:38.046839 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.044770 2545 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 22:03:38.046839 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.044774 2545 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 22:03:38.046839 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.044777 2545 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 22:03:38.046839 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.044779 2545 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 22:03:38.046839 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.044782 2545 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 22:03:38.046839 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.044784 2545 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 22:03:38.046839 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.044787 2545 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 22:03:38.046839 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.044790 2545 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 22:03:38.046839 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.044794 2545 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 22:03:38.046839 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.044796 2545 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 22:03:38.046839 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.044799 2545 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 22:03:38.046839 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.044802 2545 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 22:03:38.046839 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.044804 2545 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 22:03:38.046839 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.044807 2545 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 22:03:38.047335 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.044809 2545 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 22:03:38.047335 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.044812 2545 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 22:03:38.047335 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.044814 2545 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 22:03:38.047335 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.044817 2545 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 22:03:38.047335 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.044820 2545 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 22:03:38.047335 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.044822 2545 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 22:03:38.047335 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.044824 2545 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 22:03:38.047335 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.044827 2545 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 22:03:38.047335 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.044830 2545 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 22:03:38.047335 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.044832 2545 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 22:03:38.047335 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.044835 2545 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 22:03:38.047335 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.044838 2545 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 22:03:38.047335 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.044840 2545 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 22:03:38.047335 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.044844 2545 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 22:03:38.047335 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.044848 2545 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 22:03:38.047335 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.044850 2545 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 22:03:38.047335 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.044853 2545 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 22:03:38.047335 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.044856 2545 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 22:03:38.047335 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.044858 2545 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 22:03:38.047786 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.044861 2545 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 22:03:38.047786 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.044863 2545 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 22:03:38.047786 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.044866 2545 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 22:03:38.047786 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.044869 2545 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 22:03:38.047786 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.044871 2545 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 22:03:38.047786 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.044874 2545 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 22:03:38.047786 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.044877 2545 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 22:03:38.047786 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.044879 2545 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 22:03:38.047786 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.044882 2545 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 22:03:38.047786 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.044885 2545 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 22:03:38.047786 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.044888 2545 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 22:03:38.047786 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.044890 2545 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 22:03:38.047786 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.044893 2545 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 22:03:38.047786 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.044895 2545 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 22:03:38.047786 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.044898 2545 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 22:03:38.047786 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.044900 2545 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 22:03:38.047786 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.044903 2545 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 22:03:38.047786 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.044905 2545 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 22:03:38.047786 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.044908 2545 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 22:03:38.047786 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.044911 2545 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 22:03:38.048289 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.044913 2545 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 22:03:38.048289 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.044915 2545 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 22:03:38.048289 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.044918 2545 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 22:03:38.048289 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.044921 2545 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 22:03:38.048289 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.044923 2545 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 22:03:38.048289 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.044926 2545 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 22:03:38.048289 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.044928 2545 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 22:03:38.048289 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.044931 2545 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 22:03:38.048289 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.044933 2545 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 22:03:38.048289 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.044936 2545 feature_gate.go:328] unrecognized feature gate: Example Jul 07 22:03:38.048289 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.044938 2545 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 22:03:38.048289 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.044940 2545 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 22:03:38.048289 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.044943 2545 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 22:03:38.048289 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.044946 2545 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 22:03:38.048289 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.044948 2545 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 22:03:38.048289 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.044951 2545 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 22:03:38.048289 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.044954 2545 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 22:03:38.048289 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.044956 2545 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 22:03:38.048289 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.044959 2545 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 22:03:38.048289 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.044961 2545 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 22:03:38.048751 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.044964 2545 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 22:03:38.048751 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.044967 2545 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 22:03:38.048751 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.044970 2545 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 22:03:38.048751 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.044972 2545 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 22:03:38.048751 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.044975 2545 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 22:03:38.048751 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.044980 2545 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 07 22:03:38.048751 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.045102 2545 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 07 22:03:38.048751 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.045107 2545 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 07 22:03:38.048751 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.045110 2545 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 07 22:03:38.048751 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.045113 2545 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 07 22:03:38.048751 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.045116 2545 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 07 22:03:38.048751 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.045119 2545 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 07 22:03:38.048751 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.045123 2545 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 07 22:03:38.048751 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.045128 2545 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 07 22:03:38.049096 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.045131 2545 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 07 22:03:38.049096 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.045134 2545 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 07 22:03:38.049096 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.045137 2545 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 07 22:03:38.049096 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.045140 2545 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 07 22:03:38.049096 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.045143 2545 feature_gate.go:328] unrecognized feature gate: Example2 Jul 07 22:03:38.049096 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.045154 2545 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 07 22:03:38.049096 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.045158 2545 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 07 22:03:38.049096 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.045160 2545 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 07 22:03:38.049096 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.045163 2545 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 07 22:03:38.049096 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.045166 2545 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 07 22:03:38.049096 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.045169 2545 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 07 22:03:38.049096 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.045172 2545 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 07 22:03:38.049096 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.045174 2545 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 07 22:03:38.049096 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.045177 2545 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 07 22:03:38.049096 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.045180 2545 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 07 22:03:38.049096 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.045182 2545 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 07 22:03:38.049096 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.045185 2545 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 07 22:03:38.049096 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.045188 2545 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 07 22:03:38.049096 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.045190 2545 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 07 22:03:38.049096 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.045193 2545 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 07 22:03:38.049586 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.045196 2545 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 07 22:03:38.049586 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.045199 2545 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 07 22:03:38.049586 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.045202 2545 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 07 22:03:38.049586 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.045205 2545 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 07 22:03:38.049586 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.045207 2545 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 07 22:03:38.049586 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.045210 2545 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 07 22:03:38.049586 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.045212 2545 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 07 22:03:38.049586 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.045215 2545 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 07 22:03:38.049586 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.045217 2545 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 07 22:03:38.049586 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.045220 2545 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 07 22:03:38.049586 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.045222 2545 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 07 22:03:38.049586 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.045225 2545 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 07 22:03:38.049586 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.045227 2545 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 07 22:03:38.049586 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.045230 2545 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 07 22:03:38.049586 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.045232 2545 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 07 22:03:38.049586 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.045235 2545 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 07 22:03:38.049586 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.045237 2545 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 07 22:03:38.049586 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.045240 2545 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 07 22:03:38.049586 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.045242 2545 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 07 22:03:38.050024 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.045245 2545 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 07 22:03:38.050024 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.045247 2545 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 07 22:03:38.050024 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.045250 2545 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 07 22:03:38.050024 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.045252 2545 feature_gate.go:328] unrecognized feature gate: Example Jul 07 22:03:38.050024 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.045255 2545 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 07 22:03:38.050024 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.045257 2545 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 07 22:03:38.050024 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.045260 2545 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 07 22:03:38.050024 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.045263 2545 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 07 22:03:38.050024 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.045265 2545 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 07 22:03:38.050024 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.045267 2545 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 07 22:03:38.050024 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.045270 2545 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 07 22:03:38.050024 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.045273 2545 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 07 22:03:38.050024 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.045275 2545 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 07 22:03:38.050024 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.045278 2545 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 07 22:03:38.050024 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.045281 2545 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 07 22:03:38.050024 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.045283 2545 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 07 22:03:38.050024 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.045286 2545 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 07 22:03:38.050024 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.045288 2545 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 07 22:03:38.050024 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.045291 2545 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 07 22:03:38.050024 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.045294 2545 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 07 22:03:38.050520 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.045298 2545 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 07 22:03:38.050520 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.045300 2545 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 07 22:03:38.050520 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.045303 2545 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 07 22:03:38.050520 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.045305 2545 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 07 22:03:38.050520 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.045308 2545 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 07 22:03:38.050520 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.045310 2545 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 07 22:03:38.050520 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.045313 2545 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 07 22:03:38.050520 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.045315 2545 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 07 22:03:38.050520 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.045318 2545 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 07 22:03:38.050520 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.045320 2545 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 07 22:03:38.050520 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.045323 2545 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 07 22:03:38.050520 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.045325 2545 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 07 22:03:38.050520 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.045328 2545 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 07 22:03:38.050520 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.045330 2545 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 07 22:03:38.050520 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.045333 2545 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 07 22:03:38.050520 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.045335 2545 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 07 22:03:38.050520 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.045338 2545 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 07 22:03:38.050520 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.045340 2545 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 07 22:03:38.050520 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.045343 2545 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 07 22:03:38.050520 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.045345 2545 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 07 22:03:38.050983 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.045348 2545 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 07 22:03:38.050983 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.045350 2545 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 07 22:03:38.050983 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.045353 2545 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 07 22:03:38.050983 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.045356 2545 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 07 22:03:38.050983 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.045358 2545 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 07 22:03:38.050983 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.045361 2545 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 07 22:03:38.050983 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.045363 2545 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 07 22:03:38.050983 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.045366 2545 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 07 22:03:38.050983 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.045369 2545 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 07 22:03:38.050983 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.045371 2545 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 07 22:03:38.050983 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.045374 2545 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 07 22:03:38.050983 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:38.045376 2545 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 07 22:03:38.050983 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.045381 2545 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 07 22:03:38.050983 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.045487 2545 server.go:962] "Client rotation is on, will bootstrap in background" Jul 07 22:03:38.051339 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.047441 2545 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 07 22:03:38.051339 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.049270 2545 server.go:1019] "Starting client certificate rotation" Jul 07 22:03:38.051339 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.049381 2545 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 07 22:03:38.051339 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.049418 2545 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 07 22:03:38.053380 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.053355 2545 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 07 22:03:38.055349 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.055327 2545 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 07 22:03:38.068102 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.068073 2545 log.go:25] "Validated CRI v1 runtime API" Jul 07 22:03:38.070031 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.070013 2545 log.go:25] "Validated CRI v1 image API" Jul 07 22:03:38.070823 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.070807 2545 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 07 22:03:38.076749 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.076705 2545 fs.go:135] Filesystem UUIDs: map[2e0adf16-787b-4db9-9ea7-0850d9487c46:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 fd887ab6-634f-4b1a-a31d-5cabe0bfb991:/dev/nvme0n1p3] Jul 07 22:03:38.076823 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.076747 2545 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 07 22:03:38.081915 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.081796 2545 manager.go:216] Machine: {Timestamp:2026-07-07 22:03:38.080448175 +0000 UTC m=+0.394671185 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3112040 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec27c2b98993d7067532fddf11fca590 SystemUUID:ec27c2b9-8993-d706-7532-fddf11fca590 BootID:0fef0d97-d90f-493f-ac06-f9dcded69ca8 Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:1b:b8:d3:14:6b Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:1b:b8:d3:14:6b Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ca:1f:1a:c0:fc:47 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 07 22:03:38.081915 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.081904 2545 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 07 22:03:38.082073 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.082003 2545 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 07 22:03:38.083238 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.083221 2545 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 07 22:03:38.083653 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.083631 2545 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 07 22:03:38.083792 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.083657 2545 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-131-55.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 07 22:03:38.084414 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.084404 2545 topology_manager.go:138] "Creating topology manager with none policy" Jul 07 22:03:38.084456 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.084415 2545 container_manager_linux.go:309] "Creating device plugin manager" Jul 07 22:03:38.084456 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.084423 2545 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 07 22:03:38.085068 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.085058 2545 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 07 22:03:38.086643 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.086633 2545 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 07 22:03:38.090005 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.089994 2545 state_mem.go:36] "Initialized new in-memory state store" Jul 07 22:03:38.090142 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.090132 2545 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 07 22:03:38.090173 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.090166 2545 kubelet.go:486] "Attempting to sync node with API server" Jul 07 22:03:38.090210 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.090179 2545 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 07 22:03:38.090210 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.090202 2545 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 07 22:03:38.090274 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.090215 2545 kubelet.go:398] "Adding apiserver pod source" Jul 07 22:03:38.090274 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.090227 2545 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 07 22:03:38.090623 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.090611 2545 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 07 22:03:38.090659 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.090630 2545 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 07 22:03:38.093119 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.093102 2545 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 07 22:03:38.095166 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.095153 2545 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 07 22:03:38.095221 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.095183 2545 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 07 22:03:38.096380 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.096367 2545 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 07 22:03:38.096434 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.096389 2545 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 07 22:03:38.096465 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.096398 2545 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 07 22:03:38.096465 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.096460 2545 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 07 22:03:38.096531 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.096479 2545 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 07 22:03:38.096531 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.096488 2545 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 07 22:03:38.096531 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.096499 2545 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 07 22:03:38.096531 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.096507 2545 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 07 22:03:38.096531 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.096523 2545 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 07 22:03:38.096531 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.096532 2545 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 07 22:03:38.096700 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.096556 2545 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 07 22:03:38.096700 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.096564 2545 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 07 22:03:38.097508 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.097498 2545 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 07 22:03:38.097540 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.097524 2545 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 07 22:03:38.101608 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.101593 2545 server.go:1268] "Started kubelet" Jul 07 22:03:38.101732 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.101684 2545 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 07 22:03:38.101792 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.101722 2545 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 07 22:03:38.101792 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.101755 2545 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 07 22:03:38.102113 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.102073 2545 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-131-55.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 07 22:03:38.102113 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:03:38.102104 2545 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-131-55.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 07 22:03:38.102248 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:03:38.102164 2545 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 07 22:03:38.102636 ip-10-0-131-55 systemd[1]: Started Kubernetes Kubelet. Jul 07 22:03:38.104112 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.104096 2545 server.go:310] "Adding debug handlers to kubelet server" Jul 07 22:03:38.105445 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.105429 2545 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 07 22:03:38.106949 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:03:38.106931 2545 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 07 22:03:38.108167 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:03:38.106510 2545 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-55.ec2.internal.18c021130a6a8749 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-55.ec2.internal,UID:ip-10-0-131-55.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-131-55.ec2.internal,},FirstTimestamp:2026-07-07 22:03:38.101565257 +0000 UTC m=+0.415788267,LastTimestamp:2026-07-07 22:03:38.101565257 +0000 UTC m=+0.415788267,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-55.ec2.internal,}" Jul 07 22:03:38.111720 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.111706 2545 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 07 22:03:38.111763 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.111750 2545 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 07 22:03:38.112163 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:03:38.111816 2545 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-55.ec2.internal\" not found" Jul 07 22:03:38.112504 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.112484 2545 factory.go:55] Registering systemd factory Jul 07 22:03:38.112600 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.112511 2545 factory.go:223] Registration of the systemd container factory successfully Jul 07 22:03:38.112664 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.112649 2545 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 07 22:03:38.112664 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.112652 2545 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 07 22:03:38.112664 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.112661 2545 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 07 22:03:38.112789 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.112777 2545 reconstruct.go:97] "Volume reconstruction finished" Jul 07 22:03:38.112834 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.112796 2545 reconciler.go:29] "Reconciler: start to sync state" Jul 07 22:03:38.114404 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.113370 2545 factory.go:153] Registering CRI-O factory Jul 07 22:03:38.114404 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.113505 2545 factory.go:223] Registration of the crio container factory successfully Jul 07 22:03:38.114404 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.113598 2545 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 07 22:03:38.114404 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.113653 2545 factory.go:103] Registering Raw factory Jul 07 22:03:38.114404 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.113673 2545 manager.go:1196] Started watching for new ooms in manager Jul 07 22:03:38.114404 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.114228 2545 manager.go:349] Starting recovery of all containers Jul 07 22:03:38.125680 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:03:38.125551 2545 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 07 22:03:38.125680 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:03:38.125598 2545 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-131-55.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 07 22:03:38.125916 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.125903 2545 manager.go:354] Recovery completed Jul 07 22:03:38.128405 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.128376 2545 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-v94fr" Jul 07 22:03:38.130212 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.130200 2545 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 22:03:38.132303 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.132288 2545 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-55.ec2.internal" event="NodeHasSufficientMemory" Jul 07 22:03:38.132392 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.132318 2545 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-55.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 22:03:38.132392 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.132334 2545 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-55.ec2.internal" event="NodeHasSufficientPID" Jul 07 22:03:38.132847 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.132832 2545 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 07 22:03:38.132847 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.132845 2545 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 22:03:38.132946 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.132846 2545 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 07 22:03:38.132946 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.132884 2545 state_mem.go:36] "Initialized new in-memory state store" Jul 07 22:03:38.133573 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.133560 2545 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-55.ec2.internal" event="NodeHasSufficientMemory" Jul 07 22:03:38.133648 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.133582 2545 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-55.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 22:03:38.133648 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.133596 2545 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-55.ec2.internal" event="NodeHasSufficientPID" Jul 07 22:03:38.134436 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.134424 2545 policy_none.go:49] "None policy: Start" Jul 07 22:03:38.134483 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.134438 2545 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 07 22:03:38.134483 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.134449 2545 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 07 22:03:38.136284 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.136268 2545 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-v94fr" Jul 07 22:03:38.136359 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.136348 2545 policy_none.go:47] "Start" Jul 07 22:03:38.173647 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.173630 2545 manager.go:337] "Starting Device Plugin manager" Jul 07 22:03:38.173751 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:03:38.173668 2545 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 07 22:03:38.173751 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.173681 2545 server.go:87] "Starting device plugin registration server" Jul 07 22:03:38.173937 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.173924 2545 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 07 22:03:38.173988 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.173938 2545 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 07 22:03:38.174032 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.174003 2545 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 07 22:03:38.174122 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.174109 2545 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 07 22:03:38.174122 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.174122 2545 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 07 22:03:38.174631 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:03:38.174614 2545 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 07 22:03:38.174714 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:03:38.174661 2545 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-131-55.ec2.internal\" not found" Jul 07 22:03:38.274544 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.274523 2545 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 22:03:38.277153 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.277125 2545 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-55.ec2.internal" event="NodeHasSufficientMemory" Jul 07 22:03:38.277259 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.277172 2545 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-55.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 22:03:38.277259 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.277185 2545 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-55.ec2.internal" event="NodeHasSufficientPID" Jul 07 22:03:38.277259 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.277210 2545 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-131-55.ec2.internal" Jul 07 22:03:38.288039 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.288021 2545 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-131-55.ec2.internal" Jul 07 22:03:38.288109 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:03:38.288041 2545 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-131-55.ec2.internal\": node \"ip-10-0-131-55.ec2.internal\" not found" Jul 07 22:03:38.303783 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.303758 2545 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 07 22:03:38.304890 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.304872 2545 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 07 22:03:38.304957 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.304903 2545 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 07 22:03:38.304957 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.304921 2545 kubelet.go:2448] "Starting kubelet main sync loop" Jul 07 22:03:38.304957 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:03:38.304949 2545 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 07 22:03:38.308609 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.308588 2545 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 22:03:38.340240 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:03:38.340220 2545 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-55.ec2.internal\" not found" Jul 07 22:03:38.405248 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.405209 2545 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-55.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-131-55.ec2.internal"] Jul 07 22:03:38.405333 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.405292 2545 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 22:03:38.406864 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.406850 2545 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-55.ec2.internal" event="NodeHasSufficientMemory" Jul 07 22:03:38.406927 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.406881 2545 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-55.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 22:03:38.406927 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.406892 2545 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-55.ec2.internal" event="NodeHasSufficientPID" Jul 07 22:03:38.408331 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.408317 2545 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 22:03:38.408908 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.408879 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-55.ec2.internal" Jul 07 22:03:38.408908 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.408908 2545 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 22:03:38.409095 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.409068 2545 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-55.ec2.internal" event="NodeHasSufficientMemory" Jul 07 22:03:38.409169 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.409117 2545 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-55.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 22:03:38.409169 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.409132 2545 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-55.ec2.internal" event="NodeHasSufficientPID" Jul 07 22:03:38.409583 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.409567 2545 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-55.ec2.internal" event="NodeHasSufficientMemory" Jul 07 22:03:38.409650 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.409597 2545 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-55.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 22:03:38.409650 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.409606 2545 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-55.ec2.internal" event="NodeHasSufficientPID" Jul 07 22:03:38.410492 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.410476 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-55.ec2.internal" Jul 07 22:03:38.410580 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.410507 2545 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 07 22:03:38.411181 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.411165 2545 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-55.ec2.internal" event="NodeHasSufficientMemory" Jul 07 22:03:38.411268 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.411195 2545 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-55.ec2.internal" event="NodeHasNoDiskPressure" Jul 07 22:03:38.411268 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.411208 2545 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-55.ec2.internal" event="NodeHasSufficientPID" Jul 07 22:03:38.413943 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.413922 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/164cedd539cf1aa15ead9eedbab978df-config\") pod \"kube-apiserver-proxy-ip-10-0-131-55.ec2.internal\" (UID: \"164cedd539cf1aa15ead9eedbab978df\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-55.ec2.internal" Jul 07 22:03:38.414043 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.413960 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a6a963ffa8cd0fda182ebe6d33d409ac-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-55.ec2.internal\" (UID: \"a6a963ffa8cd0fda182ebe6d33d409ac\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-55.ec2.internal" Jul 07 22:03:38.414043 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.413994 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a6a963ffa8cd0fda182ebe6d33d409ac-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-55.ec2.internal\" (UID: \"a6a963ffa8cd0fda182ebe6d33d409ac\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-55.ec2.internal" Jul 07 22:03:38.440602 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:03:38.440578 2545 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-55.ec2.internal\" not found" Jul 07 22:03:38.440715 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:03:38.440699 2545 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-55.ec2.internal\" not found" node="ip-10-0-131-55.ec2.internal" Jul 07 22:03:38.445058 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:03:38.445041 2545 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-55.ec2.internal\" not found" node="ip-10-0-131-55.ec2.internal" Jul 07 22:03:38.514883 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.514841 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a6a963ffa8cd0fda182ebe6d33d409ac-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-55.ec2.internal\" (UID: \"a6a963ffa8cd0fda182ebe6d33d409ac\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-55.ec2.internal" Jul 07 22:03:38.514883 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.514885 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a6a963ffa8cd0fda182ebe6d33d409ac-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-55.ec2.internal\" (UID: \"a6a963ffa8cd0fda182ebe6d33d409ac\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-55.ec2.internal" Jul 07 22:03:38.515129 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.514937 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a6a963ffa8cd0fda182ebe6d33d409ac-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-55.ec2.internal\" (UID: \"a6a963ffa8cd0fda182ebe6d33d409ac\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-55.ec2.internal" Jul 07 22:03:38.515129 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.515004 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a6a963ffa8cd0fda182ebe6d33d409ac-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-55.ec2.internal\" (UID: \"a6a963ffa8cd0fda182ebe6d33d409ac\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-55.ec2.internal" Jul 07 22:03:38.515129 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.515040 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/164cedd539cf1aa15ead9eedbab978df-config\") pod \"kube-apiserver-proxy-ip-10-0-131-55.ec2.internal\" (UID: \"164cedd539cf1aa15ead9eedbab978df\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-55.ec2.internal" Jul 07 22:03:38.515129 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.515068 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/164cedd539cf1aa15ead9eedbab978df-config\") pod \"kube-apiserver-proxy-ip-10-0-131-55.ec2.internal\" (UID: \"164cedd539cf1aa15ead9eedbab978df\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-55.ec2.internal" Jul 07 22:03:38.541030 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:03:38.540958 2545 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-55.ec2.internal\" not found" Jul 07 22:03:38.641833 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:03:38.641795 2545 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-55.ec2.internal\" not found" Jul 07 22:03:38.742277 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:03:38.742246 2545 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-55.ec2.internal\" not found" Jul 07 22:03:38.743350 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.743331 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-55.ec2.internal" Jul 07 22:03:38.748675 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.748661 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-55.ec2.internal" Jul 07 22:03:38.843154 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:03:38.843062 2545 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-55.ec2.internal\" not found" Jul 07 22:03:38.943625 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:03:38.943599 2545 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-55.ec2.internal\" not found" Jul 07 22:03:38.989957 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:38.989927 2545 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 22:03:39.044334 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:03:39.044297 2545 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-55.ec2.internal\" not found" Jul 07 22:03:39.048465 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:39.048445 2545 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 07 22:03:39.048619 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:39.048597 2545 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 07 22:03:39.048655 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:39.048606 2545 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 07 22:03:39.064108 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:39.064088 2545 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 22:03:39.140349 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:39.140271 2545 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-06 21:58:38 +0000 UTC" deadline="2028-04-08 10:41:48.31541913 +0000 UTC" Jul 07 22:03:39.140349 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:39.140301 2545 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15372h38m9.175120888s" Jul 07 22:03:39.145418 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:03:39.145399 2545 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-55.ec2.internal\" not found" Jul 07 22:03:39.246019 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:03:39.245993 2545 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-55.ec2.internal\" not found" Jul 07 22:03:39.290897 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:39.290865 2545 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 22:03:39.305448 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:39.305415 2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod164cedd539cf1aa15ead9eedbab978df.slice/crio-c34aaebae86c8c9d615ce475b497a0eac9c7daad59d827fa09c2915fc80dbef5 WatchSource:0}: Error finding container c34aaebae86c8c9d615ce475b497a0eac9c7daad59d827fa09c2915fc80dbef5: Status 404 returned error can't find the container with id c34aaebae86c8c9d615ce475b497a0eac9c7daad59d827fa09c2915fc80dbef5 Jul 07 22:03:39.306014 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:39.305982 2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6a963ffa8cd0fda182ebe6d33d409ac.slice/crio-29d30d31d9baed12af512028e89614beb337324392b843a0e6ce4270ebe91f2d WatchSource:0}: Error finding container 29d30d31d9baed12af512028e89614beb337324392b843a0e6ce4270ebe91f2d: Status 404 returned error can't find the container with id 29d30d31d9baed12af512028e89614beb337324392b843a0e6ce4270ebe91f2d Jul 07 22:03:39.311103 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:39.311073 2545 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 22:03:39.312701 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:39.312683 2545 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-55.ec2.internal" Jul 07 22:03:39.325601 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:39.325585 2545 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 07 22:03:39.326570 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:39.326555 2545 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-55.ec2.internal" Jul 07 22:03:39.335263 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:39.335250 2545 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 07 22:03:40.091424 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.091394 2545 apiserver.go:52] "Watching apiserver" Jul 07 22:03:40.097802 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.097773 2545 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 07 22:03:40.099350 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.099322 2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ovn-kubernetes/ovnkube-node-6h4fg","kube-system/global-pull-secret-syncer-nqfk8","kube-system/kube-apiserver-proxy-ip-10-0-131-55.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bgm75","openshift-image-registry/node-ca-cc72r","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-55.ec2.internal","openshift-multus/multus-additional-cni-plugins-ztkm2","openshift-multus/network-metrics-daemon-88qj9","openshift-network-diagnostics/network-check-target-82xqf","kube-system/konnectivity-agent-ms5bj","openshift-cluster-node-tuning-operator/tuned-j8q2m","openshift-dns/node-resolver-9vvqt","openshift-multus/multus-fg88n","openshift-network-operator/iptables-alerter-xshfx"] Jul 07 22:03:40.101490 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.101470 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-ztkm2" Jul 07 22:03:40.103200 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.102918 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-fg88n" Jul 07 22:03:40.104294 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.104163 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-xshfx" Jul 07 22:03:40.104537 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.104514 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 07 22:03:40.104834 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.104798 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 07 22:03:40.105406 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.104976 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 07 22:03:40.105406 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.105032 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 07 22:03:40.105406 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.104984 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 07 22:03:40.105406 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.105278 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-w5wtv\"" Jul 07 22:03:40.105406 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.105361 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 07 22:03:40.105715 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.105564 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-nvcvm\"" Jul 07 22:03:40.106358 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.105922 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bgm75" Jul 07 22:03:40.106358 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.106172 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 07 22:03:40.107029 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.106753 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 07 22:03:40.107029 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.106860 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 07 22:03:40.107029 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.106938 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-c9wvd\"" Jul 07 22:03:40.108319 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.107565 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-cc72r" Jul 07 22:03:40.112153 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.112127 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-88qj9" Jul 07 22:03:40.112247 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:03:40.112220 2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-88qj9" podUID="f7c3d30c-9d3a-43c7-9a22-92d1fca6d2ea" Jul 07 22:03:40.113127 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.112623 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-ncx4t\"" Jul 07 22:03:40.113127 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.112700 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 07 22:03:40.113127 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.112773 2545 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 07 22:03:40.113127 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.112961 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 07 22:03:40.114182 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.113447 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 07 22:03:40.114182 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.113737 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 07 22:03:40.114182 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.113922 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 07 22:03:40.114378 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.114282 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-82xqf" Jul 07 22:03:40.114378 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:03:40.114354 2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-82xqf" podUID="30024b54-40f6-40b2-9b04-a82f5650ceba" Jul 07 22:03:40.115783 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.115348 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 07 22:03:40.115783 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.115654 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-r4vcl\"" Jul 07 22:03:40.116348 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.116315 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-6h4fg" Jul 07 22:03:40.121344 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.118334 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nqfk8" Jul 07 22:03:40.121344 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:03:40.118433 2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nqfk8" podUID="01e846e0-a94f-446e-8ec7-fdeb51040759" Jul 07 22:03:40.121344 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.119206 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 07 22:03:40.121344 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.119667 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 07 22:03:40.121344 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.119673 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-ms5bj" Jul 07 22:03:40.121344 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.120668 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-mzcd6\"" Jul 07 22:03:40.121344 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.120696 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 07 22:03:40.121344 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.120818 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 07 22:03:40.121344 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.120960 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 07 22:03:40.121344 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.121042 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 07 22:03:40.122256 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.121994 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-647l9\"" Jul 07 22:03:40.122256 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.122137 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 07 22:03:40.122395 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.122345 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 07 22:03:40.122848 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.122539 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-j8q2m" Jul 07 22:03:40.122940 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.122867 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f7c107a3-d0f6-479d-b548-c4c59b612631-multus-daemon-config\") pod \"multus-fg88n\" (UID: \"f7c107a3-d0f6-479d-b548-c4c59b612631\") " pod="openshift-multus/multus-fg88n" Jul 07 22:03:40.123003 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.122936 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sscwl\" (UniqueName: \"kubernetes.io/projected/2f1fe6ee-68ca-4a89-8715-32c449d0b63b-kube-api-access-sscwl\") pod \"iptables-alerter-xshfx\" (UID: \"2f1fe6ee-68ca-4a89-8715-32c449d0b63b\") " pod="openshift-network-operator/iptables-alerter-xshfx" Jul 07 22:03:40.123003 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.122962 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/00baab1c-b8e0-4b5e-bd42-647ba72c5aca-system-cni-dir\") pod \"multus-additional-cni-plugins-ztkm2\" (UID: \"00baab1c-b8e0-4b5e-bd42-647ba72c5aca\") " pod="openshift-multus/multus-additional-cni-plugins-ztkm2" Jul 07 22:03:40.123003 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.122985 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9bd24331-107d-4dcf-a0e2-8221ca7ff5d0-kubelet-dir\") pod \"aws-ebs-csi-driver-node-bgm75\" (UID: \"9bd24331-107d-4dcf-a0e2-8221ca7ff5d0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bgm75" Jul 07 22:03:40.123186 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.123011 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hgb4z\" (UniqueName: \"kubernetes.io/projected/381be182-aba1-4465-a899-8f217bbeacd2-kube-api-access-hgb4z\") pod \"node-ca-cc72r\" (UID: \"381be182-aba1-4465-a899-8f217bbeacd2\") " pod="openshift-image-registry/node-ca-cc72r" Jul 07 22:03:40.123186 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.123049 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f7c3d30c-9d3a-43c7-9a22-92d1fca6d2ea-metrics-certs\") pod \"network-metrics-daemon-88qj9\" (UID: \"f7c3d30c-9d3a-43c7-9a22-92d1fca6d2ea\") " pod="openshift-multus/network-metrics-daemon-88qj9" Jul 07 22:03:40.123186 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.123072 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zt9ts\" (UniqueName: \"kubernetes.io/projected/f7c3d30c-9d3a-43c7-9a22-92d1fca6d2ea-kube-api-access-zt9ts\") pod \"network-metrics-daemon-88qj9\" (UID: \"f7c3d30c-9d3a-43c7-9a22-92d1fca6d2ea\") " pod="openshift-multus/network-metrics-daemon-88qj9" Jul 07 22:03:40.123186 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.123136 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f7c107a3-d0f6-479d-b548-c4c59b612631-system-cni-dir\") pod \"multus-fg88n\" (UID: \"f7c107a3-d0f6-479d-b548-c4c59b612631\") " pod="openshift-multus/multus-fg88n" Jul 07 22:03:40.123186 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.123158 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f7c107a3-d0f6-479d-b548-c4c59b612631-host-run-netns\") pod \"multus-fg88n\" (UID: \"f7c107a3-d0f6-479d-b548-c4c59b612631\") " pod="openshift-multus/multus-fg88n" Jul 07 22:03:40.123413 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.123187 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f7c107a3-d0f6-479d-b548-c4c59b612631-host-var-lib-cni-bin\") pod \"multus-fg88n\" (UID: \"f7c107a3-d0f6-479d-b548-c4c59b612631\") " pod="openshift-multus/multus-fg88n" Jul 07 22:03:40.123413 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.123212 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f7c107a3-d0f6-479d-b548-c4c59b612631-hostroot\") pod \"multus-fg88n\" (UID: \"f7c107a3-d0f6-479d-b548-c4c59b612631\") " pod="openshift-multus/multus-fg88n" Jul 07 22:03:40.123413 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.123269 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2f1fe6ee-68ca-4a89-8715-32c449d0b63b-iptables-alerter-script\") pod \"iptables-alerter-xshfx\" (UID: \"2f1fe6ee-68ca-4a89-8715-32c449d0b63b\") " pod="openshift-network-operator/iptables-alerter-xshfx" Jul 07 22:03:40.123413 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.123306 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/00baab1c-b8e0-4b5e-bd42-647ba72c5aca-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ztkm2\" (UID: \"00baab1c-b8e0-4b5e-bd42-647ba72c5aca\") " pod="openshift-multus/multus-additional-cni-plugins-ztkm2" Jul 07 22:03:40.123413 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.123335 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f7c107a3-d0f6-479d-b548-c4c59b612631-cni-binary-copy\") pod \"multus-fg88n\" (UID: \"f7c107a3-d0f6-479d-b548-c4c59b612631\") " pod="openshift-multus/multus-fg88n" Jul 07 22:03:40.123413 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.123361 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rplgx\" (UniqueName: \"kubernetes.io/projected/f7c107a3-d0f6-479d-b548-c4c59b612631-kube-api-access-rplgx\") pod \"multus-fg88n\" (UID: \"f7c107a3-d0f6-479d-b548-c4c59b612631\") " pod="openshift-multus/multus-fg88n" Jul 07 22:03:40.123659 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.123423 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9bd24331-107d-4dcf-a0e2-8221ca7ff5d0-device-dir\") pod \"aws-ebs-csi-driver-node-bgm75\" (UID: \"9bd24331-107d-4dcf-a0e2-8221ca7ff5d0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bgm75" Jul 07 22:03:40.123659 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.123444 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9bd24331-107d-4dcf-a0e2-8221ca7ff5d0-sys-fs\") pod \"aws-ebs-csi-driver-node-bgm75\" (UID: \"9bd24331-107d-4dcf-a0e2-8221ca7ff5d0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bgm75" Jul 07 22:03:40.123659 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.123468 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f7c107a3-d0f6-479d-b548-c4c59b612631-host-var-lib-cni-multus\") pod \"multus-fg88n\" (UID: \"f7c107a3-d0f6-479d-b548-c4c59b612631\") " pod="openshift-multus/multus-fg88n" Jul 07 22:03:40.123659 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.123489 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2f1fe6ee-68ca-4a89-8715-32c449d0b63b-host-slash\") pod \"iptables-alerter-xshfx\" (UID: \"2f1fe6ee-68ca-4a89-8715-32c449d0b63b\") " pod="openshift-network-operator/iptables-alerter-xshfx" Jul 07 22:03:40.123659 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.123512 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/00baab1c-b8e0-4b5e-bd42-647ba72c5aca-cnibin\") pod \"multus-additional-cni-plugins-ztkm2\" (UID: \"00baab1c-b8e0-4b5e-bd42-647ba72c5aca\") " pod="openshift-multus/multus-additional-cni-plugins-ztkm2" Jul 07 22:03:40.123659 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.123549 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/00baab1c-b8e0-4b5e-bd42-647ba72c5aca-os-release\") pod \"multus-additional-cni-plugins-ztkm2\" (UID: \"00baab1c-b8e0-4b5e-bd42-647ba72c5aca\") " pod="openshift-multus/multus-additional-cni-plugins-ztkm2" Jul 07 22:03:40.123659 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.123585 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/00baab1c-b8e0-4b5e-bd42-647ba72c5aca-cni-binary-copy\") pod \"multus-additional-cni-plugins-ztkm2\" (UID: \"00baab1c-b8e0-4b5e-bd42-647ba72c5aca\") " pod="openshift-multus/multus-additional-cni-plugins-ztkm2" Jul 07 22:03:40.123659 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.123603 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-9vvqt" Jul 07 22:03:40.123659 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.123616 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2f7m7\" (UniqueName: \"kubernetes.io/projected/00baab1c-b8e0-4b5e-bd42-647ba72c5aca-kube-api-access-2f7m7\") pod \"multus-additional-cni-plugins-ztkm2\" (UID: \"00baab1c-b8e0-4b5e-bd42-647ba72c5aca\") " pod="openshift-multus/multus-additional-cni-plugins-ztkm2" Jul 07 22:03:40.123659 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.123642 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vsrvx\" (UniqueName: \"kubernetes.io/projected/9bd24331-107d-4dcf-a0e2-8221ca7ff5d0-kube-api-access-vsrvx\") pod \"aws-ebs-csi-driver-node-bgm75\" (UID: \"9bd24331-107d-4dcf-a0e2-8221ca7ff5d0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bgm75" Jul 07 22:03:40.124139 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.123665 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9bd24331-107d-4dcf-a0e2-8221ca7ff5d0-etc-selinux\") pod \"aws-ebs-csi-driver-node-bgm75\" (UID: \"9bd24331-107d-4dcf-a0e2-8221ca7ff5d0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bgm75" Jul 07 22:03:40.124139 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.123691 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f7c107a3-d0f6-479d-b548-c4c59b612631-cnibin\") pod \"multus-fg88n\" (UID: \"f7c107a3-d0f6-479d-b548-c4c59b612631\") " pod="openshift-multus/multus-fg88n" Jul 07 22:03:40.124139 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.123716 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f7c107a3-d0f6-479d-b548-c4c59b612631-multus-socket-dir-parent\") pod \"multus-fg88n\" (UID: \"f7c107a3-d0f6-479d-b548-c4c59b612631\") " pod="openshift-multus/multus-fg88n" Jul 07 22:03:40.124139 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.123737 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f7c107a3-d0f6-479d-b548-c4c59b612631-host-run-k8s-cni-cncf-io\") pod \"multus-fg88n\" (UID: \"f7c107a3-d0f6-479d-b548-c4c59b612631\") " pod="openshift-multus/multus-fg88n" Jul 07 22:03:40.124139 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.123762 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/00baab1c-b8e0-4b5e-bd42-647ba72c5aca-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ztkm2\" (UID: \"00baab1c-b8e0-4b5e-bd42-647ba72c5aca\") " pod="openshift-multus/multus-additional-cni-plugins-ztkm2" Jul 07 22:03:40.124139 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.123785 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9bd24331-107d-4dcf-a0e2-8221ca7ff5d0-socket-dir\") pod \"aws-ebs-csi-driver-node-bgm75\" (UID: \"9bd24331-107d-4dcf-a0e2-8221ca7ff5d0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bgm75" Jul 07 22:03:40.124139 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.123807 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/381be182-aba1-4465-a899-8f217bbeacd2-host\") pod \"node-ca-cc72r\" (UID: \"381be182-aba1-4465-a899-8f217bbeacd2\") " pod="openshift-image-registry/node-ca-cc72r" Jul 07 22:03:40.124139 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.123829 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/381be182-aba1-4465-a899-8f217bbeacd2-serviceca\") pod \"node-ca-cc72r\" (UID: \"381be182-aba1-4465-a899-8f217bbeacd2\") " pod="openshift-image-registry/node-ca-cc72r" Jul 07 22:03:40.124139 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.123852 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f7c107a3-d0f6-479d-b548-c4c59b612631-multus-cni-dir\") pod \"multus-fg88n\" (UID: \"f7c107a3-d0f6-479d-b548-c4c59b612631\") " pod="openshift-multus/multus-fg88n" Jul 07 22:03:40.124139 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.123873 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f7c107a3-d0f6-479d-b548-c4c59b612631-os-release\") pod \"multus-fg88n\" (UID: \"f7c107a3-d0f6-479d-b548-c4c59b612631\") " pod="openshift-multus/multus-fg88n" Jul 07 22:03:40.124139 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.123932 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f7c107a3-d0f6-479d-b548-c4c59b612631-host-var-lib-kubelet\") pod \"multus-fg88n\" (UID: \"f7c107a3-d0f6-479d-b548-c4c59b612631\") " pod="openshift-multus/multus-fg88n" Jul 07 22:03:40.124139 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.123956 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f7c107a3-d0f6-479d-b548-c4c59b612631-multus-conf-dir\") pod \"multus-fg88n\" (UID: \"f7c107a3-d0f6-479d-b548-c4c59b612631\") " pod="openshift-multus/multus-fg88n" Jul 07 22:03:40.124139 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.123978 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f7c107a3-d0f6-479d-b548-c4c59b612631-host-run-multus-certs\") pod \"multus-fg88n\" (UID: \"f7c107a3-d0f6-479d-b548-c4c59b612631\") " pod="openshift-multus/multus-fg88n" Jul 07 22:03:40.124139 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.124001 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f7c107a3-d0f6-479d-b548-c4c59b612631-etc-kubernetes\") pod \"multus-fg88n\" (UID: \"f7c107a3-d0f6-479d-b548-c4c59b612631\") " pod="openshift-multus/multus-fg88n" Jul 07 22:03:40.124139 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.124027 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/00baab1c-b8e0-4b5e-bd42-647ba72c5aca-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ztkm2\" (UID: \"00baab1c-b8e0-4b5e-bd42-647ba72c5aca\") " pod="openshift-multus/multus-additional-cni-plugins-ztkm2" Jul 07 22:03:40.124828 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.124193 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9bd24331-107d-4dcf-a0e2-8221ca7ff5d0-registration-dir\") pod \"aws-ebs-csi-driver-node-bgm75\" (UID: \"9bd24331-107d-4dcf-a0e2-8221ca7ff5d0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bgm75" Jul 07 22:03:40.124828 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.124243 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4vbkc\" (UniqueName: \"kubernetes.io/projected/30024b54-40f6-40b2-9b04-a82f5650ceba-kube-api-access-4vbkc\") pod \"network-check-target-82xqf\" (UID: \"30024b54-40f6-40b2-9b04-a82f5650ceba\") " pod="openshift-network-diagnostics/network-check-target-82xqf" Jul 07 22:03:40.124828 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.124802 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-x7r65\"" Jul 07 22:03:40.124828 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.124813 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 07 22:03:40.125217 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.125037 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 07 22:03:40.125893 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.125567 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 07 22:03:40.126115 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.125906 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 07 22:03:40.126115 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.126109 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-kspkn\"" Jul 07 22:03:40.126369 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.126351 2545 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 07 22:03:40.147572 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.147262 2545 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-k9wdg" Jul 07 22:03:40.156067 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.156044 2545 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-k9wdg" Jul 07 22:03:40.214418 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.214393 2545 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 07 22:03:40.225031 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.225003 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/00baab1c-b8e0-4b5e-bd42-647ba72c5aca-cnibin\") pod \"multus-additional-cni-plugins-ztkm2\" (UID: \"00baab1c-b8e0-4b5e-bd42-647ba72c5aca\") " pod="openshift-multus/multus-additional-cni-plugins-ztkm2" Jul 07 22:03:40.225199 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.225037 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/00baab1c-b8e0-4b5e-bd42-647ba72c5aca-os-release\") pod \"multus-additional-cni-plugins-ztkm2\" (UID: \"00baab1c-b8e0-4b5e-bd42-647ba72c5aca\") " pod="openshift-multus/multus-additional-cni-plugins-ztkm2" Jul 07 22:03:40.225199 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.225064 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/00baab1c-b8e0-4b5e-bd42-647ba72c5aca-cni-binary-copy\") pod \"multus-additional-cni-plugins-ztkm2\" (UID: \"00baab1c-b8e0-4b5e-bd42-647ba72c5aca\") " pod="openshift-multus/multus-additional-cni-plugins-ztkm2" Jul 07 22:03:40.225199 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.225104 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2f7m7\" (UniqueName: \"kubernetes.io/projected/00baab1c-b8e0-4b5e-bd42-647ba72c5aca-kube-api-access-2f7m7\") pod \"multus-additional-cni-plugins-ztkm2\" (UID: \"00baab1c-b8e0-4b5e-bd42-647ba72c5aca\") " pod="openshift-multus/multus-additional-cni-plugins-ztkm2" Jul 07 22:03:40.225199 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.225111 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/00baab1c-b8e0-4b5e-bd42-647ba72c5aca-cnibin\") pod \"multus-additional-cni-plugins-ztkm2\" (UID: \"00baab1c-b8e0-4b5e-bd42-647ba72c5aca\") " pod="openshift-multus/multus-additional-cni-plugins-ztkm2" Jul 07 22:03:40.225199 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.225133 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/065087ce-e289-40dc-be53-bea81382bf96-sys\") pod \"tuned-j8q2m\" (UID: \"065087ce-e289-40dc-be53-bea81382bf96\") " pod="openshift-cluster-node-tuning-operator/tuned-j8q2m" Jul 07 22:03:40.225199 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.225164 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/065087ce-e289-40dc-be53-bea81382bf96-tmp\") pod \"tuned-j8q2m\" (UID: \"065087ce-e289-40dc-be53-bea81382bf96\") " pod="openshift-cluster-node-tuning-operator/tuned-j8q2m" Jul 07 22:03:40.225199 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.225190 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9bd24331-107d-4dcf-a0e2-8221ca7ff5d0-etc-selinux\") pod \"aws-ebs-csi-driver-node-bgm75\" (UID: \"9bd24331-107d-4dcf-a0e2-8221ca7ff5d0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bgm75" Jul 07 22:03:40.225533 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.225214 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/065087ce-e289-40dc-be53-bea81382bf96-etc-sysctl-d\") pod \"tuned-j8q2m\" (UID: \"065087ce-e289-40dc-be53-bea81382bf96\") " pod="openshift-cluster-node-tuning-operator/tuned-j8q2m" Jul 07 22:03:40.225533 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.225235 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/065087ce-e289-40dc-be53-bea81382bf96-host\") pod \"tuned-j8q2m\" (UID: \"065087ce-e289-40dc-be53-bea81382bf96\") " pod="openshift-cluster-node-tuning-operator/tuned-j8q2m" Jul 07 22:03:40.225533 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.225260 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f7c107a3-d0f6-479d-b548-c4c59b612631-cnibin\") pod \"multus-fg88n\" (UID: \"f7c107a3-d0f6-479d-b548-c4c59b612631\") " pod="openshift-multus/multus-fg88n" Jul 07 22:03:40.225533 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.225282 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9bd24331-107d-4dcf-a0e2-8221ca7ff5d0-socket-dir\") pod \"aws-ebs-csi-driver-node-bgm75\" (UID: \"9bd24331-107d-4dcf-a0e2-8221ca7ff5d0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bgm75" Jul 07 22:03:40.225533 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.225300 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/381be182-aba1-4465-a899-8f217bbeacd2-host\") pod \"node-ca-cc72r\" (UID: \"381be182-aba1-4465-a899-8f217bbeacd2\") " pod="openshift-image-registry/node-ca-cc72r" Jul 07 22:03:40.225533 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.225320 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/381be182-aba1-4465-a899-8f217bbeacd2-serviceca\") pod \"node-ca-cc72r\" (UID: \"381be182-aba1-4465-a899-8f217bbeacd2\") " pod="openshift-image-registry/node-ca-cc72r" Jul 07 22:03:40.225533 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.225345 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hgb4z\" (UniqueName: \"kubernetes.io/projected/381be182-aba1-4465-a899-8f217bbeacd2-kube-api-access-hgb4z\") pod \"node-ca-cc72r\" (UID: \"381be182-aba1-4465-a899-8f217bbeacd2\") " pod="openshift-image-registry/node-ca-cc72r" Jul 07 22:03:40.225533 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.225366 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/04b1516f-c72c-499c-9c45-8eff8b82ffa2-host-cni-bin\") pod \"ovnkube-node-6h4fg\" (UID: \"04b1516f-c72c-499c-9c45-8eff8b82ffa2\") " pod="openshift-ovn-kubernetes/ovnkube-node-6h4fg" Jul 07 22:03:40.225533 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.225396 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/04b1516f-c72c-499c-9c45-8eff8b82ffa2-ovn-node-metrics-cert\") pod \"ovnkube-node-6h4fg\" (UID: \"04b1516f-c72c-499c-9c45-8eff8b82ffa2\") " pod="openshift-ovn-kubernetes/ovnkube-node-6h4fg" Jul 07 22:03:40.225533 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.225419 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f7c107a3-d0f6-479d-b548-c4c59b612631-etc-kubernetes\") pod \"multus-fg88n\" (UID: \"f7c107a3-d0f6-479d-b548-c4c59b612631\") " pod="openshift-multus/multus-fg88n" Jul 07 22:03:40.225533 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.225437 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9bd24331-107d-4dcf-a0e2-8221ca7ff5d0-registration-dir\") pod \"aws-ebs-csi-driver-node-bgm75\" (UID: \"9bd24331-107d-4dcf-a0e2-8221ca7ff5d0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bgm75" Jul 07 22:03:40.225533 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.225471 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/04b1516f-c72c-499c-9c45-8eff8b82ffa2-systemd-units\") pod \"ovnkube-node-6h4fg\" (UID: \"04b1516f-c72c-499c-9c45-8eff8b82ffa2\") " pod="openshift-ovn-kubernetes/ovnkube-node-6h4fg" Jul 07 22:03:40.225533 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.225494 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/04b1516f-c72c-499c-9c45-8eff8b82ffa2-host-run-netns\") pod \"ovnkube-node-6h4fg\" (UID: \"04b1516f-c72c-499c-9c45-8eff8b82ffa2\") " pod="openshift-ovn-kubernetes/ovnkube-node-6h4fg" Jul 07 22:03:40.225533 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.225512 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/04b1516f-c72c-499c-9c45-8eff8b82ffa2-run-openvswitch\") pod \"ovnkube-node-6h4fg\" (UID: \"04b1516f-c72c-499c-9c45-8eff8b82ffa2\") " pod="openshift-ovn-kubernetes/ovnkube-node-6h4fg" Jul 07 22:03:40.225533 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.225527 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7ac39c08-2ce6-4578-8b01-b68587adf750-hosts-file\") pod \"node-resolver-9vvqt\" (UID: \"7ac39c08-2ce6-4578-8b01-b68587adf750\") " pod="openshift-dns/node-resolver-9vvqt" Jul 07 22:03:40.225533 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.225541 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/04b1516f-c72c-499c-9c45-8eff8b82ffa2-host-slash\") pod \"ovnkube-node-6h4fg\" (UID: \"04b1516f-c72c-499c-9c45-8eff8b82ffa2\") " pod="openshift-ovn-kubernetes/ovnkube-node-6h4fg" Jul 07 22:03:40.226312 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.225560 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/04b1516f-c72c-499c-9c45-8eff8b82ffa2-log-socket\") pod \"ovnkube-node-6h4fg\" (UID: \"04b1516f-c72c-499c-9c45-8eff8b82ffa2\") " pod="openshift-ovn-kubernetes/ovnkube-node-6h4fg" Jul 07 22:03:40.226312 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.225583 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/065087ce-e289-40dc-be53-bea81382bf96-etc-sysconfig\") pod \"tuned-j8q2m\" (UID: \"065087ce-e289-40dc-be53-bea81382bf96\") " pod="openshift-cluster-node-tuning-operator/tuned-j8q2m" Jul 07 22:03:40.226312 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.225609 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/00baab1c-b8e0-4b5e-bd42-647ba72c5aca-system-cni-dir\") pod \"multus-additional-cni-plugins-ztkm2\" (UID: \"00baab1c-b8e0-4b5e-bd42-647ba72c5aca\") " pod="openshift-multus/multus-additional-cni-plugins-ztkm2" Jul 07 22:03:40.226312 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.225630 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/065087ce-e289-40dc-be53-bea81382bf96-etc-sysctl-conf\") pod \"tuned-j8q2m\" (UID: \"065087ce-e289-40dc-be53-bea81382bf96\") " pod="openshift-cluster-node-tuning-operator/tuned-j8q2m" Jul 07 22:03:40.226312 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.225655 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/065087ce-e289-40dc-be53-bea81382bf96-var-lib-kubelet\") pod \"tuned-j8q2m\" (UID: \"065087ce-e289-40dc-be53-bea81382bf96\") " pod="openshift-cluster-node-tuning-operator/tuned-j8q2m" Jul 07 22:03:40.226312 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.225678 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f7c107a3-d0f6-479d-b548-c4c59b612631-system-cni-dir\") pod \"multus-fg88n\" (UID: \"f7c107a3-d0f6-479d-b548-c4c59b612631\") " pod="openshift-multus/multus-fg88n" Jul 07 22:03:40.226312 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.225701 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f7c107a3-d0f6-479d-b548-c4c59b612631-host-var-lib-cni-bin\") pod \"multus-fg88n\" (UID: \"f7c107a3-d0f6-479d-b548-c4c59b612631\") " pod="openshift-multus/multus-fg88n" Jul 07 22:03:40.226312 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.225725 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2f1fe6ee-68ca-4a89-8715-32c449d0b63b-iptables-alerter-script\") pod \"iptables-alerter-xshfx\" (UID: \"2f1fe6ee-68ca-4a89-8715-32c449d0b63b\") " pod="openshift-network-operator/iptables-alerter-xshfx" Jul 07 22:03:40.226312 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.225742 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/00baab1c-b8e0-4b5e-bd42-647ba72c5aca-cni-binary-copy\") pod \"multus-additional-cni-plugins-ztkm2\" (UID: \"00baab1c-b8e0-4b5e-bd42-647ba72c5aca\") " pod="openshift-multus/multus-additional-cni-plugins-ztkm2" Jul 07 22:03:40.226312 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.225752 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/00baab1c-b8e0-4b5e-bd42-647ba72c5aca-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ztkm2\" (UID: \"00baab1c-b8e0-4b5e-bd42-647ba72c5aca\") " pod="openshift-multus/multus-additional-cni-plugins-ztkm2" Jul 07 22:03:40.226312 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.225779 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s48xd\" (UniqueName: \"kubernetes.io/projected/04b1516f-c72c-499c-9c45-8eff8b82ffa2-kube-api-access-s48xd\") pod \"ovnkube-node-6h4fg\" (UID: \"04b1516f-c72c-499c-9c45-8eff8b82ffa2\") " pod="openshift-ovn-kubernetes/ovnkube-node-6h4fg" Jul 07 22:03:40.226312 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.225805 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rplgx\" (UniqueName: \"kubernetes.io/projected/f7c107a3-d0f6-479d-b548-c4c59b612631-kube-api-access-rplgx\") pod \"multus-fg88n\" (UID: \"f7c107a3-d0f6-479d-b548-c4c59b612631\") " pod="openshift-multus/multus-fg88n" Jul 07 22:03:40.226312 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.225903 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f7c107a3-d0f6-479d-b548-c4c59b612631-etc-kubernetes\") pod \"multus-fg88n\" (UID: \"f7c107a3-d0f6-479d-b548-c4c59b612631\") " pod="openshift-multus/multus-fg88n" Jul 07 22:03:40.226312 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.225911 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/381be182-aba1-4465-a899-8f217bbeacd2-host\") pod \"node-ca-cc72r\" (UID: \"381be182-aba1-4465-a899-8f217bbeacd2\") " pod="openshift-image-registry/node-ca-cc72r" Jul 07 22:03:40.226312 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.225928 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/00baab1c-b8e0-4b5e-bd42-647ba72c5aca-system-cni-dir\") pod \"multus-additional-cni-plugins-ztkm2\" (UID: \"00baab1c-b8e0-4b5e-bd42-647ba72c5aca\") " pod="openshift-multus/multus-additional-cni-plugins-ztkm2" Jul 07 22:03:40.226312 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.225963 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/9bd24331-107d-4dcf-a0e2-8221ca7ff5d0-registration-dir\") pod \"aws-ebs-csi-driver-node-bgm75\" (UID: \"9bd24331-107d-4dcf-a0e2-8221ca7ff5d0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bgm75" Jul 07 22:03:40.226312 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.225989 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/381be182-aba1-4465-a899-8f217bbeacd2-serviceca\") pod \"node-ca-cc72r\" (UID: \"381be182-aba1-4465-a899-8f217bbeacd2\") " pod="openshift-image-registry/node-ca-cc72r" Jul 07 22:03:40.227089 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.225996 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9bd24331-107d-4dcf-a0e2-8221ca7ff5d0-sys-fs\") pod \"aws-ebs-csi-driver-node-bgm75\" (UID: \"9bd24331-107d-4dcf-a0e2-8221ca7ff5d0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bgm75" Jul 07 22:03:40.227089 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.226030 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/04b1516f-c72c-499c-9c45-8eff8b82ffa2-host-run-ovn-kubernetes\") pod \"ovnkube-node-6h4fg\" (UID: \"04b1516f-c72c-499c-9c45-8eff8b82ffa2\") " pod="openshift-ovn-kubernetes/ovnkube-node-6h4fg" Jul 07 22:03:40.227089 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.226037 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/9bd24331-107d-4dcf-a0e2-8221ca7ff5d0-etc-selinux\") pod \"aws-ebs-csi-driver-node-bgm75\" (UID: \"9bd24331-107d-4dcf-a0e2-8221ca7ff5d0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bgm75" Jul 07 22:03:40.227089 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.226042 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/00baab1c-b8e0-4b5e-bd42-647ba72c5aca-os-release\") pod \"multus-additional-cni-plugins-ztkm2\" (UID: \"00baab1c-b8e0-4b5e-bd42-647ba72c5aca\") " pod="openshift-multus/multus-additional-cni-plugins-ztkm2" Jul 07 22:03:40.227089 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.226063 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lnps5\" (UniqueName: \"kubernetes.io/projected/065087ce-e289-40dc-be53-bea81382bf96-kube-api-access-lnps5\") pod \"tuned-j8q2m\" (UID: \"065087ce-e289-40dc-be53-bea81382bf96\") " pod="openshift-cluster-node-tuning-operator/tuned-j8q2m" Jul 07 22:03:40.227089 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.226108 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xlsv2\" (UniqueName: \"kubernetes.io/projected/7ac39c08-2ce6-4578-8b01-b68587adf750-kube-api-access-xlsv2\") pod \"node-resolver-9vvqt\" (UID: \"7ac39c08-2ce6-4578-8b01-b68587adf750\") " pod="openshift-dns/node-resolver-9vvqt" Jul 07 22:03:40.227089 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.226121 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/9bd24331-107d-4dcf-a0e2-8221ca7ff5d0-sys-fs\") pod \"aws-ebs-csi-driver-node-bgm75\" (UID: \"9bd24331-107d-4dcf-a0e2-8221ca7ff5d0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bgm75" Jul 07 22:03:40.227089 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.226134 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f7c107a3-d0f6-479d-b548-c4c59b612631-cnibin\") pod \"multus-fg88n\" (UID: \"f7c107a3-d0f6-479d-b548-c4c59b612631\") " pod="openshift-multus/multus-fg88n" Jul 07 22:03:40.227089 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.226142 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f7c107a3-d0f6-479d-b548-c4c59b612631-host-var-lib-cni-multus\") pod \"multus-fg88n\" (UID: \"f7c107a3-d0f6-479d-b548-c4c59b612631\") " pod="openshift-multus/multus-fg88n" Jul 07 22:03:40.227089 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.226169 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2f1fe6ee-68ca-4a89-8715-32c449d0b63b-host-slash\") pod \"iptables-alerter-xshfx\" (UID: \"2f1fe6ee-68ca-4a89-8715-32c449d0b63b\") " pod="openshift-network-operator/iptables-alerter-xshfx" Jul 07 22:03:40.227089 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.226201 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f7c107a3-d0f6-479d-b548-c4c59b612631-system-cni-dir\") pod \"multus-fg88n\" (UID: \"f7c107a3-d0f6-479d-b548-c4c59b612631\") " pod="openshift-multus/multus-fg88n" Jul 07 22:03:40.227089 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.226228 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2f1fe6ee-68ca-4a89-8715-32c449d0b63b-host-slash\") pod \"iptables-alerter-xshfx\" (UID: \"2f1fe6ee-68ca-4a89-8715-32c449d0b63b\") " pod="openshift-network-operator/iptables-alerter-xshfx" Jul 07 22:03:40.227089 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.226227 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/9bd24331-107d-4dcf-a0e2-8221ca7ff5d0-socket-dir\") pod \"aws-ebs-csi-driver-node-bgm75\" (UID: \"9bd24331-107d-4dcf-a0e2-8221ca7ff5d0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bgm75" Jul 07 22:03:40.227089 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.226274 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f7c107a3-d0f6-479d-b548-c4c59b612631-host-var-lib-cni-multus\") pod \"multus-fg88n\" (UID: \"f7c107a3-d0f6-479d-b548-c4c59b612631\") " pod="openshift-multus/multus-fg88n" Jul 07 22:03:40.227089 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.226286 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f7c107a3-d0f6-479d-b548-c4c59b612631-host-var-lib-cni-bin\") pod \"multus-fg88n\" (UID: \"f7c107a3-d0f6-479d-b548-c4c59b612631\") " pod="openshift-multus/multus-fg88n" Jul 07 22:03:40.227089 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.226303 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vsrvx\" (UniqueName: \"kubernetes.io/projected/9bd24331-107d-4dcf-a0e2-8221ca7ff5d0-kube-api-access-vsrvx\") pod \"aws-ebs-csi-driver-node-bgm75\" (UID: \"9bd24331-107d-4dcf-a0e2-8221ca7ff5d0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bgm75" Jul 07 22:03:40.227089 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.226340 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/04b1516f-c72c-499c-9c45-8eff8b82ffa2-run-systemd\") pod \"ovnkube-node-6h4fg\" (UID: \"04b1516f-c72c-499c-9c45-8eff8b82ffa2\") " pod="openshift-ovn-kubernetes/ovnkube-node-6h4fg" Jul 07 22:03:40.227810 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.226343 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/00baab1c-b8e0-4b5e-bd42-647ba72c5aca-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ztkm2\" (UID: \"00baab1c-b8e0-4b5e-bd42-647ba72c5aca\") " pod="openshift-multus/multus-additional-cni-plugins-ztkm2" Jul 07 22:03:40.227810 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.226405 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2f1fe6ee-68ca-4a89-8715-32c449d0b63b-iptables-alerter-script\") pod \"iptables-alerter-xshfx\" (UID: \"2f1fe6ee-68ca-4a89-8715-32c449d0b63b\") " pod="openshift-network-operator/iptables-alerter-xshfx" Jul 07 22:03:40.227810 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.226452 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/04b1516f-c72c-499c-9c45-8eff8b82ffa2-ovnkube-script-lib\") pod \"ovnkube-node-6h4fg\" (UID: \"04b1516f-c72c-499c-9c45-8eff8b82ffa2\") " pod="openshift-ovn-kubernetes/ovnkube-node-6h4fg" Jul 07 22:03:40.227810 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.226484 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/35a9b908-7fda-4fb9-80b9-56b119f92de0-agent-certs\") pod \"konnectivity-agent-ms5bj\" (UID: \"35a9b908-7fda-4fb9-80b9-56b119f92de0\") " pod="kube-system/konnectivity-agent-ms5bj" Jul 07 22:03:40.227810 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.226522 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/065087ce-e289-40dc-be53-bea81382bf96-etc-kubernetes\") pod \"tuned-j8q2m\" (UID: \"065087ce-e289-40dc-be53-bea81382bf96\") " pod="openshift-cluster-node-tuning-operator/tuned-j8q2m" Jul 07 22:03:40.227810 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.226562 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f7c107a3-d0f6-479d-b548-c4c59b612631-multus-socket-dir-parent\") pod \"multus-fg88n\" (UID: \"f7c107a3-d0f6-479d-b548-c4c59b612631\") " pod="openshift-multus/multus-fg88n" Jul 07 22:03:40.227810 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.226602 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f7c107a3-d0f6-479d-b548-c4c59b612631-host-run-k8s-cni-cncf-io\") pod \"multus-fg88n\" (UID: \"f7c107a3-d0f6-479d-b548-c4c59b612631\") " pod="openshift-multus/multus-fg88n" Jul 07 22:03:40.227810 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.226663 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f7c107a3-d0f6-479d-b548-c4c59b612631-multus-socket-dir-parent\") pod \"multus-fg88n\" (UID: \"f7c107a3-d0f6-479d-b548-c4c59b612631\") " pod="openshift-multus/multus-fg88n" Jul 07 22:03:40.227810 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.226668 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/00baab1c-b8e0-4b5e-bd42-647ba72c5aca-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ztkm2\" (UID: \"00baab1c-b8e0-4b5e-bd42-647ba72c5aca\") " pod="openshift-multus/multus-additional-cni-plugins-ztkm2" Jul 07 22:03:40.227810 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.226701 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/04b1516f-c72c-499c-9c45-8eff8b82ffa2-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6h4fg\" (UID: \"04b1516f-c72c-499c-9c45-8eff8b82ffa2\") " pod="openshift-ovn-kubernetes/ovnkube-node-6h4fg" Jul 07 22:03:40.227810 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.226671 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f7c107a3-d0f6-479d-b548-c4c59b612631-host-run-k8s-cni-cncf-io\") pod \"multus-fg88n\" (UID: \"f7c107a3-d0f6-479d-b548-c4c59b612631\") " pod="openshift-multus/multus-fg88n" Jul 07 22:03:40.227810 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.226736 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/065087ce-e289-40dc-be53-bea81382bf96-etc-tuned\") pod \"tuned-j8q2m\" (UID: \"065087ce-e289-40dc-be53-bea81382bf96\") " pod="openshift-cluster-node-tuning-operator/tuned-j8q2m" Jul 07 22:03:40.227810 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.226816 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f7c107a3-d0f6-479d-b548-c4c59b612631-multus-cni-dir\") pod \"multus-fg88n\" (UID: \"f7c107a3-d0f6-479d-b548-c4c59b612631\") " pod="openshift-multus/multus-fg88n" Jul 07 22:03:40.227810 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.226845 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f7c107a3-d0f6-479d-b548-c4c59b612631-os-release\") pod \"multus-fg88n\" (UID: \"f7c107a3-d0f6-479d-b548-c4c59b612631\") " pod="openshift-multus/multus-fg88n" Jul 07 22:03:40.227810 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.226870 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f7c107a3-d0f6-479d-b548-c4c59b612631-host-var-lib-kubelet\") pod \"multus-fg88n\" (UID: \"f7c107a3-d0f6-479d-b548-c4c59b612631\") " pod="openshift-multus/multus-fg88n" Jul 07 22:03:40.227810 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.226928 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f7c107a3-d0f6-479d-b548-c4c59b612631-multus-conf-dir\") pod \"multus-fg88n\" (UID: \"f7c107a3-d0f6-479d-b548-c4c59b612631\") " pod="openshift-multus/multus-fg88n" Jul 07 22:03:40.227810 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.226936 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f7c107a3-d0f6-479d-b548-c4c59b612631-multus-cni-dir\") pod \"multus-fg88n\" (UID: \"f7c107a3-d0f6-479d-b548-c4c59b612631\") " pod="openshift-multus/multus-fg88n" Jul 07 22:03:40.228615 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.226960 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f7c107a3-d0f6-479d-b548-c4c59b612631-host-run-multus-certs\") pod \"multus-fg88n\" (UID: \"f7c107a3-d0f6-479d-b548-c4c59b612631\") " pod="openshift-multus/multus-fg88n" Jul 07 22:03:40.228615 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.226976 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f7c107a3-d0f6-479d-b548-c4c59b612631-host-var-lib-kubelet\") pod \"multus-fg88n\" (UID: \"f7c107a3-d0f6-479d-b548-c4c59b612631\") " pod="openshift-multus/multus-fg88n" Jul 07 22:03:40.228615 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.226999 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/00baab1c-b8e0-4b5e-bd42-647ba72c5aca-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ztkm2\" (UID: \"00baab1c-b8e0-4b5e-bd42-647ba72c5aca\") " pod="openshift-multus/multus-additional-cni-plugins-ztkm2" Jul 07 22:03:40.228615 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.227020 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f7c107a3-d0f6-479d-b548-c4c59b612631-multus-conf-dir\") pod \"multus-fg88n\" (UID: \"f7c107a3-d0f6-479d-b548-c4c59b612631\") " pod="openshift-multus/multus-fg88n" Jul 07 22:03:40.228615 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.227029 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/04b1516f-c72c-499c-9c45-8eff8b82ffa2-host-kubelet\") pod \"ovnkube-node-6h4fg\" (UID: \"04b1516f-c72c-499c-9c45-8eff8b82ffa2\") " pod="openshift-ovn-kubernetes/ovnkube-node-6h4fg" Jul 07 22:03:40.228615 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.227070 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/04b1516f-c72c-499c-9c45-8eff8b82ffa2-var-lib-openvswitch\") pod \"ovnkube-node-6h4fg\" (UID: \"04b1516f-c72c-499c-9c45-8eff8b82ffa2\") " pod="openshift-ovn-kubernetes/ovnkube-node-6h4fg" Jul 07 22:03:40.228615 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.227115 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4vbkc\" (UniqueName: \"kubernetes.io/projected/30024b54-40f6-40b2-9b04-a82f5650ceba-kube-api-access-4vbkc\") pod \"network-check-target-82xqf\" (UID: \"30024b54-40f6-40b2-9b04-a82f5650ceba\") " pod="openshift-network-diagnostics/network-check-target-82xqf" Jul 07 22:03:40.228615 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.227141 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/04b1516f-c72c-499c-9c45-8eff8b82ffa2-etc-openvswitch\") pod \"ovnkube-node-6h4fg\" (UID: \"04b1516f-c72c-499c-9c45-8eff8b82ffa2\") " pod="openshift-ovn-kubernetes/ovnkube-node-6h4fg" Jul 07 22:03:40.228615 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.227072 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f7c107a3-d0f6-479d-b548-c4c59b612631-os-release\") pod \"multus-fg88n\" (UID: \"f7c107a3-d0f6-479d-b548-c4c59b612631\") " pod="openshift-multus/multus-fg88n" Jul 07 22:03:40.228615 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.227187 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/04b1516f-c72c-499c-9c45-8eff8b82ffa2-run-ovn\") pod \"ovnkube-node-6h4fg\" (UID: \"04b1516f-c72c-499c-9c45-8eff8b82ffa2\") " pod="openshift-ovn-kubernetes/ovnkube-node-6h4fg" Jul 07 22:03:40.228615 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.227212 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/04b1516f-c72c-499c-9c45-8eff8b82ffa2-host-cni-netd\") pod \"ovnkube-node-6h4fg\" (UID: \"04b1516f-c72c-499c-9c45-8eff8b82ffa2\") " pod="openshift-ovn-kubernetes/ovnkube-node-6h4fg" Jul 07 22:03:40.228615 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.227235 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/04b1516f-c72c-499c-9c45-8eff8b82ffa2-env-overrides\") pod \"ovnkube-node-6h4fg\" (UID: \"04b1516f-c72c-499c-9c45-8eff8b82ffa2\") " pod="openshift-ovn-kubernetes/ovnkube-node-6h4fg" Jul 07 22:03:40.228615 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.227249 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/00baab1c-b8e0-4b5e-bd42-647ba72c5aca-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ztkm2\" (UID: \"00baab1c-b8e0-4b5e-bd42-647ba72c5aca\") " pod="openshift-multus/multus-additional-cni-plugins-ztkm2" Jul 07 22:03:40.228615 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.227258 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/01e846e0-a94f-446e-8ec7-fdeb51040759-original-pull-secret\") pod \"global-pull-secret-syncer-nqfk8\" (UID: \"01e846e0-a94f-446e-8ec7-fdeb51040759\") " pod="kube-system/global-pull-secret-syncer-nqfk8" Jul 07 22:03:40.228615 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.227284 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f7c107a3-d0f6-479d-b548-c4c59b612631-multus-daemon-config\") pod \"multus-fg88n\" (UID: \"f7c107a3-d0f6-479d-b548-c4c59b612631\") " pod="openshift-multus/multus-fg88n" Jul 07 22:03:40.228615 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.227332 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sscwl\" (UniqueName: \"kubernetes.io/projected/2f1fe6ee-68ca-4a89-8715-32c449d0b63b-kube-api-access-sscwl\") pod \"iptables-alerter-xshfx\" (UID: \"2f1fe6ee-68ca-4a89-8715-32c449d0b63b\") " pod="openshift-network-operator/iptables-alerter-xshfx" Jul 07 22:03:40.228615 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.227361 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9bd24331-107d-4dcf-a0e2-8221ca7ff5d0-kubelet-dir\") pod \"aws-ebs-csi-driver-node-bgm75\" (UID: \"9bd24331-107d-4dcf-a0e2-8221ca7ff5d0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bgm75" Jul 07 22:03:40.229370 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.227387 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/04b1516f-c72c-499c-9c45-8eff8b82ffa2-node-log\") pod \"ovnkube-node-6h4fg\" (UID: \"04b1516f-c72c-499c-9c45-8eff8b82ffa2\") " pod="openshift-ovn-kubernetes/ovnkube-node-6h4fg" Jul 07 22:03:40.229370 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.227419 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/01e846e0-a94f-446e-8ec7-fdeb51040759-kubelet-config\") pod \"global-pull-secret-syncer-nqfk8\" (UID: \"01e846e0-a94f-446e-8ec7-fdeb51040759\") " pod="kube-system/global-pull-secret-syncer-nqfk8" Jul 07 22:03:40.229370 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.227442 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/01e846e0-a94f-446e-8ec7-fdeb51040759-dbus\") pod \"global-pull-secret-syncer-nqfk8\" (UID: \"01e846e0-a94f-446e-8ec7-fdeb51040759\") " pod="kube-system/global-pull-secret-syncer-nqfk8" Jul 07 22:03:40.229370 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.227467 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/35a9b908-7fda-4fb9-80b9-56b119f92de0-konnectivity-ca\") pod \"konnectivity-agent-ms5bj\" (UID: \"35a9b908-7fda-4fb9-80b9-56b119f92de0\") " pod="kube-system/konnectivity-agent-ms5bj" Jul 07 22:03:40.229370 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.227496 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f7c3d30c-9d3a-43c7-9a22-92d1fca6d2ea-metrics-certs\") pod \"network-metrics-daemon-88qj9\" (UID: \"f7c3d30c-9d3a-43c7-9a22-92d1fca6d2ea\") " pod="openshift-multus/network-metrics-daemon-88qj9" Jul 07 22:03:40.229370 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.227523 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/00baab1c-b8e0-4b5e-bd42-647ba72c5aca-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ztkm2\" (UID: \"00baab1c-b8e0-4b5e-bd42-647ba72c5aca\") " pod="openshift-multus/multus-additional-cni-plugins-ztkm2" Jul 07 22:03:40.229370 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.227539 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zt9ts\" (UniqueName: \"kubernetes.io/projected/f7c3d30c-9d3a-43c7-9a22-92d1fca6d2ea-kube-api-access-zt9ts\") pod \"network-metrics-daemon-88qj9\" (UID: \"f7c3d30c-9d3a-43c7-9a22-92d1fca6d2ea\") " pod="openshift-multus/network-metrics-daemon-88qj9" Jul 07 22:03:40.229370 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.227564 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f7c107a3-d0f6-479d-b548-c4c59b612631-host-run-netns\") pod \"multus-fg88n\" (UID: \"f7c107a3-d0f6-479d-b548-c4c59b612631\") " pod="openshift-multus/multus-fg88n" Jul 07 22:03:40.229370 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.227731 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/9bd24331-107d-4dcf-a0e2-8221ca7ff5d0-kubelet-dir\") pod \"aws-ebs-csi-driver-node-bgm75\" (UID: \"9bd24331-107d-4dcf-a0e2-8221ca7ff5d0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bgm75" Jul 07 22:03:40.229370 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.227759 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f7c107a3-d0f6-479d-b548-c4c59b612631-host-run-multus-certs\") pod \"multus-fg88n\" (UID: \"f7c107a3-d0f6-479d-b548-c4c59b612631\") " pod="openshift-multus/multus-fg88n" Jul 07 22:03:40.229370 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.227771 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f7c107a3-d0f6-479d-b548-c4c59b612631-hostroot\") pod \"multus-fg88n\" (UID: \"f7c107a3-d0f6-479d-b548-c4c59b612631\") " pod="openshift-multus/multus-fg88n" Jul 07 22:03:40.229370 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.227788 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f7c107a3-d0f6-479d-b548-c4c59b612631-multus-daemon-config\") pod \"multus-fg88n\" (UID: \"f7c107a3-d0f6-479d-b548-c4c59b612631\") " pod="openshift-multus/multus-fg88n" Jul 07 22:03:40.229370 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.227801 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/04b1516f-c72c-499c-9c45-8eff8b82ffa2-ovnkube-config\") pod \"ovnkube-node-6h4fg\" (UID: \"04b1516f-c72c-499c-9c45-8eff8b82ffa2\") " pod="openshift-ovn-kubernetes/ovnkube-node-6h4fg" Jul 07 22:03:40.229370 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.227826 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/065087ce-e289-40dc-be53-bea81382bf96-etc-modprobe-d\") pod \"tuned-j8q2m\" (UID: \"065087ce-e289-40dc-be53-bea81382bf96\") " pod="openshift-cluster-node-tuning-operator/tuned-j8q2m" Jul 07 22:03:40.229370 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.227839 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f7c107a3-d0f6-479d-b548-c4c59b612631-hostroot\") pod \"multus-fg88n\" (UID: \"f7c107a3-d0f6-479d-b548-c4c59b612631\") " pod="openshift-multus/multus-fg88n" Jul 07 22:03:40.229370 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:03:40.227848 2545 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 22:03:40.229370 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.227852 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/065087ce-e289-40dc-be53-bea81382bf96-lib-modules\") pod \"tuned-j8q2m\" (UID: \"065087ce-e289-40dc-be53-bea81382bf96\") " pod="openshift-cluster-node-tuning-operator/tuned-j8q2m" Jul 07 22:03:40.229370 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.227877 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7ac39c08-2ce6-4578-8b01-b68587adf750-tmp-dir\") pod \"node-resolver-9vvqt\" (UID: \"7ac39c08-2ce6-4578-8b01-b68587adf750\") " pod="openshift-dns/node-resolver-9vvqt" Jul 07 22:03:40.230061 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.227897 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f7c107a3-d0f6-479d-b548-c4c59b612631-host-run-netns\") pod \"multus-fg88n\" (UID: \"f7c107a3-d0f6-479d-b548-c4c59b612631\") " pod="openshift-multus/multus-fg88n" Jul 07 22:03:40.230061 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:03:40.227929 2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f7c3d30c-9d3a-43c7-9a22-92d1fca6d2ea-metrics-certs podName:f7c3d30c-9d3a-43c7-9a22-92d1fca6d2ea nodeName:}" failed. No retries permitted until 2026-07-07 22:03:40.727894567 +0000 UTC m=+3.042117581 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f7c3d30c-9d3a-43c7-9a22-92d1fca6d2ea-metrics-certs") pod "network-metrics-daemon-88qj9" (UID: "f7c3d30c-9d3a-43c7-9a22-92d1fca6d2ea") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 22:03:40.230061 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.227947 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f7c107a3-d0f6-479d-b548-c4c59b612631-cni-binary-copy\") pod \"multus-fg88n\" (UID: \"f7c107a3-d0f6-479d-b548-c4c59b612631\") " pod="openshift-multus/multus-fg88n" Jul 07 22:03:40.230061 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.227973 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9bd24331-107d-4dcf-a0e2-8221ca7ff5d0-device-dir\") pod \"aws-ebs-csi-driver-node-bgm75\" (UID: \"9bd24331-107d-4dcf-a0e2-8221ca7ff5d0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bgm75" Jul 07 22:03:40.230061 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.227999 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/065087ce-e289-40dc-be53-bea81382bf96-etc-systemd\") pod \"tuned-j8q2m\" (UID: \"065087ce-e289-40dc-be53-bea81382bf96\") " pod="openshift-cluster-node-tuning-operator/tuned-j8q2m" Jul 07 22:03:40.230061 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.228043 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/9bd24331-107d-4dcf-a0e2-8221ca7ff5d0-device-dir\") pod \"aws-ebs-csi-driver-node-bgm75\" (UID: \"9bd24331-107d-4dcf-a0e2-8221ca7ff5d0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bgm75" Jul 07 22:03:40.230061 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.228094 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/065087ce-e289-40dc-be53-bea81382bf96-run\") pod \"tuned-j8q2m\" (UID: \"065087ce-e289-40dc-be53-bea81382bf96\") " pod="openshift-cluster-node-tuning-operator/tuned-j8q2m" Jul 07 22:03:40.230061 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.228456 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f7c107a3-d0f6-479d-b548-c4c59b612631-cni-binary-copy\") pod \"multus-fg88n\" (UID: \"f7c107a3-d0f6-479d-b548-c4c59b612631\") " pod="openshift-multus/multus-fg88n" Jul 07 22:03:40.238994 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.238965 2545 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 07 22:03:40.241895 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:03:40.241768 2545 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 22:03:40.241895 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:03:40.241791 2545 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 22:03:40.241895 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:03:40.241805 2545 projected.go:196] Error preparing data for projected volume kube-api-access-4vbkc for pod openshift-network-diagnostics/network-check-target-82xqf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 22:03:40.241895 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:03:40.241870 2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/30024b54-40f6-40b2-9b04-a82f5650ceba-kube-api-access-4vbkc podName:30024b54-40f6-40b2-9b04-a82f5650ceba nodeName:}" failed. No retries permitted until 2026-07-07 22:03:40.741853 +0000 UTC m=+3.056076009 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-4vbkc" (UniqueName: "kubernetes.io/projected/30024b54-40f6-40b2-9b04-a82f5650ceba-kube-api-access-4vbkc") pod "network-check-target-82xqf" (UID: "30024b54-40f6-40b2-9b04-a82f5650ceba") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 22:03:40.243908 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.243874 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vsrvx\" (UniqueName: \"kubernetes.io/projected/9bd24331-107d-4dcf-a0e2-8221ca7ff5d0-kube-api-access-vsrvx\") pod \"aws-ebs-csi-driver-node-bgm75\" (UID: \"9bd24331-107d-4dcf-a0e2-8221ca7ff5d0\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bgm75" Jul 07 22:03:40.245126 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.244656 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2f7m7\" (UniqueName: \"kubernetes.io/projected/00baab1c-b8e0-4b5e-bd42-647ba72c5aca-kube-api-access-2f7m7\") pod \"multus-additional-cni-plugins-ztkm2\" (UID: \"00baab1c-b8e0-4b5e-bd42-647ba72c5aca\") " pod="openshift-multus/multus-additional-cni-plugins-ztkm2" Jul 07 22:03:40.245126 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.245026 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rplgx\" (UniqueName: \"kubernetes.io/projected/f7c107a3-d0f6-479d-b548-c4c59b612631-kube-api-access-rplgx\") pod \"multus-fg88n\" (UID: \"f7c107a3-d0f6-479d-b548-c4c59b612631\") " pod="openshift-multus/multus-fg88n" Jul 07 22:03:40.245126 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.245031 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hgb4z\" (UniqueName: \"kubernetes.io/projected/381be182-aba1-4465-a899-8f217bbeacd2-kube-api-access-hgb4z\") pod \"node-ca-cc72r\" (UID: \"381be182-aba1-4465-a899-8f217bbeacd2\") " pod="openshift-image-registry/node-ca-cc72r" Jul 07 22:03:40.245565 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.245542 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zt9ts\" (UniqueName: \"kubernetes.io/projected/f7c3d30c-9d3a-43c7-9a22-92d1fca6d2ea-kube-api-access-zt9ts\") pod \"network-metrics-daemon-88qj9\" (UID: \"f7c3d30c-9d3a-43c7-9a22-92d1fca6d2ea\") " pod="openshift-multus/network-metrics-daemon-88qj9" Jul 07 22:03:40.246450 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.246427 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sscwl\" (UniqueName: \"kubernetes.io/projected/2f1fe6ee-68ca-4a89-8715-32c449d0b63b-kube-api-access-sscwl\") pod \"iptables-alerter-xshfx\" (UID: \"2f1fe6ee-68ca-4a89-8715-32c449d0b63b\") " pod="openshift-network-operator/iptables-alerter-xshfx" Jul 07 22:03:40.269770 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.269739 2545 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 22:03:40.309561 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.309513 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-55.ec2.internal" event={"ID":"a6a963ffa8cd0fda182ebe6d33d409ac","Type":"ContainerStarted","Data":"29d30d31d9baed12af512028e89614beb337324392b843a0e6ce4270ebe91f2d"} Jul 07 22:03:40.310542 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.310516 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-55.ec2.internal" event={"ID":"164cedd539cf1aa15ead9eedbab978df","Type":"ContainerStarted","Data":"c34aaebae86c8c9d615ce475b497a0eac9c7daad59d827fa09c2915fc80dbef5"} Jul 07 22:03:40.329239 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.329211 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s48xd\" (UniqueName: \"kubernetes.io/projected/04b1516f-c72c-499c-9c45-8eff8b82ffa2-kube-api-access-s48xd\") pod \"ovnkube-node-6h4fg\" (UID: \"04b1516f-c72c-499c-9c45-8eff8b82ffa2\") " pod="openshift-ovn-kubernetes/ovnkube-node-6h4fg" Jul 07 22:03:40.329387 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.329244 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/04b1516f-c72c-499c-9c45-8eff8b82ffa2-host-run-ovn-kubernetes\") pod \"ovnkube-node-6h4fg\" (UID: \"04b1516f-c72c-499c-9c45-8eff8b82ffa2\") " pod="openshift-ovn-kubernetes/ovnkube-node-6h4fg" Jul 07 22:03:40.329387 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.329266 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lnps5\" (UniqueName: \"kubernetes.io/projected/065087ce-e289-40dc-be53-bea81382bf96-kube-api-access-lnps5\") pod \"tuned-j8q2m\" (UID: \"065087ce-e289-40dc-be53-bea81382bf96\") " pod="openshift-cluster-node-tuning-operator/tuned-j8q2m" Jul 07 22:03:40.329387 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.329281 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xlsv2\" (UniqueName: \"kubernetes.io/projected/7ac39c08-2ce6-4578-8b01-b68587adf750-kube-api-access-xlsv2\") pod \"node-resolver-9vvqt\" (UID: \"7ac39c08-2ce6-4578-8b01-b68587adf750\") " pod="openshift-dns/node-resolver-9vvqt" Jul 07 22:03:40.329387 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.329304 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/04b1516f-c72c-499c-9c45-8eff8b82ffa2-run-systemd\") pod \"ovnkube-node-6h4fg\" (UID: \"04b1516f-c72c-499c-9c45-8eff8b82ffa2\") " pod="openshift-ovn-kubernetes/ovnkube-node-6h4fg" Jul 07 22:03:40.329387 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.329327 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/04b1516f-c72c-499c-9c45-8eff8b82ffa2-ovnkube-script-lib\") pod \"ovnkube-node-6h4fg\" (UID: \"04b1516f-c72c-499c-9c45-8eff8b82ffa2\") " pod="openshift-ovn-kubernetes/ovnkube-node-6h4fg" Jul 07 22:03:40.329387 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.329334 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/04b1516f-c72c-499c-9c45-8eff8b82ffa2-host-run-ovn-kubernetes\") pod \"ovnkube-node-6h4fg\" (UID: \"04b1516f-c72c-499c-9c45-8eff8b82ffa2\") " pod="openshift-ovn-kubernetes/ovnkube-node-6h4fg" Jul 07 22:03:40.329691 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.329529 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/35a9b908-7fda-4fb9-80b9-56b119f92de0-agent-certs\") pod \"konnectivity-agent-ms5bj\" (UID: \"35a9b908-7fda-4fb9-80b9-56b119f92de0\") " pod="kube-system/konnectivity-agent-ms5bj" Jul 07 22:03:40.329691 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.329562 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/065087ce-e289-40dc-be53-bea81382bf96-etc-kubernetes\") pod \"tuned-j8q2m\" (UID: \"065087ce-e289-40dc-be53-bea81382bf96\") " pod="openshift-cluster-node-tuning-operator/tuned-j8q2m" Jul 07 22:03:40.329691 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.329592 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/04b1516f-c72c-499c-9c45-8eff8b82ffa2-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6h4fg\" (UID: \"04b1516f-c72c-499c-9c45-8eff8b82ffa2\") " pod="openshift-ovn-kubernetes/ovnkube-node-6h4fg" Jul 07 22:03:40.329691 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.329601 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/04b1516f-c72c-499c-9c45-8eff8b82ffa2-run-systemd\") pod \"ovnkube-node-6h4fg\" (UID: \"04b1516f-c72c-499c-9c45-8eff8b82ffa2\") " pod="openshift-ovn-kubernetes/ovnkube-node-6h4fg" Jul 07 22:03:40.329691 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.329618 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/065087ce-e289-40dc-be53-bea81382bf96-etc-tuned\") pod \"tuned-j8q2m\" (UID: \"065087ce-e289-40dc-be53-bea81382bf96\") " pod="openshift-cluster-node-tuning-operator/tuned-j8q2m" Jul 07 22:03:40.329691 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.329647 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/04b1516f-c72c-499c-9c45-8eff8b82ffa2-host-kubelet\") pod \"ovnkube-node-6h4fg\" (UID: \"04b1516f-c72c-499c-9c45-8eff8b82ffa2\") " pod="openshift-ovn-kubernetes/ovnkube-node-6h4fg" Jul 07 22:03:40.329691 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.329660 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/065087ce-e289-40dc-be53-bea81382bf96-etc-kubernetes\") pod \"tuned-j8q2m\" (UID: \"065087ce-e289-40dc-be53-bea81382bf96\") " pod="openshift-cluster-node-tuning-operator/tuned-j8q2m" Jul 07 22:03:40.329691 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.329671 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/04b1516f-c72c-499c-9c45-8eff8b82ffa2-var-lib-openvswitch\") pod \"ovnkube-node-6h4fg\" (UID: \"04b1516f-c72c-499c-9c45-8eff8b82ffa2\") " pod="openshift-ovn-kubernetes/ovnkube-node-6h4fg" Jul 07 22:03:40.330100 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.329708 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/04b1516f-c72c-499c-9c45-8eff8b82ffa2-etc-openvswitch\") pod \"ovnkube-node-6h4fg\" (UID: \"04b1516f-c72c-499c-9c45-8eff8b82ffa2\") " pod="openshift-ovn-kubernetes/ovnkube-node-6h4fg" Jul 07 22:03:40.330100 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.329731 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/04b1516f-c72c-499c-9c45-8eff8b82ffa2-run-ovn\") pod \"ovnkube-node-6h4fg\" (UID: \"04b1516f-c72c-499c-9c45-8eff8b82ffa2\") " pod="openshift-ovn-kubernetes/ovnkube-node-6h4fg" Jul 07 22:03:40.330100 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.329751 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/04b1516f-c72c-499c-9c45-8eff8b82ffa2-host-cni-netd\") pod \"ovnkube-node-6h4fg\" (UID: \"04b1516f-c72c-499c-9c45-8eff8b82ffa2\") " pod="openshift-ovn-kubernetes/ovnkube-node-6h4fg" Jul 07 22:03:40.330100 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.329774 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/04b1516f-c72c-499c-9c45-8eff8b82ffa2-env-overrides\") pod \"ovnkube-node-6h4fg\" (UID: \"04b1516f-c72c-499c-9c45-8eff8b82ffa2\") " pod="openshift-ovn-kubernetes/ovnkube-node-6h4fg" Jul 07 22:03:40.330100 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.329796 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/01e846e0-a94f-446e-8ec7-fdeb51040759-original-pull-secret\") pod \"global-pull-secret-syncer-nqfk8\" (UID: \"01e846e0-a94f-446e-8ec7-fdeb51040759\") " pod="kube-system/global-pull-secret-syncer-nqfk8" Jul 07 22:03:40.330100 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.329823 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/04b1516f-c72c-499c-9c45-8eff8b82ffa2-node-log\") pod \"ovnkube-node-6h4fg\" (UID: \"04b1516f-c72c-499c-9c45-8eff8b82ffa2\") " pod="openshift-ovn-kubernetes/ovnkube-node-6h4fg" Jul 07 22:03:40.330100 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.329846 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/01e846e0-a94f-446e-8ec7-fdeb51040759-kubelet-config\") pod \"global-pull-secret-syncer-nqfk8\" (UID: \"01e846e0-a94f-446e-8ec7-fdeb51040759\") " pod="kube-system/global-pull-secret-syncer-nqfk8" Jul 07 22:03:40.330100 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.329869 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/01e846e0-a94f-446e-8ec7-fdeb51040759-dbus\") pod \"global-pull-secret-syncer-nqfk8\" (UID: \"01e846e0-a94f-446e-8ec7-fdeb51040759\") " pod="kube-system/global-pull-secret-syncer-nqfk8" Jul 07 22:03:40.330100 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.329892 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/35a9b908-7fda-4fb9-80b9-56b119f92de0-konnectivity-ca\") pod \"konnectivity-agent-ms5bj\" (UID: \"35a9b908-7fda-4fb9-80b9-56b119f92de0\") " pod="kube-system/konnectivity-agent-ms5bj" Jul 07 22:03:40.330100 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.329904 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/04b1516f-c72c-499c-9c45-8eff8b82ffa2-ovnkube-script-lib\") pod \"ovnkube-node-6h4fg\" (UID: \"04b1516f-c72c-499c-9c45-8eff8b82ffa2\") " pod="openshift-ovn-kubernetes/ovnkube-node-6h4fg" Jul 07 22:03:40.330100 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.329972 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/04b1516f-c72c-499c-9c45-8eff8b82ffa2-etc-openvswitch\") pod \"ovnkube-node-6h4fg\" (UID: \"04b1516f-c72c-499c-9c45-8eff8b82ffa2\") " pod="openshift-ovn-kubernetes/ovnkube-node-6h4fg" Jul 07 22:03:40.330100 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.329972 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/04b1516f-c72c-499c-9c45-8eff8b82ffa2-var-lib-openvswitch\") pod \"ovnkube-node-6h4fg\" (UID: \"04b1516f-c72c-499c-9c45-8eff8b82ffa2\") " pod="openshift-ovn-kubernetes/ovnkube-node-6h4fg" Jul 07 22:03:40.330100 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.330007 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/04b1516f-c72c-499c-9c45-8eff8b82ffa2-host-kubelet\") pod \"ovnkube-node-6h4fg\" (UID: \"04b1516f-c72c-499c-9c45-8eff8b82ffa2\") " pod="openshift-ovn-kubernetes/ovnkube-node-6h4fg" Jul 07 22:03:40.330100 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.330027 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/04b1516f-c72c-499c-9c45-8eff8b82ffa2-run-ovn\") pod \"ovnkube-node-6h4fg\" (UID: \"04b1516f-c72c-499c-9c45-8eff8b82ffa2\") " pod="openshift-ovn-kubernetes/ovnkube-node-6h4fg" Jul 07 22:03:40.330100 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.330050 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/04b1516f-c72c-499c-9c45-8eff8b82ffa2-host-cni-netd\") pod \"ovnkube-node-6h4fg\" (UID: \"04b1516f-c72c-499c-9c45-8eff8b82ffa2\") " pod="openshift-ovn-kubernetes/ovnkube-node-6h4fg" Jul 07 22:03:40.330100 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.330053 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/01e846e0-a94f-446e-8ec7-fdeb51040759-dbus\") pod \"global-pull-secret-syncer-nqfk8\" (UID: \"01e846e0-a94f-446e-8ec7-fdeb51040759\") " pod="kube-system/global-pull-secret-syncer-nqfk8" Jul 07 22:03:40.330806 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.330117 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/04b1516f-c72c-499c-9c45-8eff8b82ffa2-ovnkube-config\") pod \"ovnkube-node-6h4fg\" (UID: \"04b1516f-c72c-499c-9c45-8eff8b82ffa2\") " pod="openshift-ovn-kubernetes/ovnkube-node-6h4fg" Jul 07 22:03:40.330806 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.330141 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/065087ce-e289-40dc-be53-bea81382bf96-etc-modprobe-d\") pod \"tuned-j8q2m\" (UID: \"065087ce-e289-40dc-be53-bea81382bf96\") " pod="openshift-cluster-node-tuning-operator/tuned-j8q2m" Jul 07 22:03:40.330806 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:03:40.330143 2545 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 22:03:40.330806 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.330163 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/065087ce-e289-40dc-be53-bea81382bf96-lib-modules\") pod \"tuned-j8q2m\" (UID: \"065087ce-e289-40dc-be53-bea81382bf96\") " pod="openshift-cluster-node-tuning-operator/tuned-j8q2m" Jul 07 22:03:40.330806 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:03:40.330211 2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/01e846e0-a94f-446e-8ec7-fdeb51040759-original-pull-secret podName:01e846e0-a94f-446e-8ec7-fdeb51040759 nodeName:}" failed. No retries permitted until 2026-07-07 22:03:40.830192004 +0000 UTC m=+3.144415014 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/01e846e0-a94f-446e-8ec7-fdeb51040759-original-pull-secret") pod "global-pull-secret-syncer-nqfk8" (UID: "01e846e0-a94f-446e-8ec7-fdeb51040759") : object "kube-system"/"original-pull-secret" not registered Jul 07 22:03:40.330806 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.330228 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7ac39c08-2ce6-4578-8b01-b68587adf750-tmp-dir\") pod \"node-resolver-9vvqt\" (UID: \"7ac39c08-2ce6-4578-8b01-b68587adf750\") " pod="openshift-dns/node-resolver-9vvqt" Jul 07 22:03:40.330806 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.330254 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/065087ce-e289-40dc-be53-bea81382bf96-etc-systemd\") pod \"tuned-j8q2m\" (UID: \"065087ce-e289-40dc-be53-bea81382bf96\") " pod="openshift-cluster-node-tuning-operator/tuned-j8q2m" Jul 07 22:03:40.330806 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.330264 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/01e846e0-a94f-446e-8ec7-fdeb51040759-kubelet-config\") pod \"global-pull-secret-syncer-nqfk8\" (UID: \"01e846e0-a94f-446e-8ec7-fdeb51040759\") " pod="kube-system/global-pull-secret-syncer-nqfk8" Jul 07 22:03:40.330806 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.330278 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/065087ce-e289-40dc-be53-bea81382bf96-run\") pod \"tuned-j8q2m\" (UID: \"065087ce-e289-40dc-be53-bea81382bf96\") " pod="openshift-cluster-node-tuning-operator/tuned-j8q2m" Jul 07 22:03:40.330806 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.330307 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/065087ce-e289-40dc-be53-bea81382bf96-sys\") pod \"tuned-j8q2m\" (UID: \"065087ce-e289-40dc-be53-bea81382bf96\") " pod="openshift-cluster-node-tuning-operator/tuned-j8q2m" Jul 07 22:03:40.330806 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.330308 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/04b1516f-c72c-499c-9c45-8eff8b82ffa2-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6h4fg\" (UID: \"04b1516f-c72c-499c-9c45-8eff8b82ffa2\") " pod="openshift-ovn-kubernetes/ovnkube-node-6h4fg" Jul 07 22:03:40.330806 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.330328 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/065087ce-e289-40dc-be53-bea81382bf96-tmp\") pod \"tuned-j8q2m\" (UID: \"065087ce-e289-40dc-be53-bea81382bf96\") " pod="openshift-cluster-node-tuning-operator/tuned-j8q2m" Jul 07 22:03:40.330806 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.330361 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/065087ce-e289-40dc-be53-bea81382bf96-etc-sysctl-d\") pod \"tuned-j8q2m\" (UID: \"065087ce-e289-40dc-be53-bea81382bf96\") " pod="openshift-cluster-node-tuning-operator/tuned-j8q2m" Jul 07 22:03:40.330806 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.330401 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/065087ce-e289-40dc-be53-bea81382bf96-lib-modules\") pod \"tuned-j8q2m\" (UID: \"065087ce-e289-40dc-be53-bea81382bf96\") " pod="openshift-cluster-node-tuning-operator/tuned-j8q2m" Jul 07 22:03:40.330806 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.330403 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/065087ce-e289-40dc-be53-bea81382bf96-host\") pod \"tuned-j8q2m\" (UID: \"065087ce-e289-40dc-be53-bea81382bf96\") " pod="openshift-cluster-node-tuning-operator/tuned-j8q2m" Jul 07 22:03:40.330806 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.330445 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/04b1516f-c72c-499c-9c45-8eff8b82ffa2-host-cni-bin\") pod \"ovnkube-node-6h4fg\" (UID: \"04b1516f-c72c-499c-9c45-8eff8b82ffa2\") " pod="openshift-ovn-kubernetes/ovnkube-node-6h4fg" Jul 07 22:03:40.330806 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.330474 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/04b1516f-c72c-499c-9c45-8eff8b82ffa2-ovn-node-metrics-cert\") pod \"ovnkube-node-6h4fg\" (UID: \"04b1516f-c72c-499c-9c45-8eff8b82ffa2\") " pod="openshift-ovn-kubernetes/ovnkube-node-6h4fg" Jul 07 22:03:40.331426 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.330503 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/04b1516f-c72c-499c-9c45-8eff8b82ffa2-systemd-units\") pod \"ovnkube-node-6h4fg\" (UID: \"04b1516f-c72c-499c-9c45-8eff8b82ffa2\") " pod="openshift-ovn-kubernetes/ovnkube-node-6h4fg" Jul 07 22:03:40.331426 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.330527 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/04b1516f-c72c-499c-9c45-8eff8b82ffa2-host-run-netns\") pod \"ovnkube-node-6h4fg\" (UID: \"04b1516f-c72c-499c-9c45-8eff8b82ffa2\") " pod="openshift-ovn-kubernetes/ovnkube-node-6h4fg" Jul 07 22:03:40.331426 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.330537 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/35a9b908-7fda-4fb9-80b9-56b119f92de0-konnectivity-ca\") pod \"konnectivity-agent-ms5bj\" (UID: \"35a9b908-7fda-4fb9-80b9-56b119f92de0\") " pod="kube-system/konnectivity-agent-ms5bj" Jul 07 22:03:40.331426 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.330551 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/04b1516f-c72c-499c-9c45-8eff8b82ffa2-run-openvswitch\") pod \"ovnkube-node-6h4fg\" (UID: \"04b1516f-c72c-499c-9c45-8eff8b82ffa2\") " pod="openshift-ovn-kubernetes/ovnkube-node-6h4fg" Jul 07 22:03:40.331426 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.330579 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7ac39c08-2ce6-4578-8b01-b68587adf750-hosts-file\") pod \"node-resolver-9vvqt\" (UID: \"7ac39c08-2ce6-4578-8b01-b68587adf750\") " pod="openshift-dns/node-resolver-9vvqt" Jul 07 22:03:40.331426 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.330588 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/04b1516f-c72c-499c-9c45-8eff8b82ffa2-node-log\") pod \"ovnkube-node-6h4fg\" (UID: \"04b1516f-c72c-499c-9c45-8eff8b82ffa2\") " pod="openshift-ovn-kubernetes/ovnkube-node-6h4fg" Jul 07 22:03:40.331426 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.330602 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/04b1516f-c72c-499c-9c45-8eff8b82ffa2-host-slash\") pod \"ovnkube-node-6h4fg\" (UID: \"04b1516f-c72c-499c-9c45-8eff8b82ffa2\") " pod="openshift-ovn-kubernetes/ovnkube-node-6h4fg" Jul 07 22:03:40.331426 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.330627 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/04b1516f-c72c-499c-9c45-8eff8b82ffa2-log-socket\") pod \"ovnkube-node-6h4fg\" (UID: \"04b1516f-c72c-499c-9c45-8eff8b82ffa2\") " pod="openshift-ovn-kubernetes/ovnkube-node-6h4fg" Jul 07 22:03:40.331426 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.330650 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/065087ce-e289-40dc-be53-bea81382bf96-etc-sysconfig\") pod \"tuned-j8q2m\" (UID: \"065087ce-e289-40dc-be53-bea81382bf96\") " pod="openshift-cluster-node-tuning-operator/tuned-j8q2m" Jul 07 22:03:40.331426 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.330674 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/065087ce-e289-40dc-be53-bea81382bf96-etc-sysctl-conf\") pod \"tuned-j8q2m\" (UID: \"065087ce-e289-40dc-be53-bea81382bf96\") " pod="openshift-cluster-node-tuning-operator/tuned-j8q2m" Jul 07 22:03:40.331426 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.330689 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7ac39c08-2ce6-4578-8b01-b68587adf750-tmp-dir\") pod \"node-resolver-9vvqt\" (UID: \"7ac39c08-2ce6-4578-8b01-b68587adf750\") " pod="openshift-dns/node-resolver-9vvqt" Jul 07 22:03:40.331426 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.330698 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/065087ce-e289-40dc-be53-bea81382bf96-var-lib-kubelet\") pod \"tuned-j8q2m\" (UID: \"065087ce-e289-40dc-be53-bea81382bf96\") " pod="openshift-cluster-node-tuning-operator/tuned-j8q2m" Jul 07 22:03:40.331426 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.330783 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/065087ce-e289-40dc-be53-bea81382bf96-etc-systemd\") pod \"tuned-j8q2m\" (UID: \"065087ce-e289-40dc-be53-bea81382bf96\") " pod="openshift-cluster-node-tuning-operator/tuned-j8q2m" Jul 07 22:03:40.331426 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.330786 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/04b1516f-c72c-499c-9c45-8eff8b82ffa2-env-overrides\") pod \"ovnkube-node-6h4fg\" (UID: \"04b1516f-c72c-499c-9c45-8eff8b82ffa2\") " pod="openshift-ovn-kubernetes/ovnkube-node-6h4fg" Jul 07 22:03:40.331426 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.330829 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/065087ce-e289-40dc-be53-bea81382bf96-var-lib-kubelet\") pod \"tuned-j8q2m\" (UID: \"065087ce-e289-40dc-be53-bea81382bf96\") " pod="openshift-cluster-node-tuning-operator/tuned-j8q2m" Jul 07 22:03:40.331426 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.330870 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/065087ce-e289-40dc-be53-bea81382bf96-run\") pod \"tuned-j8q2m\" (UID: \"065087ce-e289-40dc-be53-bea81382bf96\") " pod="openshift-cluster-node-tuning-operator/tuned-j8q2m" Jul 07 22:03:40.331426 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.330925 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/065087ce-e289-40dc-be53-bea81382bf96-sys\") pod \"tuned-j8q2m\" (UID: \"065087ce-e289-40dc-be53-bea81382bf96\") " pod="openshift-cluster-node-tuning-operator/tuned-j8q2m" Jul 07 22:03:40.331426 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.331189 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/04b1516f-c72c-499c-9c45-8eff8b82ffa2-ovnkube-config\") pod \"ovnkube-node-6h4fg\" (UID: \"04b1516f-c72c-499c-9c45-8eff8b82ffa2\") " pod="openshift-ovn-kubernetes/ovnkube-node-6h4fg" Jul 07 22:03:40.332279 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.331218 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/04b1516f-c72c-499c-9c45-8eff8b82ffa2-host-run-netns\") pod \"ovnkube-node-6h4fg\" (UID: \"04b1516f-c72c-499c-9c45-8eff8b82ffa2\") " pod="openshift-ovn-kubernetes/ovnkube-node-6h4fg" Jul 07 22:03:40.332279 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.331240 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/04b1516f-c72c-499c-9c45-8eff8b82ffa2-run-openvswitch\") pod \"ovnkube-node-6h4fg\" (UID: \"04b1516f-c72c-499c-9c45-8eff8b82ffa2\") " pod="openshift-ovn-kubernetes/ovnkube-node-6h4fg" Jul 07 22:03:40.332279 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.331239 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/065087ce-e289-40dc-be53-bea81382bf96-etc-sysctl-d\") pod \"tuned-j8q2m\" (UID: \"065087ce-e289-40dc-be53-bea81382bf96\") " pod="openshift-cluster-node-tuning-operator/tuned-j8q2m" Jul 07 22:03:40.332279 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.330447 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/065087ce-e289-40dc-be53-bea81382bf96-host\") pod \"tuned-j8q2m\" (UID: \"065087ce-e289-40dc-be53-bea81382bf96\") " pod="openshift-cluster-node-tuning-operator/tuned-j8q2m" Jul 07 22:03:40.332279 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.331284 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7ac39c08-2ce6-4578-8b01-b68587adf750-hosts-file\") pod \"node-resolver-9vvqt\" (UID: \"7ac39c08-2ce6-4578-8b01-b68587adf750\") " pod="openshift-dns/node-resolver-9vvqt" Jul 07 22:03:40.332279 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.331308 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/065087ce-e289-40dc-be53-bea81382bf96-etc-modprobe-d\") pod \"tuned-j8q2m\" (UID: \"065087ce-e289-40dc-be53-bea81382bf96\") " pod="openshift-cluster-node-tuning-operator/tuned-j8q2m" Jul 07 22:03:40.332279 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.331313 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/04b1516f-c72c-499c-9c45-8eff8b82ffa2-host-cni-bin\") pod \"ovnkube-node-6h4fg\" (UID: \"04b1516f-c72c-499c-9c45-8eff8b82ffa2\") " pod="openshift-ovn-kubernetes/ovnkube-node-6h4fg" Jul 07 22:03:40.332279 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.331324 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/04b1516f-c72c-499c-9c45-8eff8b82ffa2-log-socket\") pod \"ovnkube-node-6h4fg\" (UID: \"04b1516f-c72c-499c-9c45-8eff8b82ffa2\") " pod="openshift-ovn-kubernetes/ovnkube-node-6h4fg" Jul 07 22:03:40.332279 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.331350 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/04b1516f-c72c-499c-9c45-8eff8b82ffa2-systemd-units\") pod \"ovnkube-node-6h4fg\" (UID: \"04b1516f-c72c-499c-9c45-8eff8b82ffa2\") " pod="openshift-ovn-kubernetes/ovnkube-node-6h4fg" Jul 07 22:03:40.332279 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.331367 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/065087ce-e289-40dc-be53-bea81382bf96-etc-sysconfig\") pod \"tuned-j8q2m\" (UID: \"065087ce-e289-40dc-be53-bea81382bf96\") " pod="openshift-cluster-node-tuning-operator/tuned-j8q2m" Jul 07 22:03:40.332279 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.331386 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/04b1516f-c72c-499c-9c45-8eff8b82ffa2-host-slash\") pod \"ovnkube-node-6h4fg\" (UID: \"04b1516f-c72c-499c-9c45-8eff8b82ffa2\") " pod="openshift-ovn-kubernetes/ovnkube-node-6h4fg" Jul 07 22:03:40.332279 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.331445 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/065087ce-e289-40dc-be53-bea81382bf96-etc-sysctl-conf\") pod \"tuned-j8q2m\" (UID: \"065087ce-e289-40dc-be53-bea81382bf96\") " pod="openshift-cluster-node-tuning-operator/tuned-j8q2m" Jul 07 22:03:40.332841 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.332544 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/35a9b908-7fda-4fb9-80b9-56b119f92de0-agent-certs\") pod \"konnectivity-agent-ms5bj\" (UID: \"35a9b908-7fda-4fb9-80b9-56b119f92de0\") " pod="kube-system/konnectivity-agent-ms5bj" Jul 07 22:03:40.333015 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.332994 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/065087ce-e289-40dc-be53-bea81382bf96-etc-tuned\") pod \"tuned-j8q2m\" (UID: \"065087ce-e289-40dc-be53-bea81382bf96\") " pod="openshift-cluster-node-tuning-operator/tuned-j8q2m" Jul 07 22:03:40.333429 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.333408 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/065087ce-e289-40dc-be53-bea81382bf96-tmp\") pod \"tuned-j8q2m\" (UID: \"065087ce-e289-40dc-be53-bea81382bf96\") " pod="openshift-cluster-node-tuning-operator/tuned-j8q2m" Jul 07 22:03:40.333618 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.333602 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/04b1516f-c72c-499c-9c45-8eff8b82ffa2-ovn-node-metrics-cert\") pod \"ovnkube-node-6h4fg\" (UID: \"04b1516f-c72c-499c-9c45-8eff8b82ffa2\") " pod="openshift-ovn-kubernetes/ovnkube-node-6h4fg" Jul 07 22:03:40.344733 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.344646 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lnps5\" (UniqueName: \"kubernetes.io/projected/065087ce-e289-40dc-be53-bea81382bf96-kube-api-access-lnps5\") pod \"tuned-j8q2m\" (UID: \"065087ce-e289-40dc-be53-bea81382bf96\") " pod="openshift-cluster-node-tuning-operator/tuned-j8q2m" Jul 07 22:03:40.345375 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.345346 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s48xd\" (UniqueName: \"kubernetes.io/projected/04b1516f-c72c-499c-9c45-8eff8b82ffa2-kube-api-access-s48xd\") pod \"ovnkube-node-6h4fg\" (UID: \"04b1516f-c72c-499c-9c45-8eff8b82ffa2\") " pod="openshift-ovn-kubernetes/ovnkube-node-6h4fg" Jul 07 22:03:40.345481 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.345449 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xlsv2\" (UniqueName: \"kubernetes.io/projected/7ac39c08-2ce6-4578-8b01-b68587adf750-kube-api-access-xlsv2\") pod \"node-resolver-9vvqt\" (UID: \"7ac39c08-2ce6-4578-8b01-b68587adf750\") " pod="openshift-dns/node-resolver-9vvqt" Jul 07 22:03:40.421865 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.421830 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-ztkm2" Jul 07 22:03:40.430660 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.430637 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-fg88n" Jul 07 22:03:40.443278 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.443253 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-xshfx" Jul 07 22:03:40.449132 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.449112 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bgm75" Jul 07 22:03:40.456809 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.456788 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-cc72r" Jul 07 22:03:40.463804 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.463784 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-6h4fg" Jul 07 22:03:40.471494 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.471475 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-ms5bj" Jul 07 22:03:40.479155 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.479131 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-j8q2m" Jul 07 22:03:40.484747 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.484729 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-9vvqt" Jul 07 22:03:40.733669 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.733639 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f7c3d30c-9d3a-43c7-9a22-92d1fca6d2ea-metrics-certs\") pod \"network-metrics-daemon-88qj9\" (UID: \"f7c3d30c-9d3a-43c7-9a22-92d1fca6d2ea\") " pod="openshift-multus/network-metrics-daemon-88qj9" Jul 07 22:03:40.733827 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:03:40.733753 2545 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 22:03:40.733827 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:03:40.733805 2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f7c3d30c-9d3a-43c7-9a22-92d1fca6d2ea-metrics-certs podName:f7c3d30c-9d3a-43c7-9a22-92d1fca6d2ea nodeName:}" failed. No retries permitted until 2026-07-07 22:03:41.733791769 +0000 UTC m=+4.048014766 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f7c3d30c-9d3a-43c7-9a22-92d1fca6d2ea-metrics-certs") pod "network-metrics-daemon-88qj9" (UID: "f7c3d30c-9d3a-43c7-9a22-92d1fca6d2ea") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 22:03:40.834268 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.834224 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4vbkc\" (UniqueName: \"kubernetes.io/projected/30024b54-40f6-40b2-9b04-a82f5650ceba-kube-api-access-4vbkc\") pod \"network-check-target-82xqf\" (UID: \"30024b54-40f6-40b2-9b04-a82f5650ceba\") " pod="openshift-network-diagnostics/network-check-target-82xqf" Jul 07 22:03:40.834268 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:40.834269 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/01e846e0-a94f-446e-8ec7-fdeb51040759-original-pull-secret\") pod \"global-pull-secret-syncer-nqfk8\" (UID: \"01e846e0-a94f-446e-8ec7-fdeb51040759\") " pod="kube-system/global-pull-secret-syncer-nqfk8" Jul 07 22:03:40.834493 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:03:40.834379 2545 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 22:03:40.834493 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:03:40.834440 2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/01e846e0-a94f-446e-8ec7-fdeb51040759-original-pull-secret podName:01e846e0-a94f-446e-8ec7-fdeb51040759 nodeName:}" failed. No retries permitted until 2026-07-07 22:03:41.83442114 +0000 UTC m=+4.148644140 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/01e846e0-a94f-446e-8ec7-fdeb51040759-original-pull-secret") pod "global-pull-secret-syncer-nqfk8" (UID: "01e846e0-a94f-446e-8ec7-fdeb51040759") : object "kube-system"/"original-pull-secret" not registered Jul 07 22:03:40.834742 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:03:40.834719 2545 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 22:03:40.834742 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:03:40.834747 2545 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 22:03:40.834949 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:03:40.834758 2545 projected.go:196] Error preparing data for projected volume kube-api-access-4vbkc for pod openshift-network-diagnostics/network-check-target-82xqf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 22:03:40.834949 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:03:40.834813 2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/30024b54-40f6-40b2-9b04-a82f5650ceba-kube-api-access-4vbkc podName:30024b54-40f6-40b2-9b04-a82f5650ceba nodeName:}" failed. No retries permitted until 2026-07-07 22:03:41.834796552 +0000 UTC m=+4.149019561 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-4vbkc" (UniqueName: "kubernetes.io/projected/30024b54-40f6-40b2-9b04-a82f5650ceba-kube-api-access-4vbkc") pod "network-check-target-82xqf" (UID: "30024b54-40f6-40b2-9b04-a82f5650ceba") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 22:03:41.073676 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:41.073644 2545 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 07 22:03:41.080860 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:41.080695 2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04b1516f_c72c_499c_9c45_8eff8b82ffa2.slice/crio-6e07fc0bcb8e392337865e0a637547b803c33a825d1c5ec7a5260d15955dc879 WatchSource:0}: Error finding container 6e07fc0bcb8e392337865e0a637547b803c33a825d1c5ec7a5260d15955dc879: Status 404 returned error can't find the container with id 6e07fc0bcb8e392337865e0a637547b803c33a825d1c5ec7a5260d15955dc879 Jul 07 22:03:41.082384 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:41.082359 2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00baab1c_b8e0_4b5e_bd42_647ba72c5aca.slice/crio-ab66f1e50ad426fc99201a04910931dc182cdd7e79174f011f36a56762d338c3 WatchSource:0}: Error finding container ab66f1e50ad426fc99201a04910931dc182cdd7e79174f011f36a56762d338c3: Status 404 returned error can't find the container with id ab66f1e50ad426fc99201a04910931dc182cdd7e79174f011f36a56762d338c3 Jul 07 22:03:41.082994 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:41.082971 2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9bd24331_107d_4dcf_a0e2_8221ca7ff5d0.slice/crio-1a834dd371f1fbe2045e658ea2fc52599d4ca484455993e4926eeff064daef2b WatchSource:0}: Error finding container 1a834dd371f1fbe2045e658ea2fc52599d4ca484455993e4926eeff064daef2b: Status 404 returned error can't find the container with id 1a834dd371f1fbe2045e658ea2fc52599d4ca484455993e4926eeff064daef2b Jul 07 22:03:41.084353 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:41.084307 2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod381be182_aba1_4465_a899_8f217bbeacd2.slice/crio-0fc2e3cd662ab849f79bc5cff2174fb5f7dab42bbac4ca266c40cfb53946420e WatchSource:0}: Error finding container 0fc2e3cd662ab849f79bc5cff2174fb5f7dab42bbac4ca266c40cfb53946420e: Status 404 returned error can't find the container with id 0fc2e3cd662ab849f79bc5cff2174fb5f7dab42bbac4ca266c40cfb53946420e Jul 07 22:03:41.088407 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:41.088384 2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7c107a3_d0f6_479d_b548_c4c59b612631.slice/crio-892a484de47029022458c3124eb8afbe88977d42cd35c148b3dfa12dcdece9d2 WatchSource:0}: Error finding container 892a484de47029022458c3124eb8afbe88977d42cd35c148b3dfa12dcdece9d2: Status 404 returned error can't find the container with id 892a484de47029022458c3124eb8afbe88977d42cd35c148b3dfa12dcdece9d2 Jul 07 22:03:41.091206 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:41.091187 2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ac39c08_2ce6_4578_8b01_b68587adf750.slice/crio-7492f4eeced9a19c5025938ce2159ac1a16f869ecd43825916b8c1f888a87e15 WatchSource:0}: Error finding container 7492f4eeced9a19c5025938ce2159ac1a16f869ecd43825916b8c1f888a87e15: Status 404 returned error can't find the container with id 7492f4eeced9a19c5025938ce2159ac1a16f869ecd43825916b8c1f888a87e15 Jul 07 22:03:41.091947 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:41.091930 2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35a9b908_7fda_4fb9_80b9_56b119f92de0.slice/crio-543022e967fba31267d199da23495d293c783ae75b570733576c1893cbf9165b WatchSource:0}: Error finding container 543022e967fba31267d199da23495d293c783ae75b570733576c1893cbf9165b: Status 404 returned error can't find the container with id 543022e967fba31267d199da23495d293c783ae75b570733576c1893cbf9165b Jul 07 22:03:41.092839 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:41.092818 2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f1fe6ee_68ca_4a89_8715_32c449d0b63b.slice/crio-b442c1a64c9a912fd3420d62b90067ee5a4fe11430ddc6402b5d2a332aff82c6 WatchSource:0}: Error finding container b442c1a64c9a912fd3420d62b90067ee5a4fe11430ddc6402b5d2a332aff82c6: Status 404 returned error can't find the container with id b442c1a64c9a912fd3420d62b90067ee5a4fe11430ddc6402b5d2a332aff82c6 Jul 07 22:03:41.094394 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:03:41.094135 2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod065087ce_e289_40dc_be53_bea81382bf96.slice/crio-10eda0123ceea108291d883f1f181e58e9160b4520afd07716716988ffcf728c WatchSource:0}: Error finding container 10eda0123ceea108291d883f1f181e58e9160b4520afd07716716988ffcf728c: Status 404 returned error can't find the container with id 10eda0123ceea108291d883f1f181e58e9160b4520afd07716716988ffcf728c Jul 07 22:03:41.158108 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:41.158048 2545 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-06 21:58:40 +0000 UTC" deadline="2028-02-19 02:42:28.977954824 +0000 UTC" Jul 07 22:03:41.158108 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:41.158108 2545 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14188h38m47.819850776s" Jul 07 22:03:41.305548 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:41.305461 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-88qj9" Jul 07 22:03:41.305658 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:03:41.305566 2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-88qj9" podUID="f7c3d30c-9d3a-43c7-9a22-92d1fca6d2ea" Jul 07 22:03:41.312967 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:41.312935 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-9vvqt" event={"ID":"7ac39c08-2ce6-4578-8b01-b68587adf750","Type":"ContainerStarted","Data":"7492f4eeced9a19c5025938ce2159ac1a16f869ecd43825916b8c1f888a87e15"} Jul 07 22:03:41.314227 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:41.314199 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-cc72r" event={"ID":"381be182-aba1-4465-a899-8f217bbeacd2","Type":"ContainerStarted","Data":"0fc2e3cd662ab849f79bc5cff2174fb5f7dab42bbac4ca266c40cfb53946420e"} Jul 07 22:03:41.315293 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:41.315270 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bgm75" event={"ID":"9bd24331-107d-4dcf-a0e2-8221ca7ff5d0","Type":"ContainerStarted","Data":"1a834dd371f1fbe2045e658ea2fc52599d4ca484455993e4926eeff064daef2b"} Jul 07 22:03:41.317064 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:41.317042 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-55.ec2.internal" event={"ID":"164cedd539cf1aa15ead9eedbab978df","Type":"ContainerStarted","Data":"2adabf675250c9ee8ce99b1fc1ea6db91bf1d0145055321c9b2086433ebb55d1"} Jul 07 22:03:41.318162 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:41.318143 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-ms5bj" event={"ID":"35a9b908-7fda-4fb9-80b9-56b119f92de0","Type":"ContainerStarted","Data":"543022e967fba31267d199da23495d293c783ae75b570733576c1893cbf9165b"} Jul 07 22:03:41.319172 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:41.319150 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-fg88n" event={"ID":"f7c107a3-d0f6-479d-b548-c4c59b612631","Type":"ContainerStarted","Data":"892a484de47029022458c3124eb8afbe88977d42cd35c148b3dfa12dcdece9d2"} Jul 07 22:03:41.320222 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:41.320199 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ztkm2" event={"ID":"00baab1c-b8e0-4b5e-bd42-647ba72c5aca","Type":"ContainerStarted","Data":"ab66f1e50ad426fc99201a04910931dc182cdd7e79174f011f36a56762d338c3"} Jul 07 22:03:41.321421 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:41.321394 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6h4fg" event={"ID":"04b1516f-c72c-499c-9c45-8eff8b82ffa2","Type":"ContainerStarted","Data":"6e07fc0bcb8e392337865e0a637547b803c33a825d1c5ec7a5260d15955dc879"} Jul 07 22:03:41.322441 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:41.322420 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-xshfx" event={"ID":"2f1fe6ee-68ca-4a89-8715-32c449d0b63b","Type":"ContainerStarted","Data":"b442c1a64c9a912fd3420d62b90067ee5a4fe11430ddc6402b5d2a332aff82c6"} Jul 07 22:03:41.323327 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:41.323306 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-j8q2m" event={"ID":"065087ce-e289-40dc-be53-bea81382bf96","Type":"ContainerStarted","Data":"10eda0123ceea108291d883f1f181e58e9160b4520afd07716716988ffcf728c"} Jul 07 22:03:41.330873 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:41.330835 2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-55.ec2.internal" podStartSLOduration=2.33082399 podStartE2EDuration="2.33082399s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 22:03:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:03:41.33046229 +0000 UTC m=+3.644685311" watchObservedRunningTime="2026-07-07 22:03:41.33082399 +0000 UTC m=+3.645047034" Jul 07 22:03:41.741321 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:41.741284 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f7c3d30c-9d3a-43c7-9a22-92d1fca6d2ea-metrics-certs\") pod \"network-metrics-daemon-88qj9\" (UID: \"f7c3d30c-9d3a-43c7-9a22-92d1fca6d2ea\") " pod="openshift-multus/network-metrics-daemon-88qj9" Jul 07 22:03:41.741506 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:03:41.741446 2545 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 22:03:41.741570 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:03:41.741514 2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f7c3d30c-9d3a-43c7-9a22-92d1fca6d2ea-metrics-certs podName:f7c3d30c-9d3a-43c7-9a22-92d1fca6d2ea nodeName:}" failed. No retries permitted until 2026-07-07 22:03:43.741496296 +0000 UTC m=+6.055719308 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f7c3d30c-9d3a-43c7-9a22-92d1fca6d2ea-metrics-certs") pod "network-metrics-daemon-88qj9" (UID: "f7c3d30c-9d3a-43c7-9a22-92d1fca6d2ea") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 22:03:41.842454 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:41.842414 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4vbkc\" (UniqueName: \"kubernetes.io/projected/30024b54-40f6-40b2-9b04-a82f5650ceba-kube-api-access-4vbkc\") pod \"network-check-target-82xqf\" (UID: \"30024b54-40f6-40b2-9b04-a82f5650ceba\") " pod="openshift-network-diagnostics/network-check-target-82xqf" Jul 07 22:03:41.842622 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:41.842466 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/01e846e0-a94f-446e-8ec7-fdeb51040759-original-pull-secret\") pod \"global-pull-secret-syncer-nqfk8\" (UID: \"01e846e0-a94f-446e-8ec7-fdeb51040759\") " pod="kube-system/global-pull-secret-syncer-nqfk8" Jul 07 22:03:41.842698 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:03:41.842622 2545 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 22:03:41.842698 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:03:41.842690 2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/01e846e0-a94f-446e-8ec7-fdeb51040759-original-pull-secret podName:01e846e0-a94f-446e-8ec7-fdeb51040759 nodeName:}" failed. No retries permitted until 2026-07-07 22:03:43.842668863 +0000 UTC m=+6.156891876 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/01e846e0-a94f-446e-8ec7-fdeb51040759-original-pull-secret") pod "global-pull-secret-syncer-nqfk8" (UID: "01e846e0-a94f-446e-8ec7-fdeb51040759") : object "kube-system"/"original-pull-secret" not registered Jul 07 22:03:41.843293 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:03:41.843272 2545 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 22:03:41.843411 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:03:41.843299 2545 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 22:03:41.843411 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:03:41.843313 2545 projected.go:196] Error preparing data for projected volume kube-api-access-4vbkc for pod openshift-network-diagnostics/network-check-target-82xqf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 22:03:41.843411 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:03:41.843357 2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/30024b54-40f6-40b2-9b04-a82f5650ceba-kube-api-access-4vbkc podName:30024b54-40f6-40b2-9b04-a82f5650ceba nodeName:}" failed. No retries permitted until 2026-07-07 22:03:43.843342787 +0000 UTC m=+6.157565785 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-4vbkc" (UniqueName: "kubernetes.io/projected/30024b54-40f6-40b2-9b04-a82f5650ceba-kube-api-access-4vbkc") pod "network-check-target-82xqf" (UID: "30024b54-40f6-40b2-9b04-a82f5650ceba") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 22:03:42.158685 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:42.158642 2545 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-06 21:58:40 +0000 UTC" deadline="2028-01-26 15:34:17.732207908 +0000 UTC" Jul 07 22:03:42.158685 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:42.158680 2545 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13625h30m35.573532039s" Jul 07 22:03:42.307994 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:42.307957 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-82xqf" Jul 07 22:03:42.308194 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:03:42.308074 2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-82xqf" podUID="30024b54-40f6-40b2-9b04-a82f5650ceba" Jul 07 22:03:42.308505 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:42.308483 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nqfk8" Jul 07 22:03:42.308630 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:03:42.308590 2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nqfk8" podUID="01e846e0-a94f-446e-8ec7-fdeb51040759" Jul 07 22:03:42.342159 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:42.341052 2545 generic.go:356] "Generic (PLEG): container finished" podID="a6a963ffa8cd0fda182ebe6d33d409ac" containerID="2dd8d87f587888b08ae7cd17b7dffba6475742fbb22ef073f6c83376feba8f8e" exitCode=0 Jul 07 22:03:42.342159 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:42.341955 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-55.ec2.internal" event={"ID":"a6a963ffa8cd0fda182ebe6d33d409ac","Type":"ContainerDied","Data":"2dd8d87f587888b08ae7cd17b7dffba6475742fbb22ef073f6c83376feba8f8e"} Jul 07 22:03:43.305917 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:43.305420 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-88qj9" Jul 07 22:03:43.305917 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:03:43.305560 2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-88qj9" podUID="f7c3d30c-9d3a-43c7-9a22-92d1fca6d2ea" Jul 07 22:03:43.366242 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:43.365914 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-55.ec2.internal" event={"ID":"a6a963ffa8cd0fda182ebe6d33d409ac","Type":"ContainerStarted","Data":"2f9f4b1d519fd16b003fe3256f1672c42f6c5fbcd29e74b5385abd355ee5b96e"} Jul 07 22:03:43.759361 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:43.758760 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f7c3d30c-9d3a-43c7-9a22-92d1fca6d2ea-metrics-certs\") pod \"network-metrics-daemon-88qj9\" (UID: \"f7c3d30c-9d3a-43c7-9a22-92d1fca6d2ea\") " pod="openshift-multus/network-metrics-daemon-88qj9" Jul 07 22:03:43.759361 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:03:43.758952 2545 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 22:03:43.759361 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:03:43.759014 2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f7c3d30c-9d3a-43c7-9a22-92d1fca6d2ea-metrics-certs podName:f7c3d30c-9d3a-43c7-9a22-92d1fca6d2ea nodeName:}" failed. No retries permitted until 2026-07-07 22:03:47.758995218 +0000 UTC m=+10.073218221 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f7c3d30c-9d3a-43c7-9a22-92d1fca6d2ea-metrics-certs") pod "network-metrics-daemon-88qj9" (UID: "f7c3d30c-9d3a-43c7-9a22-92d1fca6d2ea") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 22:03:43.859861 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:43.859133 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4vbkc\" (UniqueName: \"kubernetes.io/projected/30024b54-40f6-40b2-9b04-a82f5650ceba-kube-api-access-4vbkc\") pod \"network-check-target-82xqf\" (UID: \"30024b54-40f6-40b2-9b04-a82f5650ceba\") " pod="openshift-network-diagnostics/network-check-target-82xqf" Jul 07 22:03:43.859861 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:43.859181 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/01e846e0-a94f-446e-8ec7-fdeb51040759-original-pull-secret\") pod \"global-pull-secret-syncer-nqfk8\" (UID: \"01e846e0-a94f-446e-8ec7-fdeb51040759\") " pod="kube-system/global-pull-secret-syncer-nqfk8" Jul 07 22:03:43.859861 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:03:43.859343 2545 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 22:03:43.859861 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:03:43.859402 2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/01e846e0-a94f-446e-8ec7-fdeb51040759-original-pull-secret podName:01e846e0-a94f-446e-8ec7-fdeb51040759 nodeName:}" failed. No retries permitted until 2026-07-07 22:03:47.859384884 +0000 UTC m=+10.173607895 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/01e846e0-a94f-446e-8ec7-fdeb51040759-original-pull-secret") pod "global-pull-secret-syncer-nqfk8" (UID: "01e846e0-a94f-446e-8ec7-fdeb51040759") : object "kube-system"/"original-pull-secret" not registered Jul 07 22:03:43.859861 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:03:43.859483 2545 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 22:03:43.859861 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:03:43.859497 2545 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 22:03:43.859861 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:03:43.859512 2545 projected.go:196] Error preparing data for projected volume kube-api-access-4vbkc for pod openshift-network-diagnostics/network-check-target-82xqf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 22:03:43.859861 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:03:43.859545 2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/30024b54-40f6-40b2-9b04-a82f5650ceba-kube-api-access-4vbkc podName:30024b54-40f6-40b2-9b04-a82f5650ceba nodeName:}" failed. No retries permitted until 2026-07-07 22:03:47.859534071 +0000 UTC m=+10.173757072 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-4vbkc" (UniqueName: "kubernetes.io/projected/30024b54-40f6-40b2-9b04-a82f5650ceba-kube-api-access-4vbkc") pod "network-check-target-82xqf" (UID: "30024b54-40f6-40b2-9b04-a82f5650ceba") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 22:03:44.306002 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:44.305969 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-82xqf" Jul 07 22:03:44.306457 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:44.305979 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nqfk8" Jul 07 22:03:44.306457 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:03:44.306108 2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-82xqf" podUID="30024b54-40f6-40b2-9b04-a82f5650ceba" Jul 07 22:03:44.306457 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:03:44.306198 2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nqfk8" podUID="01e846e0-a94f-446e-8ec7-fdeb51040759" Jul 07 22:03:45.306129 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:45.306027 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-88qj9" Jul 07 22:03:45.306576 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:03:45.306197 2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-88qj9" podUID="f7c3d30c-9d3a-43c7-9a22-92d1fca6d2ea" Jul 07 22:03:46.305245 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:46.305213 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-82xqf" Jul 07 22:03:46.305410 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:46.305255 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nqfk8" Jul 07 22:03:46.305410 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:03:46.305347 2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-82xqf" podUID="30024b54-40f6-40b2-9b04-a82f5650ceba" Jul 07 22:03:46.305515 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:03:46.305443 2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nqfk8" podUID="01e846e0-a94f-446e-8ec7-fdeb51040759" Jul 07 22:03:47.305701 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:47.305667 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-88qj9" Jul 07 22:03:47.306305 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:03:47.305814 2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-88qj9" podUID="f7c3d30c-9d3a-43c7-9a22-92d1fca6d2ea" Jul 07 22:03:47.789011 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:47.788912 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f7c3d30c-9d3a-43c7-9a22-92d1fca6d2ea-metrics-certs\") pod \"network-metrics-daemon-88qj9\" (UID: \"f7c3d30c-9d3a-43c7-9a22-92d1fca6d2ea\") " pod="openshift-multus/network-metrics-daemon-88qj9" Jul 07 22:03:47.789202 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:03:47.789028 2545 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 22:03:47.789202 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:03:47.789115 2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f7c3d30c-9d3a-43c7-9a22-92d1fca6d2ea-metrics-certs podName:f7c3d30c-9d3a-43c7-9a22-92d1fca6d2ea nodeName:}" failed. No retries permitted until 2026-07-07 22:03:55.789074364 +0000 UTC m=+18.103297366 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f7c3d30c-9d3a-43c7-9a22-92d1fca6d2ea-metrics-certs") pod "network-metrics-daemon-88qj9" (UID: "f7c3d30c-9d3a-43c7-9a22-92d1fca6d2ea") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 22:03:47.890051 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:47.890010 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/01e846e0-a94f-446e-8ec7-fdeb51040759-original-pull-secret\") pod \"global-pull-secret-syncer-nqfk8\" (UID: \"01e846e0-a94f-446e-8ec7-fdeb51040759\") " pod="kube-system/global-pull-secret-syncer-nqfk8" Jul 07 22:03:47.890255 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:47.890123 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4vbkc\" (UniqueName: \"kubernetes.io/projected/30024b54-40f6-40b2-9b04-a82f5650ceba-kube-api-access-4vbkc\") pod \"network-check-target-82xqf\" (UID: \"30024b54-40f6-40b2-9b04-a82f5650ceba\") " pod="openshift-network-diagnostics/network-check-target-82xqf" Jul 07 22:03:47.890255 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:03:47.890199 2545 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 22:03:47.890255 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:03:47.890232 2545 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 22:03:47.890255 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:03:47.890246 2545 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 22:03:47.890255 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:03:47.890259 2545 projected.go:196] Error preparing data for projected volume kube-api-access-4vbkc for pod openshift-network-diagnostics/network-check-target-82xqf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 22:03:47.890491 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:03:47.890282 2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/01e846e0-a94f-446e-8ec7-fdeb51040759-original-pull-secret podName:01e846e0-a94f-446e-8ec7-fdeb51040759 nodeName:}" failed. No retries permitted until 2026-07-07 22:03:55.890258776 +0000 UTC m=+18.204481797 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/01e846e0-a94f-446e-8ec7-fdeb51040759-original-pull-secret") pod "global-pull-secret-syncer-nqfk8" (UID: "01e846e0-a94f-446e-8ec7-fdeb51040759") : object "kube-system"/"original-pull-secret" not registered Jul 07 22:03:47.890491 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:03:47.890309 2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/30024b54-40f6-40b2-9b04-a82f5650ceba-kube-api-access-4vbkc podName:30024b54-40f6-40b2-9b04-a82f5650ceba nodeName:}" failed. No retries permitted until 2026-07-07 22:03:55.890294263 +0000 UTC m=+18.204517287 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-4vbkc" (UniqueName: "kubernetes.io/projected/30024b54-40f6-40b2-9b04-a82f5650ceba-kube-api-access-4vbkc") pod "network-check-target-82xqf" (UID: "30024b54-40f6-40b2-9b04-a82f5650ceba") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 22:03:48.306488 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:48.306457 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nqfk8" Jul 07 22:03:48.306885 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:03:48.306566 2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nqfk8" podUID="01e846e0-a94f-446e-8ec7-fdeb51040759" Jul 07 22:03:48.306885 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:48.306607 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-82xqf" Jul 07 22:03:48.306885 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:03:48.306685 2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-82xqf" podUID="30024b54-40f6-40b2-9b04-a82f5650ceba" Jul 07 22:03:49.305784 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:49.305748 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-88qj9" Jul 07 22:03:49.305950 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:03:49.305879 2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-88qj9" podUID="f7c3d30c-9d3a-43c7-9a22-92d1fca6d2ea" Jul 07 22:03:50.305445 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:50.305406 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nqfk8" Jul 07 22:03:50.305445 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:50.305438 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-82xqf" Jul 07 22:03:50.305929 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:03:50.305555 2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nqfk8" podUID="01e846e0-a94f-446e-8ec7-fdeb51040759" Jul 07 22:03:50.305929 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:03:50.305699 2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-82xqf" podUID="30024b54-40f6-40b2-9b04-a82f5650ceba" Jul 07 22:03:51.305301 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:51.305271 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-88qj9" Jul 07 22:03:51.305468 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:03:51.305369 2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-88qj9" podUID="f7c3d30c-9d3a-43c7-9a22-92d1fca6d2ea" Jul 07 22:03:52.305810 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:52.305776 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-82xqf" Jul 07 22:03:52.306275 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:03:52.305908 2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-82xqf" podUID="30024b54-40f6-40b2-9b04-a82f5650ceba" Jul 07 22:03:52.306275 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:52.305971 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nqfk8" Jul 07 22:03:52.306275 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:03:52.306108 2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nqfk8" podUID="01e846e0-a94f-446e-8ec7-fdeb51040759" Jul 07 22:03:53.305763 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:53.305732 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-88qj9" Jul 07 22:03:53.305916 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:03:53.305847 2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-88qj9" podUID="f7c3d30c-9d3a-43c7-9a22-92d1fca6d2ea" Jul 07 22:03:54.305422 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:54.305383 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-82xqf" Jul 07 22:03:54.305607 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:54.305383 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nqfk8" Jul 07 22:03:54.305607 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:03:54.305515 2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-82xqf" podUID="30024b54-40f6-40b2-9b04-a82f5650ceba" Jul 07 22:03:54.305607 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:03:54.305585 2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nqfk8" podUID="01e846e0-a94f-446e-8ec7-fdeb51040759" Jul 07 22:03:55.306008 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:55.305972 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-88qj9" Jul 07 22:03:55.306523 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:03:55.306127 2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-88qj9" podUID="f7c3d30c-9d3a-43c7-9a22-92d1fca6d2ea" Jul 07 22:03:55.853049 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:55.853014 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f7c3d30c-9d3a-43c7-9a22-92d1fca6d2ea-metrics-certs\") pod \"network-metrics-daemon-88qj9\" (UID: \"f7c3d30c-9d3a-43c7-9a22-92d1fca6d2ea\") " pod="openshift-multus/network-metrics-daemon-88qj9" Jul 07 22:03:55.853322 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:03:55.853180 2545 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 22:03:55.853322 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:03:55.853241 2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f7c3d30c-9d3a-43c7-9a22-92d1fca6d2ea-metrics-certs podName:f7c3d30c-9d3a-43c7-9a22-92d1fca6d2ea nodeName:}" failed. No retries permitted until 2026-07-07 22:04:11.853223729 +0000 UTC m=+34.167446742 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f7c3d30c-9d3a-43c7-9a22-92d1fca6d2ea-metrics-certs") pod "network-metrics-daemon-88qj9" (UID: "f7c3d30c-9d3a-43c7-9a22-92d1fca6d2ea") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 22:03:55.953898 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:55.953859 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4vbkc\" (UniqueName: \"kubernetes.io/projected/30024b54-40f6-40b2-9b04-a82f5650ceba-kube-api-access-4vbkc\") pod \"network-check-target-82xqf\" (UID: \"30024b54-40f6-40b2-9b04-a82f5650ceba\") " pod="openshift-network-diagnostics/network-check-target-82xqf" Jul 07 22:03:55.953898 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:55.953902 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/01e846e0-a94f-446e-8ec7-fdeb51040759-original-pull-secret\") pod \"global-pull-secret-syncer-nqfk8\" (UID: \"01e846e0-a94f-446e-8ec7-fdeb51040759\") " pod="kube-system/global-pull-secret-syncer-nqfk8" Jul 07 22:03:55.954179 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:03:55.954013 2545 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 22:03:55.954179 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:03:55.954024 2545 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 22:03:55.954179 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:03:55.954046 2545 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 22:03:55.954179 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:03:55.954055 2545 projected.go:196] Error preparing data for projected volume kube-api-access-4vbkc for pod openshift-network-diagnostics/network-check-target-82xqf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 22:03:55.954179 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:03:55.954072 2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/01e846e0-a94f-446e-8ec7-fdeb51040759-original-pull-secret podName:01e846e0-a94f-446e-8ec7-fdeb51040759 nodeName:}" failed. No retries permitted until 2026-07-07 22:04:11.95405515 +0000 UTC m=+34.268278161 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/01e846e0-a94f-446e-8ec7-fdeb51040759-original-pull-secret") pod "global-pull-secret-syncer-nqfk8" (UID: "01e846e0-a94f-446e-8ec7-fdeb51040759") : object "kube-system"/"original-pull-secret" not registered Jul 07 22:03:55.954179 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:03:55.954119 2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/30024b54-40f6-40b2-9b04-a82f5650ceba-kube-api-access-4vbkc podName:30024b54-40f6-40b2-9b04-a82f5650ceba nodeName:}" failed. No retries permitted until 2026-07-07 22:04:11.954107106 +0000 UTC m=+34.268330118 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-4vbkc" (UniqueName: "kubernetes.io/projected/30024b54-40f6-40b2-9b04-a82f5650ceba-kube-api-access-4vbkc") pod "network-check-target-82xqf" (UID: "30024b54-40f6-40b2-9b04-a82f5650ceba") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 22:03:56.305832 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:56.305794 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-82xqf" Jul 07 22:03:56.306021 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:56.305794 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nqfk8" Jul 07 22:03:56.306021 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:03:56.305917 2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-82xqf" podUID="30024b54-40f6-40b2-9b04-a82f5650ceba" Jul 07 22:03:56.306437 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:03:56.306014 2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nqfk8" podUID="01e846e0-a94f-446e-8ec7-fdeb51040759" Jul 07 22:03:57.305968 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:57.305933 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-88qj9" Jul 07 22:03:57.306150 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:03:57.306064 2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-88qj9" podUID="f7c3d30c-9d3a-43c7-9a22-92d1fca6d2ea" Jul 07 22:03:58.306256 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:58.306213 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nqfk8" Jul 07 22:03:58.306717 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:03:58.306327 2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nqfk8" podUID="01e846e0-a94f-446e-8ec7-fdeb51040759" Jul 07 22:03:58.306717 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:58.306391 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-82xqf" Jul 07 22:03:58.306717 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:03:58.306513 2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-82xqf" podUID="30024b54-40f6-40b2-9b04-a82f5650ceba" Jul 07 22:03:59.305939 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:03:59.305908 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-88qj9" Jul 07 22:03:59.306141 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:03:59.306035 2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-88qj9" podUID="f7c3d30c-9d3a-43c7-9a22-92d1fca6d2ea" Jul 07 22:04:00.307105 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:00.306334 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nqfk8" Jul 07 22:04:00.307105 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:04:00.306638 2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nqfk8" podUID="01e846e0-a94f-446e-8ec7-fdeb51040759" Jul 07 22:04:00.307105 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:00.306339 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-82xqf" Jul 07 22:04:00.307105 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:04:00.307047 2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-82xqf" podUID="30024b54-40f6-40b2-9b04-a82f5650ceba" Jul 07 22:04:00.402398 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:00.402198 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-j8q2m" event={"ID":"065087ce-e289-40dc-be53-bea81382bf96","Type":"ContainerStarted","Data":"6be10835c558897e6a4d716a6442e38010b63462bf49c0695f7295e746e04168"} Jul 07 22:04:00.408159 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:00.408122 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-9vvqt" event={"ID":"7ac39c08-2ce6-4578-8b01-b68587adf750","Type":"ContainerStarted","Data":"5572072c45dd0fa355ac443d56bdf360d2a6ea29bb84dc56a59bb20cc6f18ed8"} Jul 07 22:04:00.409784 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:00.409752 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-cc72r" event={"ID":"381be182-aba1-4465-a899-8f217bbeacd2","Type":"ContainerStarted","Data":"f7b0e811e156e1d4e83d2b07b1979370fd3eb35a78099b84fc028e09f5dd49f9"} Jul 07 22:04:00.411768 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:00.411706 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-ms5bj" event={"ID":"35a9b908-7fda-4fb9-80b9-56b119f92de0","Type":"ContainerStarted","Data":"b9c2ba509d3e0cc2f406a76e0c73502a3700daa292bbaf15ccb4e20b6df3c34f"} Jul 07 22:04:00.412968 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:00.412950 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-fg88n" event={"ID":"f7c107a3-d0f6-479d-b548-c4c59b612631","Type":"ContainerStarted","Data":"6b0e99fee3fc598b1857d1f4d1d6a544d6efe052d9804b3a5e6fa6a66556fe2b"} Jul 07 22:04:00.414627 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:00.414605 2545 generic.go:356] "Generic (PLEG): container finished" podID="00baab1c-b8e0-4b5e-bd42-647ba72c5aca" containerID="7ea21157a40c099b10e8257860a0c177237a3f61e7786812d928921f6c4451b3" exitCode=0 Jul 07 22:04:00.414723 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:00.414686 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ztkm2" event={"ID":"00baab1c-b8e0-4b5e-bd42-647ba72c5aca","Type":"ContainerDied","Data":"7ea21157a40c099b10e8257860a0c177237a3f61e7786812d928921f6c4451b3"} Jul 07 22:04:00.416295 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:00.416131 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6h4fg" event={"ID":"04b1516f-c72c-499c-9c45-8eff8b82ffa2","Type":"ContainerStarted","Data":"e1d2aa336e7733bddb188a5b862f85a753e1009cd92b134fc5025955769a9dad"} Jul 07 22:04:00.420663 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:00.420608 2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-55.ec2.internal" podStartSLOduration=21.420589448 podStartE2EDuration="21.420589448s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 22:03:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:03:43.386498925 +0000 UTC m=+5.700721945" watchObservedRunningTime="2026-07-07 22:04:00.420589448 +0000 UTC m=+22.734812480" Jul 07 22:04:00.434360 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:00.434028 2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-j8q2m" podStartSLOduration=3.458439984 podStartE2EDuration="22.434009552s" totalImagesPullingTime="18.975569568s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 22:03:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:04:00.420200154 +0000 UTC m=+22.734423177" watchObservedRunningTime="2026-07-07 22:04:00.434009552 +0000 UTC m=+22.748232571" Jul 07 22:04:00.434360 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:00.434195 2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-9vvqt" podStartSLOduration=3.502811613 podStartE2EDuration="22.434184833s" totalImagesPullingTime="18.93137322s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 22:03:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:04:00.433454131 +0000 UTC m=+22.747677150" watchObservedRunningTime="2026-07-07 22:04:00.434184833 +0000 UTC m=+22.748407854" Jul 07 22:04:00.482828 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:00.482770 2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-ms5bj" podStartSLOduration=7.602231815 podStartE2EDuration="22.482748039s" totalImagesPullingTime="14.880516224s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 22:03:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:04:00.482417042 +0000 UTC m=+22.796640072" watchObservedRunningTime="2026-07-07 22:04:00.482748039 +0000 UTC m=+22.796971059" Jul 07 22:04:00.498353 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:00.498308 2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-fg88n" podStartSLOduration=3.30037295 podStartE2EDuration="22.498294339s" totalImagesPullingTime="19.197921389s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 22:03:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:04:00.497973399 +0000 UTC m=+22.812196429" watchObservedRunningTime="2026-07-07 22:04:00.498294339 +0000 UTC m=+22.812517368" Jul 07 22:04:00.511216 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:00.511172 2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-cc72r" podStartSLOduration=3.552426436 podStartE2EDuration="22.511157352s" totalImagesPullingTime="18.958730916s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 22:03:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:04:00.510757647 +0000 UTC m=+22.824980666" watchObservedRunningTime="2026-07-07 22:04:00.511157352 +0000 UTC m=+22.825380370" Jul 07 22:04:01.305795 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:01.305760 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-88qj9" Jul 07 22:04:01.305966 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:04:01.305892 2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-88qj9" podUID="f7c3d30c-9d3a-43c7-9a22-92d1fca6d2ea" Jul 07 22:04:01.396499 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:01.396355 2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-ms5bj" Jul 07 22:04:01.396900 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:01.396883 2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-ms5bj" Jul 07 22:04:01.420646 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:01.420604 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bgm75" event={"ID":"9bd24331-107d-4dcf-a0e2-8221ca7ff5d0","Type":"ContainerStarted","Data":"94795d4bc3923e799a2a125437cfb7ba6eed85dc91ca29d1d14b3668eb2fa460"} Jul 07 22:04:01.425024 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:01.424990 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6h4fg" event={"ID":"04b1516f-c72c-499c-9c45-8eff8b82ffa2","Type":"ContainerStarted","Data":"bd1706ac555a7cbfb6488d6bbf725f49e6ded1980e1f8723e15591ed859c0ff2"} Jul 07 22:04:01.425181 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:01.425070 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6h4fg" event={"ID":"04b1516f-c72c-499c-9c45-8eff8b82ffa2","Type":"ContainerStarted","Data":"3da6bc4a9501bfc4671428ffe5b7c7603076fee4ba2a5dc521db37a5431b2201"} Jul 07 22:04:01.425181 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:01.425142 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6h4fg" event={"ID":"04b1516f-c72c-499c-9c45-8eff8b82ffa2","Type":"ContainerStarted","Data":"e971efcb3529a175b3f495a671c2e95b361fe8da871380bdf272a1f037b4f9c4"} Jul 07 22:04:01.425307 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:01.425183 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6h4fg" event={"ID":"04b1516f-c72c-499c-9c45-8eff8b82ffa2","Type":"ContainerStarted","Data":"85156b37ffda11e4701e65e30607e9d63cf2406b2debf0f7e616696ca39dadcf"} Jul 07 22:04:01.425307 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:01.425212 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6h4fg" event={"ID":"04b1516f-c72c-499c-9c45-8eff8b82ffa2","Type":"ContainerStarted","Data":"70cd78067e7329fa515d12a6c70a4a309864f8fe4410d79e8aa2a0ede5402240"} Jul 07 22:04:01.426798 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:01.426768 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-xshfx" event={"ID":"2f1fe6ee-68ca-4a89-8715-32c449d0b63b","Type":"ContainerStarted","Data":"c7abe54e55b4d8ce986c982422efafc27fb681fe5f16a0c96c6de715ddf23450"} Jul 07 22:04:01.441232 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:01.441184 2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-xshfx" podStartSLOduration=4.466607506 podStartE2EDuration="23.441166075s" totalImagesPullingTime="18.974558569s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 22:03:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:04:01.44104011 +0000 UTC m=+23.755263133" watchObservedRunningTime="2026-07-07 22:04:01.441166075 +0000 UTC m=+23.755389095" Jul 07 22:04:02.021992 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:02.021829 2545 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 07 22:04:02.187935 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:02.187805 2545 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-07T22:04:02.021989051Z","UUID":"562a8b75-5c5d-48f7-89b7-7cc5d19438d0","Handler":null,"Name":"","Endpoint":""} Jul 07 22:04:02.189492 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:02.189395 2545 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 07 22:04:02.189492 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:02.189440 2545 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 07 22:04:02.305410 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:02.305380 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-82xqf" Jul 07 22:04:02.305582 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:04:02.305508 2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-82xqf" podUID="30024b54-40f6-40b2-9b04-a82f5650ceba" Jul 07 22:04:02.305582 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:02.305554 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nqfk8" Jul 07 22:04:02.305664 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:04:02.305646 2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nqfk8" podUID="01e846e0-a94f-446e-8ec7-fdeb51040759" Jul 07 22:04:02.431920 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:02.431895 2545 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 07 22:04:02.431920 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:02.431898 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bgm75" event={"ID":"9bd24331-107d-4dcf-a0e2-8221ca7ff5d0","Type":"ContainerStarted","Data":"c7620224ac2bdc27e602e15edf88461453d582bf5d64582118bc8bdb0240064c"} Jul 07 22:04:03.305589 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:03.305552 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-88qj9" Jul 07 22:04:03.305814 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:04:03.305678 2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-88qj9" podUID="f7c3d30c-9d3a-43c7-9a22-92d1fca6d2ea" Jul 07 22:04:03.438851 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:03.438801 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6h4fg" event={"ID":"04b1516f-c72c-499c-9c45-8eff8b82ffa2","Type":"ContainerStarted","Data":"940ae9fea7d73713636636bd057b358c352e6ddc81a3c18f0b479da573e9329a"} Jul 07 22:04:04.305864 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:04.305827 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nqfk8" Jul 07 22:04:04.306040 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:04.305827 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-82xqf" Jul 07 22:04:04.306040 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:04:04.305970 2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nqfk8" podUID="01e846e0-a94f-446e-8ec7-fdeb51040759" Jul 07 22:04:04.306186 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:04:04.306129 2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-82xqf" podUID="30024b54-40f6-40b2-9b04-a82f5650ceba" Jul 07 22:04:04.443751 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:04.443716 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bgm75" event={"ID":"9bd24331-107d-4dcf-a0e2-8221ca7ff5d0","Type":"ContainerStarted","Data":"89cedf2461785a6475e3fae73cdb643377d58c24a32755e40a6bda9beced0c22"} Jul 07 22:04:04.460691 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:04.460635 2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-bgm75" podStartSLOduration=4.60403811 podStartE2EDuration="26.460616474s" totalImagesPullingTime="21.856578364s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 22:03:38 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:04:04.45992718 +0000 UTC m=+26.774150199" watchObservedRunningTime="2026-07-07 22:04:04.460616474 +0000 UTC m=+26.774839494" Jul 07 22:04:05.305708 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:05.305678 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-88qj9" Jul 07 22:04:05.305880 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:04:05.305806 2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-88qj9" podUID="f7c3d30c-9d3a-43c7-9a22-92d1fca6d2ea" Jul 07 22:04:06.305647 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:06.305411 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nqfk8" Jul 07 22:04:06.306134 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:06.305415 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-82xqf" Jul 07 22:04:06.306134 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:04:06.305752 2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nqfk8" podUID="01e846e0-a94f-446e-8ec7-fdeb51040759" Jul 07 22:04:06.306134 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:04:06.305795 2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-82xqf" podUID="30024b54-40f6-40b2-9b04-a82f5650ceba" Jul 07 22:04:06.449513 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:06.449482 2545 generic.go:356] "Generic (PLEG): container finished" podID="00baab1c-b8e0-4b5e-bd42-647ba72c5aca" containerID="b8610cb16501db9aba069af7f53af91a111f2091f464a8dd3fc7ee392b6d12b3" exitCode=0 Jul 07 22:04:06.449692 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:06.449565 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ztkm2" event={"ID":"00baab1c-b8e0-4b5e-bd42-647ba72c5aca","Type":"ContainerDied","Data":"b8610cb16501db9aba069af7f53af91a111f2091f464a8dd3fc7ee392b6d12b3"} Jul 07 22:04:06.453275 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:06.453251 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6h4fg" event={"ID":"04b1516f-c72c-499c-9c45-8eff8b82ffa2","Type":"ContainerStarted","Data":"e997b3f848d9302a6869ce723ed938237d8ec8745ef852d8cf646564a9e921e5"} Jul 07 22:04:06.456016 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:06.455108 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-6h4fg" Jul 07 22:04:06.456016 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:06.455252 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-6h4fg" Jul 07 22:04:06.456016 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:06.455319 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-6h4fg" Jul 07 22:04:06.471268 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:06.471245 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-6h4fg" Jul 07 22:04:06.471379 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:06.471344 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-6h4fg" Jul 07 22:04:06.497523 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:06.497482 2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-6h4fg" podStartSLOduration=9.491751071 podStartE2EDuration="28.497467907s" totalImagesPullingTime="19.005716836s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 22:03:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:04:06.496990019 +0000 UTC m=+28.811213049" watchObservedRunningTime="2026-07-07 22:04:06.497467907 +0000 UTC m=+28.811690927" Jul 07 22:04:07.305899 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:07.305673 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-88qj9" Jul 07 22:04:07.306412 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:04:07.306038 2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-88qj9" podUID="f7c3d30c-9d3a-43c7-9a22-92d1fca6d2ea" Jul 07 22:04:07.712334 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:07.712305 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-nqfk8"] Jul 07 22:04:07.712467 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:07.712414 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nqfk8" Jul 07 22:04:07.712513 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:04:07.712490 2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nqfk8" podUID="01e846e0-a94f-446e-8ec7-fdeb51040759" Jul 07 22:04:07.714691 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:07.714668 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-88qj9"] Jul 07 22:04:07.714818 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:07.714756 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-88qj9" Jul 07 22:04:07.714860 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:04:07.714841 2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-88qj9" podUID="f7c3d30c-9d3a-43c7-9a22-92d1fca6d2ea" Jul 07 22:04:07.717247 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:07.717226 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-82xqf"] Jul 07 22:04:07.717364 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:07.717319 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-82xqf" Jul 07 22:04:07.717424 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:04:07.717407 2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-82xqf" podUID="30024b54-40f6-40b2-9b04-a82f5650ceba" Jul 07 22:04:08.459434 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:08.459401 2545 generic.go:356] "Generic (PLEG): container finished" podID="00baab1c-b8e0-4b5e-bd42-647ba72c5aca" containerID="88668c82797e18ee23abc764f2e801a027b6d1f10c27f6fc10888fd2f46d3f97" exitCode=0 Jul 07 22:04:08.459880 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:08.459494 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ztkm2" event={"ID":"00baab1c-b8e0-4b5e-bd42-647ba72c5aca","Type":"ContainerDied","Data":"88668c82797e18ee23abc764f2e801a027b6d1f10c27f6fc10888fd2f46d3f97"} Jul 07 22:04:09.127594 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:09.127558 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-ms5bj" Jul 07 22:04:09.127781 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:09.127717 2545 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 07 22:04:09.128337 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:09.128312 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-ms5bj" Jul 07 22:04:09.305594 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:09.305560 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nqfk8" Jul 07 22:04:09.305753 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:09.305566 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-82xqf" Jul 07 22:04:09.305753 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:09.305690 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-88qj9" Jul 07 22:04:09.305753 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:04:09.305686 2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nqfk8" podUID="01e846e0-a94f-446e-8ec7-fdeb51040759" Jul 07 22:04:09.305908 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:04:09.305769 2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-82xqf" podUID="30024b54-40f6-40b2-9b04-a82f5650ceba" Jul 07 22:04:09.305908 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:04:09.305864 2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-88qj9" podUID="f7c3d30c-9d3a-43c7-9a22-92d1fca6d2ea" Jul 07 22:04:10.469778 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:10.469606 2545 generic.go:356] "Generic (PLEG): container finished" podID="00baab1c-b8e0-4b5e-bd42-647ba72c5aca" containerID="baea1104e3cf26e322d0bac973a762899757c80f08473ab8d02765011e23fd93" exitCode=0 Jul 07 22:04:10.470410 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:10.469688 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ztkm2" event={"ID":"00baab1c-b8e0-4b5e-bd42-647ba72c5aca","Type":"ContainerDied","Data":"baea1104e3cf26e322d0bac973a762899757c80f08473ab8d02765011e23fd93"} Jul 07 22:04:11.306033 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:11.305999 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nqfk8" Jul 07 22:04:11.306033 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:11.306032 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-82xqf" Jul 07 22:04:11.306306 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:11.306045 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-88qj9" Jul 07 22:04:11.306306 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:04:11.306154 2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-nqfk8" podUID="01e846e0-a94f-446e-8ec7-fdeb51040759" Jul 07 22:04:11.306306 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:04:11.306264 2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-82xqf" podUID="30024b54-40f6-40b2-9b04-a82f5650ceba" Jul 07 22:04:11.306470 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:04:11.306343 2545 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-88qj9" podUID="f7c3d30c-9d3a-43c7-9a22-92d1fca6d2ea" Jul 07 22:04:11.872580 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:11.872537 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f7c3d30c-9d3a-43c7-9a22-92d1fca6d2ea-metrics-certs\") pod \"network-metrics-daemon-88qj9\" (UID: \"f7c3d30c-9d3a-43c7-9a22-92d1fca6d2ea\") " pod="openshift-multus/network-metrics-daemon-88qj9" Jul 07 22:04:11.873020 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:04:11.872667 2545 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 22:04:11.873020 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:04:11.872745 2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f7c3d30c-9d3a-43c7-9a22-92d1fca6d2ea-metrics-certs podName:f7c3d30c-9d3a-43c7-9a22-92d1fca6d2ea nodeName:}" failed. No retries permitted until 2026-07-07 22:04:43.872722882 +0000 UTC m=+66.186945891 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f7c3d30c-9d3a-43c7-9a22-92d1fca6d2ea-metrics-certs") pod "network-metrics-daemon-88qj9" (UID: "f7c3d30c-9d3a-43c7-9a22-92d1fca6d2ea") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 07 22:04:11.973011 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:11.972975 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4vbkc\" (UniqueName: \"kubernetes.io/projected/30024b54-40f6-40b2-9b04-a82f5650ceba-kube-api-access-4vbkc\") pod \"network-check-target-82xqf\" (UID: \"30024b54-40f6-40b2-9b04-a82f5650ceba\") " pod="openshift-network-diagnostics/network-check-target-82xqf" Jul 07 22:04:11.973011 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:11.973018 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/01e846e0-a94f-446e-8ec7-fdeb51040759-original-pull-secret\") pod \"global-pull-secret-syncer-nqfk8\" (UID: \"01e846e0-a94f-446e-8ec7-fdeb51040759\") " pod="kube-system/global-pull-secret-syncer-nqfk8" Jul 07 22:04:11.973234 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:04:11.973156 2545 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 07 22:04:11.973234 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:04:11.973175 2545 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 07 22:04:11.973234 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:04:11.973199 2545 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 07 22:04:11.973234 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:04:11.973209 2545 projected.go:196] Error preparing data for projected volume kube-api-access-4vbkc for pod openshift-network-diagnostics/network-check-target-82xqf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 22:04:11.973234 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:04:11.973219 2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/01e846e0-a94f-446e-8ec7-fdeb51040759-original-pull-secret podName:01e846e0-a94f-446e-8ec7-fdeb51040759 nodeName:}" failed. No retries permitted until 2026-07-07 22:04:43.973203281 +0000 UTC m=+66.287426287 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/01e846e0-a94f-446e-8ec7-fdeb51040759-original-pull-secret") pod "global-pull-secret-syncer-nqfk8" (UID: "01e846e0-a94f-446e-8ec7-fdeb51040759") : object "kube-system"/"original-pull-secret" not registered Jul 07 22:04:11.973390 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:04:11.973249 2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/30024b54-40f6-40b2-9b04-a82f5650ceba-kube-api-access-4vbkc podName:30024b54-40f6-40b2-9b04-a82f5650ceba nodeName:}" failed. No retries permitted until 2026-07-07 22:04:43.973237593 +0000 UTC m=+66.287460590 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-4vbkc" (UniqueName: "kubernetes.io/projected/30024b54-40f6-40b2-9b04-a82f5650ceba-kube-api-access-4vbkc") pod "network-check-target-82xqf" (UID: "30024b54-40f6-40b2-9b04-a82f5650ceba") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 07 22:04:12.894186 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:12.894156 2545 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-55.ec2.internal" event="NodeReady" Jul 07 22:04:12.894629 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:12.894335 2545 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 07 22:04:12.925659 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:12.925627 2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-b9885dbc7-lx8h7"] Jul 07 22:04:12.927647 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:12.927627 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-b9885dbc7-lx8h7" Jul 07 22:04:12.930257 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:12.930185 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 07 22:04:12.930257 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:12.930217 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 07 22:04:12.930257 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:12.930242 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 07 22:04:12.930464 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:12.930242 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-c748h\"" Jul 07 22:04:12.936652 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:12.936626 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 07 22:04:12.937487 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:12.937459 2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-wrlkn"] Jul 07 22:04:12.939973 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:12.939956 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-wrlkn" Jul 07 22:04:12.940117 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:12.940099 2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-bqv92"] Jul 07 22:04:12.941932 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:12.941914 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-bqv92" Jul 07 22:04:12.942784 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:12.942765 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 07 22:04:12.942994 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:12.942977 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 07 22:04:12.943075 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:12.943039 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 07 22:04:12.943394 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:12.943282 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-8gs5j\"" Jul 07 22:04:12.944220 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:12.944200 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-b9885dbc7-lx8h7"] Jul 07 22:04:12.944369 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:12.944243 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 07 22:04:12.944484 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:12.944457 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 07 22:04:12.944611 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:12.944582 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-kp4p7\"" Jul 07 22:04:12.951362 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:12.951341 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-wrlkn"] Jul 07 22:04:12.956391 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:12.956371 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-bqv92"] Jul 07 22:04:13.081796 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:13.081763 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/78996163-d007-4f3c-ac3a-44902fe9be66-image-registry-private-configuration\") pod \"image-registry-b9885dbc7-lx8h7\" (UID: \"78996163-d007-4f3c-ac3a-44902fe9be66\") " pod="openshift-image-registry/image-registry-b9885dbc7-lx8h7" Jul 07 22:04:13.081796 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:13.081799 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x7txf\" (UniqueName: \"kubernetes.io/projected/c80e0cce-1240-4a8f-873b-abd71e85b59a-kube-api-access-x7txf\") pod \"dns-default-bqv92\" (UID: \"c80e0cce-1240-4a8f-873b-abd71e85b59a\") " pod="openshift-dns/dns-default-bqv92" Jul 07 22:04:13.082028 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:13.081839 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/78996163-d007-4f3c-ac3a-44902fe9be66-trusted-ca\") pod \"image-registry-b9885dbc7-lx8h7\" (UID: \"78996163-d007-4f3c-ac3a-44902fe9be66\") " pod="openshift-image-registry/image-registry-b9885dbc7-lx8h7" Jul 07 22:04:13.082028 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:13.081855 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/78996163-d007-4f3c-ac3a-44902fe9be66-bound-sa-token\") pod \"image-registry-b9885dbc7-lx8h7\" (UID: \"78996163-d007-4f3c-ac3a-44902fe9be66\") " pod="openshift-image-registry/image-registry-b9885dbc7-lx8h7" Jul 07 22:04:13.082028 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:13.081870 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9xtqm\" (UniqueName: \"kubernetes.io/projected/78996163-d007-4f3c-ac3a-44902fe9be66-kube-api-access-9xtqm\") pod \"image-registry-b9885dbc7-lx8h7\" (UID: \"78996163-d007-4f3c-ac3a-44902fe9be66\") " pod="openshift-image-registry/image-registry-b9885dbc7-lx8h7" Jul 07 22:04:13.082028 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:13.081889 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/78996163-d007-4f3c-ac3a-44902fe9be66-registry-certificates\") pod \"image-registry-b9885dbc7-lx8h7\" (UID: \"78996163-d007-4f3c-ac3a-44902fe9be66\") " pod="openshift-image-registry/image-registry-b9885dbc7-lx8h7" Jul 07 22:04:13.082028 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:13.081905 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c80e0cce-1240-4a8f-873b-abd71e85b59a-config-volume\") pod \"dns-default-bqv92\" (UID: \"c80e0cce-1240-4a8f-873b-abd71e85b59a\") " pod="openshift-dns/dns-default-bqv92" Jul 07 22:04:13.082028 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:13.081919 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c80e0cce-1240-4a8f-873b-abd71e85b59a-metrics-tls\") pod \"dns-default-bqv92\" (UID: \"c80e0cce-1240-4a8f-873b-abd71e85b59a\") " pod="openshift-dns/dns-default-bqv92" Jul 07 22:04:13.082028 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:13.081934 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/230f7860-610d-430a-987b-6dfaa258d4e7-cert\") pod \"ingress-canary-wrlkn\" (UID: \"230f7860-610d-430a-987b-6dfaa258d4e7\") " pod="openshift-ingress-canary/ingress-canary-wrlkn" Jul 07 22:04:13.082028 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:13.081954 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/78996163-d007-4f3c-ac3a-44902fe9be66-registry-tls\") pod \"image-registry-b9885dbc7-lx8h7\" (UID: \"78996163-d007-4f3c-ac3a-44902fe9be66\") " pod="openshift-image-registry/image-registry-b9885dbc7-lx8h7" Jul 07 22:04:13.082028 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:13.081968 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/78996163-d007-4f3c-ac3a-44902fe9be66-ca-trust-extracted\") pod \"image-registry-b9885dbc7-lx8h7\" (UID: \"78996163-d007-4f3c-ac3a-44902fe9be66\") " pod="openshift-image-registry/image-registry-b9885dbc7-lx8h7" Jul 07 22:04:13.082366 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:13.082035 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c80e0cce-1240-4a8f-873b-abd71e85b59a-tmp-dir\") pod \"dns-default-bqv92\" (UID: \"c80e0cce-1240-4a8f-873b-abd71e85b59a\") " pod="openshift-dns/dns-default-bqv92" Jul 07 22:04:13.082366 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:13.082103 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/78996163-d007-4f3c-ac3a-44902fe9be66-installation-pull-secrets\") pod \"image-registry-b9885dbc7-lx8h7\" (UID: \"78996163-d007-4f3c-ac3a-44902fe9be66\") " pod="openshift-image-registry/image-registry-b9885dbc7-lx8h7" Jul 07 22:04:13.082366 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:13.082133 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2jnct\" (UniqueName: \"kubernetes.io/projected/230f7860-610d-430a-987b-6dfaa258d4e7-kube-api-access-2jnct\") pod \"ingress-canary-wrlkn\" (UID: \"230f7860-610d-430a-987b-6dfaa258d4e7\") " pod="openshift-ingress-canary/ingress-canary-wrlkn" Jul 07 22:04:13.183440 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:13.183351 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/78996163-d007-4f3c-ac3a-44902fe9be66-image-registry-private-configuration\") pod \"image-registry-b9885dbc7-lx8h7\" (UID: \"78996163-d007-4f3c-ac3a-44902fe9be66\") " pod="openshift-image-registry/image-registry-b9885dbc7-lx8h7" Jul 07 22:04:13.183440 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:13.183397 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x7txf\" (UniqueName: \"kubernetes.io/projected/c80e0cce-1240-4a8f-873b-abd71e85b59a-kube-api-access-x7txf\") pod \"dns-default-bqv92\" (UID: \"c80e0cce-1240-4a8f-873b-abd71e85b59a\") " pod="openshift-dns/dns-default-bqv92" Jul 07 22:04:13.183648 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:13.183444 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/78996163-d007-4f3c-ac3a-44902fe9be66-trusted-ca\") pod \"image-registry-b9885dbc7-lx8h7\" (UID: \"78996163-d007-4f3c-ac3a-44902fe9be66\") " pod="openshift-image-registry/image-registry-b9885dbc7-lx8h7" Jul 07 22:04:13.183648 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:13.183467 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/78996163-d007-4f3c-ac3a-44902fe9be66-bound-sa-token\") pod \"image-registry-b9885dbc7-lx8h7\" (UID: \"78996163-d007-4f3c-ac3a-44902fe9be66\") " pod="openshift-image-registry/image-registry-b9885dbc7-lx8h7" Jul 07 22:04:13.183648 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:13.183519 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9xtqm\" (UniqueName: \"kubernetes.io/projected/78996163-d007-4f3c-ac3a-44902fe9be66-kube-api-access-9xtqm\") pod \"image-registry-b9885dbc7-lx8h7\" (UID: \"78996163-d007-4f3c-ac3a-44902fe9be66\") " pod="openshift-image-registry/image-registry-b9885dbc7-lx8h7" Jul 07 22:04:13.183648 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:13.183578 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/78996163-d007-4f3c-ac3a-44902fe9be66-registry-certificates\") pod \"image-registry-b9885dbc7-lx8h7\" (UID: \"78996163-d007-4f3c-ac3a-44902fe9be66\") " pod="openshift-image-registry/image-registry-b9885dbc7-lx8h7" Jul 07 22:04:13.183648 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:13.183611 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c80e0cce-1240-4a8f-873b-abd71e85b59a-config-volume\") pod \"dns-default-bqv92\" (UID: \"c80e0cce-1240-4a8f-873b-abd71e85b59a\") " pod="openshift-dns/dns-default-bqv92" Jul 07 22:04:13.183648 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:13.183637 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c80e0cce-1240-4a8f-873b-abd71e85b59a-metrics-tls\") pod \"dns-default-bqv92\" (UID: \"c80e0cce-1240-4a8f-873b-abd71e85b59a\") " pod="openshift-dns/dns-default-bqv92" Jul 07 22:04:13.183916 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:13.183661 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/230f7860-610d-430a-987b-6dfaa258d4e7-cert\") pod \"ingress-canary-wrlkn\" (UID: \"230f7860-610d-430a-987b-6dfaa258d4e7\") " pod="openshift-ingress-canary/ingress-canary-wrlkn" Jul 07 22:04:13.183916 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:13.183697 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/78996163-d007-4f3c-ac3a-44902fe9be66-registry-tls\") pod \"image-registry-b9885dbc7-lx8h7\" (UID: \"78996163-d007-4f3c-ac3a-44902fe9be66\") " pod="openshift-image-registry/image-registry-b9885dbc7-lx8h7" Jul 07 22:04:13.183916 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:13.183719 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/78996163-d007-4f3c-ac3a-44902fe9be66-ca-trust-extracted\") pod \"image-registry-b9885dbc7-lx8h7\" (UID: \"78996163-d007-4f3c-ac3a-44902fe9be66\") " pod="openshift-image-registry/image-registry-b9885dbc7-lx8h7" Jul 07 22:04:13.183916 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:13.183749 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c80e0cce-1240-4a8f-873b-abd71e85b59a-tmp-dir\") pod \"dns-default-bqv92\" (UID: \"c80e0cce-1240-4a8f-873b-abd71e85b59a\") " pod="openshift-dns/dns-default-bqv92" Jul 07 22:04:13.183916 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:13.183782 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/78996163-d007-4f3c-ac3a-44902fe9be66-installation-pull-secrets\") pod \"image-registry-b9885dbc7-lx8h7\" (UID: \"78996163-d007-4f3c-ac3a-44902fe9be66\") " pod="openshift-image-registry/image-registry-b9885dbc7-lx8h7" Jul 07 22:04:13.183916 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:04:13.183831 2545 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 22:04:13.183916 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:04:13.183890 2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c80e0cce-1240-4a8f-873b-abd71e85b59a-metrics-tls podName:c80e0cce-1240-4a8f-873b-abd71e85b59a nodeName:}" failed. No retries permitted until 2026-07-07 22:04:13.683869522 +0000 UTC m=+35.998092534 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c80e0cce-1240-4a8f-873b-abd71e85b59a-metrics-tls") pod "dns-default-bqv92" (UID: "c80e0cce-1240-4a8f-873b-abd71e85b59a") : secret "dns-default-metrics-tls" not found Jul 07 22:04:13.183916 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:04:13.183902 2545 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 07 22:04:13.183916 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:04:13.183918 2545 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-b9885dbc7-lx8h7: secret "image-registry-tls" not found Jul 07 22:04:13.184372 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:04:13.183979 2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/78996163-d007-4f3c-ac3a-44902fe9be66-registry-tls podName:78996163-d007-4f3c-ac3a-44902fe9be66 nodeName:}" failed. No retries permitted until 2026-07-07 22:04:13.683961668 +0000 UTC m=+35.998184676 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/78996163-d007-4f3c-ac3a-44902fe9be66-registry-tls") pod "image-registry-b9885dbc7-lx8h7" (UID: "78996163-d007-4f3c-ac3a-44902fe9be66") : secret "image-registry-tls" not found Jul 07 22:04:13.184372 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:13.184019 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2jnct\" (UniqueName: \"kubernetes.io/projected/230f7860-610d-430a-987b-6dfaa258d4e7-kube-api-access-2jnct\") pod \"ingress-canary-wrlkn\" (UID: \"230f7860-610d-430a-987b-6dfaa258d4e7\") " pod="openshift-ingress-canary/ingress-canary-wrlkn" Jul 07 22:04:13.184512 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:13.184489 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/78996163-d007-4f3c-ac3a-44902fe9be66-trusted-ca\") pod \"image-registry-b9885dbc7-lx8h7\" (UID: \"78996163-d007-4f3c-ac3a-44902fe9be66\") " pod="openshift-image-registry/image-registry-b9885dbc7-lx8h7" Jul 07 22:04:13.184633 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:13.184609 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c80e0cce-1240-4a8f-873b-abd71e85b59a-config-volume\") pod \"dns-default-bqv92\" (UID: \"c80e0cce-1240-4a8f-873b-abd71e85b59a\") " pod="openshift-dns/dns-default-bqv92" Jul 07 22:04:13.184740 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:13.184714 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/78996163-d007-4f3c-ac3a-44902fe9be66-ca-trust-extracted\") pod \"image-registry-b9885dbc7-lx8h7\" (UID: \"78996163-d007-4f3c-ac3a-44902fe9be66\") " pod="openshift-image-registry/image-registry-b9885dbc7-lx8h7" Jul 07 22:04:13.184788 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:13.184748 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/78996163-d007-4f3c-ac3a-44902fe9be66-registry-certificates\") pod \"image-registry-b9885dbc7-lx8h7\" (UID: \"78996163-d007-4f3c-ac3a-44902fe9be66\") " pod="openshift-image-registry/image-registry-b9885dbc7-lx8h7" Jul 07 22:04:13.184788 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:13.184758 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c80e0cce-1240-4a8f-873b-abd71e85b59a-tmp-dir\") pod \"dns-default-bqv92\" (UID: \"c80e0cce-1240-4a8f-873b-abd71e85b59a\") " pod="openshift-dns/dns-default-bqv92" Jul 07 22:04:13.188674 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:13.188646 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/78996163-d007-4f3c-ac3a-44902fe9be66-image-registry-private-configuration\") pod \"image-registry-b9885dbc7-lx8h7\" (UID: \"78996163-d007-4f3c-ac3a-44902fe9be66\") " pod="openshift-image-registry/image-registry-b9885dbc7-lx8h7" Jul 07 22:04:13.188779 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:13.188652 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/230f7860-610d-430a-987b-6dfaa258d4e7-cert\") pod \"ingress-canary-wrlkn\" (UID: \"230f7860-610d-430a-987b-6dfaa258d4e7\") " pod="openshift-ingress-canary/ingress-canary-wrlkn" Jul 07 22:04:13.188779 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:13.188686 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/78996163-d007-4f3c-ac3a-44902fe9be66-installation-pull-secrets\") pod \"image-registry-b9885dbc7-lx8h7\" (UID: \"78996163-d007-4f3c-ac3a-44902fe9be66\") " pod="openshift-image-registry/image-registry-b9885dbc7-lx8h7" Jul 07 22:04:13.192770 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:13.192744 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/78996163-d007-4f3c-ac3a-44902fe9be66-bound-sa-token\") pod \"image-registry-b9885dbc7-lx8h7\" (UID: \"78996163-d007-4f3c-ac3a-44902fe9be66\") " pod="openshift-image-registry/image-registry-b9885dbc7-lx8h7" Jul 07 22:04:13.192901 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:13.192877 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9xtqm\" (UniqueName: \"kubernetes.io/projected/78996163-d007-4f3c-ac3a-44902fe9be66-kube-api-access-9xtqm\") pod \"image-registry-b9885dbc7-lx8h7\" (UID: \"78996163-d007-4f3c-ac3a-44902fe9be66\") " pod="openshift-image-registry/image-registry-b9885dbc7-lx8h7" Jul 07 22:04:13.193198 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:13.193174 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x7txf\" (UniqueName: \"kubernetes.io/projected/c80e0cce-1240-4a8f-873b-abd71e85b59a-kube-api-access-x7txf\") pod \"dns-default-bqv92\" (UID: \"c80e0cce-1240-4a8f-873b-abd71e85b59a\") " pod="openshift-dns/dns-default-bqv92" Jul 07 22:04:13.193398 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:13.193381 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2jnct\" (UniqueName: \"kubernetes.io/projected/230f7860-610d-430a-987b-6dfaa258d4e7-kube-api-access-2jnct\") pod \"ingress-canary-wrlkn\" (UID: \"230f7860-610d-430a-987b-6dfaa258d4e7\") " pod="openshift-ingress-canary/ingress-canary-wrlkn" Jul 07 22:04:13.252701 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:13.252671 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-wrlkn" Jul 07 22:04:13.305158 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:13.305120 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nqfk8" Jul 07 22:04:13.305330 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:13.305162 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-82xqf" Jul 07 22:04:13.305330 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:13.305242 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-88qj9" Jul 07 22:04:13.308279 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:13.307929 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-p2w6w\"" Jul 07 22:04:13.308279 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:13.308060 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 07 22:04:13.308279 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:13.308093 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 07 22:04:13.308279 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:13.308152 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-hdlmc\"" Jul 07 22:04:13.308279 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:13.308170 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 07 22:04:13.308279 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:13.308225 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 07 22:04:13.396222 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:13.396198 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-wrlkn"] Jul 07 22:04:13.687986 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:13.687947 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c80e0cce-1240-4a8f-873b-abd71e85b59a-metrics-tls\") pod \"dns-default-bqv92\" (UID: \"c80e0cce-1240-4a8f-873b-abd71e85b59a\") " pod="openshift-dns/dns-default-bqv92" Jul 07 22:04:13.688177 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:13.687998 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/78996163-d007-4f3c-ac3a-44902fe9be66-registry-tls\") pod \"image-registry-b9885dbc7-lx8h7\" (UID: \"78996163-d007-4f3c-ac3a-44902fe9be66\") " pod="openshift-image-registry/image-registry-b9885dbc7-lx8h7" Jul 07 22:04:13.688177 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:04:13.688125 2545 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 22:04:13.688305 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:04:13.688186 2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c80e0cce-1240-4a8f-873b-abd71e85b59a-metrics-tls podName:c80e0cce-1240-4a8f-873b-abd71e85b59a nodeName:}" failed. No retries permitted until 2026-07-07 22:04:14.688170429 +0000 UTC m=+37.002393429 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c80e0cce-1240-4a8f-873b-abd71e85b59a-metrics-tls") pod "dns-default-bqv92" (UID: "c80e0cce-1240-4a8f-873b-abd71e85b59a") : secret "dns-default-metrics-tls" not found Jul 07 22:04:13.688305 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:04:13.688129 2545 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 07 22:04:13.688305 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:04:13.688221 2545 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-b9885dbc7-lx8h7: secret "image-registry-tls" not found Jul 07 22:04:13.688305 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:04:13.688261 2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/78996163-d007-4f3c-ac3a-44902fe9be66-registry-tls podName:78996163-d007-4f3c-ac3a-44902fe9be66 nodeName:}" failed. No retries permitted until 2026-07-07 22:04:14.688249902 +0000 UTC m=+37.002472914 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/78996163-d007-4f3c-ac3a-44902fe9be66-registry-tls") pod "image-registry-b9885dbc7-lx8h7" (UID: "78996163-d007-4f3c-ac3a-44902fe9be66") : secret "image-registry-tls" not found Jul 07 22:04:14.697293 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:14.697257 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c80e0cce-1240-4a8f-873b-abd71e85b59a-metrics-tls\") pod \"dns-default-bqv92\" (UID: \"c80e0cce-1240-4a8f-873b-abd71e85b59a\") " pod="openshift-dns/dns-default-bqv92" Jul 07 22:04:14.697767 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:14.697304 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/78996163-d007-4f3c-ac3a-44902fe9be66-registry-tls\") pod \"image-registry-b9885dbc7-lx8h7\" (UID: \"78996163-d007-4f3c-ac3a-44902fe9be66\") " pod="openshift-image-registry/image-registry-b9885dbc7-lx8h7" Jul 07 22:04:14.697767 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:04:14.697439 2545 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 07 22:04:14.697767 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:04:14.697443 2545 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 22:04:14.697767 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:04:14.697451 2545 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-b9885dbc7-lx8h7: secret "image-registry-tls" not found Jul 07 22:04:14.697767 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:04:14.697546 2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c80e0cce-1240-4a8f-873b-abd71e85b59a-metrics-tls podName:c80e0cce-1240-4a8f-873b-abd71e85b59a nodeName:}" failed. No retries permitted until 2026-07-07 22:04:16.697523126 +0000 UTC m=+39.011746140 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c80e0cce-1240-4a8f-873b-abd71e85b59a-metrics-tls") pod "dns-default-bqv92" (UID: "c80e0cce-1240-4a8f-873b-abd71e85b59a") : secret "dns-default-metrics-tls" not found Jul 07 22:04:14.697767 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:04:14.697603 2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/78996163-d007-4f3c-ac3a-44902fe9be66-registry-tls podName:78996163-d007-4f3c-ac3a-44902fe9be66 nodeName:}" failed. No retries permitted until 2026-07-07 22:04:16.697590229 +0000 UTC m=+39.011813226 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/78996163-d007-4f3c-ac3a-44902fe9be66-registry-tls") pod "image-registry-b9885dbc7-lx8h7" (UID: "78996163-d007-4f3c-ac3a-44902fe9be66") : secret "image-registry-tls" not found Jul 07 22:04:14.829043 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:04:14.829004 2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod230f7860_610d_430a_987b_6dfaa258d4e7.slice/crio-902b3376acca70725c3aa64a6f8a93f067735315473ddcd4e94979626beda3bd WatchSource:0}: Error finding container 902b3376acca70725c3aa64a6f8a93f067735315473ddcd4e94979626beda3bd: Status 404 returned error can't find the container with id 902b3376acca70725c3aa64a6f8a93f067735315473ddcd4e94979626beda3bd Jul 07 22:04:15.483534 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:15.483495 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-wrlkn" event={"ID":"230f7860-610d-430a-987b-6dfaa258d4e7","Type":"ContainerStarted","Data":"902b3376acca70725c3aa64a6f8a93f067735315473ddcd4e94979626beda3bd"} Jul 07 22:04:15.486009 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:15.485987 2545 generic.go:356] "Generic (PLEG): container finished" podID="00baab1c-b8e0-4b5e-bd42-647ba72c5aca" containerID="febfd31a4743020d6b36d88ebdf8898bf34710a181173f5548004e5a9cde5b20" exitCode=0 Jul 07 22:04:15.486072 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:15.486032 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ztkm2" event={"ID":"00baab1c-b8e0-4b5e-bd42-647ba72c5aca","Type":"ContainerDied","Data":"febfd31a4743020d6b36d88ebdf8898bf34710a181173f5548004e5a9cde5b20"} Jul 07 22:04:15.722396 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:15.722360 2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-82vxn"] Jul 07 22:04:15.724692 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:15.724665 2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-fjt2g"] Jul 07 22:04:15.724828 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:15.724810 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-82vxn" Jul 07 22:04:15.727443 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:15.727420 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-fjt2g" Jul 07 22:04:15.727957 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:15.727933 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 07 22:04:15.728061 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:15.727986 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-m6hr8\"" Jul 07 22:04:15.728061 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:15.727994 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 07 22:04:15.729737 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:15.729719 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 07 22:04:15.729737 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:15.729727 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 07 22:04:15.729921 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:15.729762 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 07 22:04:15.729921 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:15.729818 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-s47bj\"" Jul 07 22:04:15.735735 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:15.735673 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-82vxn"] Jul 07 22:04:15.736397 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:15.736378 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-fjt2g"] Jul 07 22:04:15.805927 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:15.805892 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qzfgt\" (UniqueName: \"kubernetes.io/projected/84813585-7321-4216-bab4-9258b9ecff6a-kube-api-access-qzfgt\") pod \"volume-data-source-validator-5d456fdbcd-82vxn\" (UID: \"84813585-7321-4216-bab4-9258b9ecff6a\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-82vxn" Jul 07 22:04:15.806114 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:15.806002 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7qtfw\" (UniqueName: \"kubernetes.io/projected/5398d2ac-a4ba-4c50-bdf4-94de0a0b284e-kube-api-access-7qtfw\") pod \"cluster-samples-operator-56fcf86bf-fjt2g\" (UID: \"5398d2ac-a4ba-4c50-bdf4-94de0a0b284e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-fjt2g" Jul 07 22:04:15.806114 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:15.806044 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5398d2ac-a4ba-4c50-bdf4-94de0a0b284e-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-fjt2g\" (UID: \"5398d2ac-a4ba-4c50-bdf4-94de0a0b284e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-fjt2g" Jul 07 22:04:15.818927 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:15.818872 2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wt5nz"] Jul 07 22:04:15.821033 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:15.821011 2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-rhg5r"] Jul 07 22:04:15.821166 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:15.821148 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wt5nz" Jul 07 22:04:15.822935 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:15.822914 2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-67496c4dcc-nnvc2"] Jul 07 22:04:15.823052 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:15.823030 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-rhg5r" Jul 07 22:04:15.824350 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:15.824318 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 07 22:04:15.824350 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:15.824338 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 07 22:04:15.824892 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:15.824734 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 07 22:04:15.824892 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:15.824786 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 07 22:04:15.826724 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:15.826107 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-67496c4dcc-nnvc2" Jul 07 22:04:15.826724 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:15.826220 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-p6w9g\"" Jul 07 22:04:15.826724 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:15.826445 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 07 22:04:15.826724 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:15.826468 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 07 22:04:15.826724 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:15.826615 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 07 22:04:15.827012 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:15.826920 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 07 22:04:15.828609 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:15.828587 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-knlk4\"" Jul 07 22:04:15.829774 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:15.829621 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 07 22:04:15.829774 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:15.829631 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 07 22:04:15.829774 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:15.829741 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 07 22:04:15.830747 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:15.830655 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-747vt\"" Jul 07 22:04:15.830747 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:15.830670 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 07 22:04:15.831072 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:15.831055 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 07 22:04:15.831235 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:15.831096 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 07 22:04:15.832116 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:15.832097 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wt5nz"] Jul 07 22:04:15.835795 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:15.835777 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 07 22:04:15.836786 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:15.836763 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-rhg5r"] Jul 07 22:04:15.837895 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:15.837876 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-67496c4dcc-nnvc2"] Jul 07 22:04:15.906580 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:15.906547 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/71f36bfa-4b8a-4572-b522-57c16392aa0b-config\") pod \"console-operator-68b757fb67-rhg5r\" (UID: \"71f36bfa-4b8a-4572-b522-57c16392aa0b\") " pod="openshift-console-operator/console-operator-68b757fb67-rhg5r" Jul 07 22:04:15.906580 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:15.906580 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-97rcp\" (UniqueName: \"kubernetes.io/projected/71f36bfa-4b8a-4572-b522-57c16392aa0b-kube-api-access-97rcp\") pod \"console-operator-68b757fb67-rhg5r\" (UID: \"71f36bfa-4b8a-4572-b522-57c16392aa0b\") " pod="openshift-console-operator/console-operator-68b757fb67-rhg5r" Jul 07 22:04:15.906810 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:15.906606 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/84447fdf-4f2d-45fd-9e94-acce197ae75b-service-ca-bundle\") pod \"router-default-67496c4dcc-nnvc2\" (UID: \"84447fdf-4f2d-45fd-9e94-acce197ae75b\") " pod="openshift-ingress/router-default-67496c4dcc-nnvc2" Jul 07 22:04:15.906810 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:15.906658 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5398d2ac-a4ba-4c50-bdf4-94de0a0b284e-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-fjt2g\" (UID: \"5398d2ac-a4ba-4c50-bdf4-94de0a0b284e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-fjt2g" Jul 07 22:04:15.906810 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:04:15.906720 2545 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 07 22:04:15.906810 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:15.906729 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/84447fdf-4f2d-45fd-9e94-acce197ae75b-default-certificate\") pod \"router-default-67496c4dcc-nnvc2\" (UID: \"84447fdf-4f2d-45fd-9e94-acce197ae75b\") " pod="openshift-ingress/router-default-67496c4dcc-nnvc2" Jul 07 22:04:15.906810 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:15.906755 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/71f36bfa-4b8a-4572-b522-57c16392aa0b-serving-cert\") pod \"console-operator-68b757fb67-rhg5r\" (UID: \"71f36bfa-4b8a-4572-b522-57c16392aa0b\") " pod="openshift-console-operator/console-operator-68b757fb67-rhg5r" Jul 07 22:04:15.906810 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:04:15.906780 2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5398d2ac-a4ba-4c50-bdf4-94de0a0b284e-samples-operator-tls podName:5398d2ac-a4ba-4c50-bdf4-94de0a0b284e nodeName:}" failed. No retries permitted until 2026-07-07 22:04:16.40676497 +0000 UTC m=+38.720987970 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/5398d2ac-a4ba-4c50-bdf4-94de0a0b284e-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-fjt2g" (UID: "5398d2ac-a4ba-4c50-bdf4-94de0a0b284e") : secret "samples-operator-tls" not found Jul 07 22:04:15.907070 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:15.906832 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/71f36bfa-4b8a-4572-b522-57c16392aa0b-trusted-ca\") pod \"console-operator-68b757fb67-rhg5r\" (UID: \"71f36bfa-4b8a-4572-b522-57c16392aa0b\") " pod="openshift-console-operator/console-operator-68b757fb67-rhg5r" Jul 07 22:04:15.907070 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:15.906875 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qzfgt\" (UniqueName: \"kubernetes.io/projected/84813585-7321-4216-bab4-9258b9ecff6a-kube-api-access-qzfgt\") pod \"volume-data-source-validator-5d456fdbcd-82vxn\" (UID: \"84813585-7321-4216-bab4-9258b9ecff6a\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-82vxn" Jul 07 22:04:15.907070 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:15.906910 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/01d309f5-5c78-4908-a057-7dcb433b7f3b-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-wt5nz\" (UID: \"01d309f5-5c78-4908-a057-7dcb433b7f3b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wt5nz" Jul 07 22:04:15.907070 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:15.906954 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/84447fdf-4f2d-45fd-9e94-acce197ae75b-metrics-certs\") pod \"router-default-67496c4dcc-nnvc2\" (UID: \"84447fdf-4f2d-45fd-9e94-acce197ae75b\") " pod="openshift-ingress/router-default-67496c4dcc-nnvc2" Jul 07 22:04:15.907070 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:15.906995 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jk92d\" (UniqueName: \"kubernetes.io/projected/01d309f5-5c78-4908-a057-7dcb433b7f3b-kube-api-access-jk92d\") pod \"kube-storage-version-migrator-operator-846bf668f7-wt5nz\" (UID: \"01d309f5-5c78-4908-a057-7dcb433b7f3b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wt5nz" Jul 07 22:04:15.907070 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:15.907032 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7qtfw\" (UniqueName: \"kubernetes.io/projected/5398d2ac-a4ba-4c50-bdf4-94de0a0b284e-kube-api-access-7qtfw\") pod \"cluster-samples-operator-56fcf86bf-fjt2g\" (UID: \"5398d2ac-a4ba-4c50-bdf4-94de0a0b284e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-fjt2g" Jul 07 22:04:15.907070 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:15.907067 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/01d309f5-5c78-4908-a057-7dcb433b7f3b-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-wt5nz\" (UID: \"01d309f5-5c78-4908-a057-7dcb433b7f3b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wt5nz" Jul 07 22:04:15.907408 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:15.907117 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/84447fdf-4f2d-45fd-9e94-acce197ae75b-stats-auth\") pod \"router-default-67496c4dcc-nnvc2\" (UID: \"84447fdf-4f2d-45fd-9e94-acce197ae75b\") " pod="openshift-ingress/router-default-67496c4dcc-nnvc2" Jul 07 22:04:15.907408 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:15.907169 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bg9m6\" (UniqueName: \"kubernetes.io/projected/84447fdf-4f2d-45fd-9e94-acce197ae75b-kube-api-access-bg9m6\") pod \"router-default-67496c4dcc-nnvc2\" (UID: \"84447fdf-4f2d-45fd-9e94-acce197ae75b\") " pod="openshift-ingress/router-default-67496c4dcc-nnvc2" Jul 07 22:04:15.918219 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:15.918186 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7qtfw\" (UniqueName: \"kubernetes.io/projected/5398d2ac-a4ba-4c50-bdf4-94de0a0b284e-kube-api-access-7qtfw\") pod \"cluster-samples-operator-56fcf86bf-fjt2g\" (UID: \"5398d2ac-a4ba-4c50-bdf4-94de0a0b284e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-fjt2g" Jul 07 22:04:15.918359 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:15.918269 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qzfgt\" (UniqueName: \"kubernetes.io/projected/84813585-7321-4216-bab4-9258b9ecff6a-kube-api-access-qzfgt\") pod \"volume-data-source-validator-5d456fdbcd-82vxn\" (UID: \"84813585-7321-4216-bab4-9258b9ecff6a\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-82vxn" Jul 07 22:04:16.008271 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:16.008196 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/84447fdf-4f2d-45fd-9e94-acce197ae75b-default-certificate\") pod \"router-default-67496c4dcc-nnvc2\" (UID: \"84447fdf-4f2d-45fd-9e94-acce197ae75b\") " pod="openshift-ingress/router-default-67496c4dcc-nnvc2" Jul 07 22:04:16.008271 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:16.008231 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/71f36bfa-4b8a-4572-b522-57c16392aa0b-serving-cert\") pod \"console-operator-68b757fb67-rhg5r\" (UID: \"71f36bfa-4b8a-4572-b522-57c16392aa0b\") " pod="openshift-console-operator/console-operator-68b757fb67-rhg5r" Jul 07 22:04:16.008271 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:16.008258 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/71f36bfa-4b8a-4572-b522-57c16392aa0b-trusted-ca\") pod \"console-operator-68b757fb67-rhg5r\" (UID: \"71f36bfa-4b8a-4572-b522-57c16392aa0b\") " pod="openshift-console-operator/console-operator-68b757fb67-rhg5r" Jul 07 22:04:16.008532 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:16.008298 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/01d309f5-5c78-4908-a057-7dcb433b7f3b-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-wt5nz\" (UID: \"01d309f5-5c78-4908-a057-7dcb433b7f3b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wt5nz" Jul 07 22:04:16.008532 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:16.008338 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/84447fdf-4f2d-45fd-9e94-acce197ae75b-metrics-certs\") pod \"router-default-67496c4dcc-nnvc2\" (UID: \"84447fdf-4f2d-45fd-9e94-acce197ae75b\") " pod="openshift-ingress/router-default-67496c4dcc-nnvc2" Jul 07 22:04:16.008532 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:16.008367 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jk92d\" (UniqueName: \"kubernetes.io/projected/01d309f5-5c78-4908-a057-7dcb433b7f3b-kube-api-access-jk92d\") pod \"kube-storage-version-migrator-operator-846bf668f7-wt5nz\" (UID: \"01d309f5-5c78-4908-a057-7dcb433b7f3b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wt5nz" Jul 07 22:04:16.008532 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:16.008424 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/01d309f5-5c78-4908-a057-7dcb433b7f3b-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-wt5nz\" (UID: \"01d309f5-5c78-4908-a057-7dcb433b7f3b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wt5nz" Jul 07 22:04:16.008532 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:16.008469 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/84447fdf-4f2d-45fd-9e94-acce197ae75b-stats-auth\") pod \"router-default-67496c4dcc-nnvc2\" (UID: \"84447fdf-4f2d-45fd-9e94-acce197ae75b\") " pod="openshift-ingress/router-default-67496c4dcc-nnvc2" Jul 07 22:04:16.008532 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:16.008501 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bg9m6\" (UniqueName: \"kubernetes.io/projected/84447fdf-4f2d-45fd-9e94-acce197ae75b-kube-api-access-bg9m6\") pod \"router-default-67496c4dcc-nnvc2\" (UID: \"84447fdf-4f2d-45fd-9e94-acce197ae75b\") " pod="openshift-ingress/router-default-67496c4dcc-nnvc2" Jul 07 22:04:16.008532 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:16.008534 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/71f36bfa-4b8a-4572-b522-57c16392aa0b-config\") pod \"console-operator-68b757fb67-rhg5r\" (UID: \"71f36bfa-4b8a-4572-b522-57c16392aa0b\") " pod="openshift-console-operator/console-operator-68b757fb67-rhg5r" Jul 07 22:04:16.008888 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:16.008557 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-97rcp\" (UniqueName: \"kubernetes.io/projected/71f36bfa-4b8a-4572-b522-57c16392aa0b-kube-api-access-97rcp\") pod \"console-operator-68b757fb67-rhg5r\" (UID: \"71f36bfa-4b8a-4572-b522-57c16392aa0b\") " pod="openshift-console-operator/console-operator-68b757fb67-rhg5r" Jul 07 22:04:16.008888 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:16.008587 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/84447fdf-4f2d-45fd-9e94-acce197ae75b-service-ca-bundle\") pod \"router-default-67496c4dcc-nnvc2\" (UID: \"84447fdf-4f2d-45fd-9e94-acce197ae75b\") " pod="openshift-ingress/router-default-67496c4dcc-nnvc2" Jul 07 22:04:16.008888 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:04:16.008740 2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/84447fdf-4f2d-45fd-9e94-acce197ae75b-service-ca-bundle podName:84447fdf-4f2d-45fd-9e94-acce197ae75b nodeName:}" failed. No retries permitted until 2026-07-07 22:04:16.508722974 +0000 UTC m=+38.822945974 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/84447fdf-4f2d-45fd-9e94-acce197ae75b-service-ca-bundle") pod "router-default-67496c4dcc-nnvc2" (UID: "84447fdf-4f2d-45fd-9e94-acce197ae75b") : configmap references non-existent config key: service-ca.crt Jul 07 22:04:16.009035 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:16.008953 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/01d309f5-5c78-4908-a057-7dcb433b7f3b-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-wt5nz\" (UID: \"01d309f5-5c78-4908-a057-7dcb433b7f3b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wt5nz" Jul 07 22:04:16.009330 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:04:16.009220 2545 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 07 22:04:16.009330 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:04:16.009280 2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/84447fdf-4f2d-45fd-9e94-acce197ae75b-metrics-certs podName:84447fdf-4f2d-45fd-9e94-acce197ae75b nodeName:}" failed. No retries permitted until 2026-07-07 22:04:16.509261127 +0000 UTC m=+38.823484124 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/84447fdf-4f2d-45fd-9e94-acce197ae75b-metrics-certs") pod "router-default-67496c4dcc-nnvc2" (UID: "84447fdf-4f2d-45fd-9e94-acce197ae75b") : secret "router-metrics-certs-default" not found Jul 07 22:04:16.009330 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:16.009274 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/71f36bfa-4b8a-4572-b522-57c16392aa0b-trusted-ca\") pod \"console-operator-68b757fb67-rhg5r\" (UID: \"71f36bfa-4b8a-4572-b522-57c16392aa0b\") " pod="openshift-console-operator/console-operator-68b757fb67-rhg5r" Jul 07 22:04:16.011074 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:16.009663 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/71f36bfa-4b8a-4572-b522-57c16392aa0b-config\") pod \"console-operator-68b757fb67-rhg5r\" (UID: \"71f36bfa-4b8a-4572-b522-57c16392aa0b\") " pod="openshift-console-operator/console-operator-68b757fb67-rhg5r" Jul 07 22:04:16.012226 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:16.012131 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/84447fdf-4f2d-45fd-9e94-acce197ae75b-default-certificate\") pod \"router-default-67496c4dcc-nnvc2\" (UID: \"84447fdf-4f2d-45fd-9e94-acce197ae75b\") " pod="openshift-ingress/router-default-67496c4dcc-nnvc2" Jul 07 22:04:16.012659 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:16.012616 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/01d309f5-5c78-4908-a057-7dcb433b7f3b-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-wt5nz\" (UID: \"01d309f5-5c78-4908-a057-7dcb433b7f3b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wt5nz" Jul 07 22:04:16.012899 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:16.012859 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/71f36bfa-4b8a-4572-b522-57c16392aa0b-serving-cert\") pod \"console-operator-68b757fb67-rhg5r\" (UID: \"71f36bfa-4b8a-4572-b522-57c16392aa0b\") " pod="openshift-console-operator/console-operator-68b757fb67-rhg5r" Jul 07 22:04:16.014106 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:16.013739 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/84447fdf-4f2d-45fd-9e94-acce197ae75b-stats-auth\") pod \"router-default-67496c4dcc-nnvc2\" (UID: \"84447fdf-4f2d-45fd-9e94-acce197ae75b\") " pod="openshift-ingress/router-default-67496c4dcc-nnvc2" Jul 07 22:04:16.018034 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:16.018011 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jk92d\" (UniqueName: \"kubernetes.io/projected/01d309f5-5c78-4908-a057-7dcb433b7f3b-kube-api-access-jk92d\") pod \"kube-storage-version-migrator-operator-846bf668f7-wt5nz\" (UID: \"01d309f5-5c78-4908-a057-7dcb433b7f3b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wt5nz" Jul 07 22:04:16.018316 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:16.018294 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-97rcp\" (UniqueName: \"kubernetes.io/projected/71f36bfa-4b8a-4572-b522-57c16392aa0b-kube-api-access-97rcp\") pod \"console-operator-68b757fb67-rhg5r\" (UID: \"71f36bfa-4b8a-4572-b522-57c16392aa0b\") " pod="openshift-console-operator/console-operator-68b757fb67-rhg5r" Jul 07 22:04:16.018883 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:16.018842 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bg9m6\" (UniqueName: \"kubernetes.io/projected/84447fdf-4f2d-45fd-9e94-acce197ae75b-kube-api-access-bg9m6\") pod \"router-default-67496c4dcc-nnvc2\" (UID: \"84447fdf-4f2d-45fd-9e94-acce197ae75b\") " pod="openshift-ingress/router-default-67496c4dcc-nnvc2" Jul 07 22:04:16.037302 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:16.037279 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-82vxn" Jul 07 22:04:16.137515 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:16.137480 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wt5nz" Jul 07 22:04:16.145560 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:16.145531 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-rhg5r" Jul 07 22:04:16.179579 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:16.179514 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-82vxn"] Jul 07 22:04:16.185098 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:04:16.185053 2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84813585_7321_4216_bab4_9258b9ecff6a.slice/crio-876d6f58678217b14f9110ea8fe632e94221059a1d0696ae2ee6b9bf2b7451cb WatchSource:0}: Error finding container 876d6f58678217b14f9110ea8fe632e94221059a1d0696ae2ee6b9bf2b7451cb: Status 404 returned error can't find the container with id 876d6f58678217b14f9110ea8fe632e94221059a1d0696ae2ee6b9bf2b7451cb Jul 07 22:04:16.310335 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:16.310304 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-rhg5r"] Jul 07 22:04:16.313371 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:04:16.313344 2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71f36bfa_4b8a_4572_b522_57c16392aa0b.slice/crio-0cf4d776e67eac0916437c18eb04e49d138ae875b1dca8d79b84ffd729f66c30 WatchSource:0}: Error finding container 0cf4d776e67eac0916437c18eb04e49d138ae875b1dca8d79b84ffd729f66c30: Status 404 returned error can't find the container with id 0cf4d776e67eac0916437c18eb04e49d138ae875b1dca8d79b84ffd729f66c30 Jul 07 22:04:16.321409 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:16.321389 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wt5nz"] Jul 07 22:04:16.325183 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:04:16.325144 2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01d309f5_5c78_4908_a057_7dcb433b7f3b.slice/crio-bd2ecb9ccac8bc93a95c35a7b3ab65acb5ce0d3034fa36e8fcc7ffccbea47120 WatchSource:0}: Error finding container bd2ecb9ccac8bc93a95c35a7b3ab65acb5ce0d3034fa36e8fcc7ffccbea47120: Status 404 returned error can't find the container with id bd2ecb9ccac8bc93a95c35a7b3ab65acb5ce0d3034fa36e8fcc7ffccbea47120 Jul 07 22:04:16.415217 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:16.415182 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5398d2ac-a4ba-4c50-bdf4-94de0a0b284e-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-fjt2g\" (UID: \"5398d2ac-a4ba-4c50-bdf4-94de0a0b284e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-fjt2g" Jul 07 22:04:16.415363 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:04:16.415333 2545 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 07 22:04:16.415412 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:04:16.415402 2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5398d2ac-a4ba-4c50-bdf4-94de0a0b284e-samples-operator-tls podName:5398d2ac-a4ba-4c50-bdf4-94de0a0b284e nodeName:}" failed. No retries permitted until 2026-07-07 22:04:17.415385987 +0000 UTC m=+39.729608984 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/5398d2ac-a4ba-4c50-bdf4-94de0a0b284e-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-fjt2g" (UID: "5398d2ac-a4ba-4c50-bdf4-94de0a0b284e") : secret "samples-operator-tls" not found Jul 07 22:04:16.489829 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:16.489788 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-rhg5r" event={"ID":"71f36bfa-4b8a-4572-b522-57c16392aa0b","Type":"ContainerStarted","Data":"0cf4d776e67eac0916437c18eb04e49d138ae875b1dca8d79b84ffd729f66c30"} Jul 07 22:04:16.493293 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:16.493265 2545 generic.go:356] "Generic (PLEG): container finished" podID="00baab1c-b8e0-4b5e-bd42-647ba72c5aca" containerID="b515dcdcb9f438aef90f8ce4a6018a7fd316f139996f9cce23d97dbcc362a855" exitCode=0 Jul 07 22:04:16.493438 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:16.493366 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ztkm2" event={"ID":"00baab1c-b8e0-4b5e-bd42-647ba72c5aca","Type":"ContainerDied","Data":"b515dcdcb9f438aef90f8ce4a6018a7fd316f139996f9cce23d97dbcc362a855"} Jul 07 22:04:16.495029 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:16.494831 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-82vxn" event={"ID":"84813585-7321-4216-bab4-9258b9ecff6a","Type":"ContainerStarted","Data":"876d6f58678217b14f9110ea8fe632e94221059a1d0696ae2ee6b9bf2b7451cb"} Jul 07 22:04:16.496949 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:16.496922 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wt5nz" event={"ID":"01d309f5-5c78-4908-a057-7dcb433b7f3b","Type":"ContainerStarted","Data":"bd2ecb9ccac8bc93a95c35a7b3ab65acb5ce0d3034fa36e8fcc7ffccbea47120"} Jul 07 22:04:16.515817 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:16.515793 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/84447fdf-4f2d-45fd-9e94-acce197ae75b-metrics-certs\") pod \"router-default-67496c4dcc-nnvc2\" (UID: \"84447fdf-4f2d-45fd-9e94-acce197ae75b\") " pod="openshift-ingress/router-default-67496c4dcc-nnvc2" Jul 07 22:04:16.515919 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:16.515861 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/84447fdf-4f2d-45fd-9e94-acce197ae75b-service-ca-bundle\") pod \"router-default-67496c4dcc-nnvc2\" (UID: \"84447fdf-4f2d-45fd-9e94-acce197ae75b\") " pod="openshift-ingress/router-default-67496c4dcc-nnvc2" Jul 07 22:04:16.515992 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:04:16.515958 2545 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 07 22:04:16.515992 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:04:16.515975 2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/84447fdf-4f2d-45fd-9e94-acce197ae75b-service-ca-bundle podName:84447fdf-4f2d-45fd-9e94-acce197ae75b nodeName:}" failed. No retries permitted until 2026-07-07 22:04:17.515956612 +0000 UTC m=+39.830179626 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/84447fdf-4f2d-45fd-9e94-acce197ae75b-service-ca-bundle") pod "router-default-67496c4dcc-nnvc2" (UID: "84447fdf-4f2d-45fd-9e94-acce197ae75b") : configmap references non-existent config key: service-ca.crt Jul 07 22:04:16.516120 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:04:16.516105 2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/84447fdf-4f2d-45fd-9e94-acce197ae75b-metrics-certs podName:84447fdf-4f2d-45fd-9e94-acce197ae75b nodeName:}" failed. No retries permitted until 2026-07-07 22:04:17.516084441 +0000 UTC m=+39.830307441 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/84447fdf-4f2d-45fd-9e94-acce197ae75b-metrics-certs") pod "router-default-67496c4dcc-nnvc2" (UID: "84447fdf-4f2d-45fd-9e94-acce197ae75b") : secret "router-metrics-certs-default" not found Jul 07 22:04:16.717446 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:16.717407 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c80e0cce-1240-4a8f-873b-abd71e85b59a-metrics-tls\") pod \"dns-default-bqv92\" (UID: \"c80e0cce-1240-4a8f-873b-abd71e85b59a\") " pod="openshift-dns/dns-default-bqv92" Jul 07 22:04:16.717631 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:16.717456 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/78996163-d007-4f3c-ac3a-44902fe9be66-registry-tls\") pod \"image-registry-b9885dbc7-lx8h7\" (UID: \"78996163-d007-4f3c-ac3a-44902fe9be66\") " pod="openshift-image-registry/image-registry-b9885dbc7-lx8h7" Jul 07 22:04:16.717631 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:04:16.717576 2545 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 22:04:16.717631 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:04:16.717614 2545 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 07 22:04:16.717631 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:04:16.717631 2545 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-b9885dbc7-lx8h7: secret "image-registry-tls" not found Jul 07 22:04:16.717823 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:04:16.717651 2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c80e0cce-1240-4a8f-873b-abd71e85b59a-metrics-tls podName:c80e0cce-1240-4a8f-873b-abd71e85b59a nodeName:}" failed. No retries permitted until 2026-07-07 22:04:20.717631191 +0000 UTC m=+43.031854189 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c80e0cce-1240-4a8f-873b-abd71e85b59a-metrics-tls") pod "dns-default-bqv92" (UID: "c80e0cce-1240-4a8f-873b-abd71e85b59a") : secret "dns-default-metrics-tls" not found Jul 07 22:04:16.717823 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:04:16.717687 2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/78996163-d007-4f3c-ac3a-44902fe9be66-registry-tls podName:78996163-d007-4f3c-ac3a-44902fe9be66 nodeName:}" failed. No retries permitted until 2026-07-07 22:04:20.71766619 +0000 UTC m=+43.031889207 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/78996163-d007-4f3c-ac3a-44902fe9be66-registry-tls") pod "image-registry-b9885dbc7-lx8h7" (UID: "78996163-d007-4f3c-ac3a-44902fe9be66") : secret "image-registry-tls" not found Jul 07 22:04:17.422349 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:17.422274 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5398d2ac-a4ba-4c50-bdf4-94de0a0b284e-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-fjt2g\" (UID: \"5398d2ac-a4ba-4c50-bdf4-94de0a0b284e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-fjt2g" Jul 07 22:04:17.422970 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:04:17.422414 2545 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 07 22:04:17.422970 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:04:17.422476 2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5398d2ac-a4ba-4c50-bdf4-94de0a0b284e-samples-operator-tls podName:5398d2ac-a4ba-4c50-bdf4-94de0a0b284e nodeName:}" failed. No retries permitted until 2026-07-07 22:04:19.422460408 +0000 UTC m=+41.736683405 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/5398d2ac-a4ba-4c50-bdf4-94de0a0b284e-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-fjt2g" (UID: "5398d2ac-a4ba-4c50-bdf4-94de0a0b284e") : secret "samples-operator-tls" not found Jul 07 22:04:17.503635 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:17.503595 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ztkm2" event={"ID":"00baab1c-b8e0-4b5e-bd42-647ba72c5aca","Type":"ContainerStarted","Data":"4120498d346ec1903d907fe75c03d0536e91f0e82f6806caac74182c58f947ef"} Jul 07 22:04:17.522940 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:17.522914 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/84447fdf-4f2d-45fd-9e94-acce197ae75b-metrics-certs\") pod \"router-default-67496c4dcc-nnvc2\" (UID: \"84447fdf-4f2d-45fd-9e94-acce197ae75b\") " pod="openshift-ingress/router-default-67496c4dcc-nnvc2" Jul 07 22:04:17.523100 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:17.523001 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/84447fdf-4f2d-45fd-9e94-acce197ae75b-service-ca-bundle\") pod \"router-default-67496c4dcc-nnvc2\" (UID: \"84447fdf-4f2d-45fd-9e94-acce197ae75b\") " pod="openshift-ingress/router-default-67496c4dcc-nnvc2" Jul 07 22:04:17.523100 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:04:17.523069 2545 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 07 22:04:17.523208 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:04:17.523158 2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/84447fdf-4f2d-45fd-9e94-acce197ae75b-metrics-certs podName:84447fdf-4f2d-45fd-9e94-acce197ae75b nodeName:}" failed. No retries permitted until 2026-07-07 22:04:19.523139453 +0000 UTC m=+41.837362451 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/84447fdf-4f2d-45fd-9e94-acce197ae75b-metrics-certs") pod "router-default-67496c4dcc-nnvc2" (UID: "84447fdf-4f2d-45fd-9e94-acce197ae75b") : secret "router-metrics-certs-default" not found Jul 07 22:04:17.523208 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:04:17.523177 2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/84447fdf-4f2d-45fd-9e94-acce197ae75b-service-ca-bundle podName:84447fdf-4f2d-45fd-9e94-acce197ae75b nodeName:}" failed. No retries permitted until 2026-07-07 22:04:19.523168509 +0000 UTC m=+41.837391523 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/84447fdf-4f2d-45fd-9e94-acce197ae75b-service-ca-bundle") pod "router-default-67496c4dcc-nnvc2" (UID: "84447fdf-4f2d-45fd-9e94-acce197ae75b") : configmap references non-existent config key: service-ca.crt Jul 07 22:04:17.526735 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:17.526692 2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-ztkm2" podStartSLOduration=8.710646763 podStartE2EDuration="39.526677296s" totalImagesPullingTime="30.816030533s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 22:03:38 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:04:17.524544199 +0000 UTC m=+39.838767218" watchObservedRunningTime="2026-07-07 22:04:17.526677296 +0000 UTC m=+39.840900315" Jul 07 22:04:18.508743 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:18.508611 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-82vxn" event={"ID":"84813585-7321-4216-bab4-9258b9ecff6a","Type":"ContainerStarted","Data":"26c9381972a7390e936e28abde57a960d6a9235a5c16b082882cf234d8c54044"} Jul 07 22:04:18.511402 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:18.511343 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-wrlkn" event={"ID":"230f7860-610d-430a-987b-6dfaa258d4e7","Type":"ContainerStarted","Data":"26d52ad0912a6d9560accf8647d9c524a8641ef960c4b892662d48a487804285"} Jul 07 22:04:18.522884 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:18.522834 2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-82vxn" podStartSLOduration=1.678682923 podStartE2EDuration="3.522818046s" totalImagesPullingTime="1.844135123s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 22:04:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:04:18.522769762 +0000 UTC m=+40.836992777" watchObservedRunningTime="2026-07-07 22:04:18.522818046 +0000 UTC m=+40.837041066" Jul 07 22:04:18.537288 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:18.537227 2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-wrlkn" podStartSLOduration=3.3383163160000002 podStartE2EDuration="6.537211818s" totalImagesPullingTime="3.198895502s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 22:04:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:04:18.535994749 +0000 UTC m=+40.850217768" watchObservedRunningTime="2026-07-07 22:04:18.537211818 +0000 UTC m=+40.851434837" Jul 07 22:04:19.440357 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:19.440320 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5398d2ac-a4ba-4c50-bdf4-94de0a0b284e-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-fjt2g\" (UID: \"5398d2ac-a4ba-4c50-bdf4-94de0a0b284e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-fjt2g" Jul 07 22:04:19.440498 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:04:19.440473 2545 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 07 22:04:19.440565 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:04:19.440555 2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5398d2ac-a4ba-4c50-bdf4-94de0a0b284e-samples-operator-tls podName:5398d2ac-a4ba-4c50-bdf4-94de0a0b284e nodeName:}" failed. No retries permitted until 2026-07-07 22:04:23.440538769 +0000 UTC m=+45.754761770 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/5398d2ac-a4ba-4c50-bdf4-94de0a0b284e-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-fjt2g" (UID: "5398d2ac-a4ba-4c50-bdf4-94de0a0b284e") : secret "samples-operator-tls" not found Jul 07 22:04:19.540895 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:19.540863 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/84447fdf-4f2d-45fd-9e94-acce197ae75b-metrics-certs\") pod \"router-default-67496c4dcc-nnvc2\" (UID: \"84447fdf-4f2d-45fd-9e94-acce197ae75b\") " pod="openshift-ingress/router-default-67496c4dcc-nnvc2" Jul 07 22:04:19.541365 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:19.540927 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/84447fdf-4f2d-45fd-9e94-acce197ae75b-service-ca-bundle\") pod \"router-default-67496c4dcc-nnvc2\" (UID: \"84447fdf-4f2d-45fd-9e94-acce197ae75b\") " pod="openshift-ingress/router-default-67496c4dcc-nnvc2" Jul 07 22:04:19.541365 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:04:19.541027 2545 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 07 22:04:19.541365 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:04:19.541040 2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/84447fdf-4f2d-45fd-9e94-acce197ae75b-service-ca-bundle podName:84447fdf-4f2d-45fd-9e94-acce197ae75b nodeName:}" failed. No retries permitted until 2026-07-07 22:04:23.54102755 +0000 UTC m=+45.855250547 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/84447fdf-4f2d-45fd-9e94-acce197ae75b-service-ca-bundle") pod "router-default-67496c4dcc-nnvc2" (UID: "84447fdf-4f2d-45fd-9e94-acce197ae75b") : configmap references non-existent config key: service-ca.crt Jul 07 22:04:19.541365 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:04:19.541119 2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/84447fdf-4f2d-45fd-9e94-acce197ae75b-metrics-certs podName:84447fdf-4f2d-45fd-9e94-acce197ae75b nodeName:}" failed. No retries permitted until 2026-07-07 22:04:23.541102528 +0000 UTC m=+45.855325534 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/84447fdf-4f2d-45fd-9e94-acce197ae75b-metrics-certs") pod "router-default-67496c4dcc-nnvc2" (UID: "84447fdf-4f2d-45fd-9e94-acce197ae75b") : secret "router-metrics-certs-default" not found Jul 07 22:04:20.517880 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:20.517848 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wt5nz" event={"ID":"01d309f5-5c78-4908-a057-7dcb433b7f3b","Type":"ContainerStarted","Data":"9259d71890afb80eb4a436ca6a2ead118eb9030887fa3ae72ac6d0c92903136a"} Jul 07 22:04:20.532690 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:20.532639 2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wt5nz" podStartSLOduration=1.456530919 podStartE2EDuration="5.532624203s" totalImagesPullingTime="4.076093284s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 22:04:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:04:20.53189072 +0000 UTC m=+42.846113740" watchObservedRunningTime="2026-07-07 22:04:20.532624203 +0000 UTC m=+42.846847222" Jul 07 22:04:20.751482 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:20.751447 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c80e0cce-1240-4a8f-873b-abd71e85b59a-metrics-tls\") pod \"dns-default-bqv92\" (UID: \"c80e0cce-1240-4a8f-873b-abd71e85b59a\") " pod="openshift-dns/dns-default-bqv92" Jul 07 22:04:20.751834 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:20.751491 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/78996163-d007-4f3c-ac3a-44902fe9be66-registry-tls\") pod \"image-registry-b9885dbc7-lx8h7\" (UID: \"78996163-d007-4f3c-ac3a-44902fe9be66\") " pod="openshift-image-registry/image-registry-b9885dbc7-lx8h7" Jul 07 22:04:20.751834 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:04:20.751609 2545 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 07 22:04:20.751834 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:04:20.751622 2545 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-b9885dbc7-lx8h7: secret "image-registry-tls" not found Jul 07 22:04:20.751834 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:04:20.751673 2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/78996163-d007-4f3c-ac3a-44902fe9be66-registry-tls podName:78996163-d007-4f3c-ac3a-44902fe9be66 nodeName:}" failed. No retries permitted until 2026-07-07 22:04:28.751654505 +0000 UTC m=+51.065877532 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/78996163-d007-4f3c-ac3a-44902fe9be66-registry-tls") pod "image-registry-b9885dbc7-lx8h7" (UID: "78996163-d007-4f3c-ac3a-44902fe9be66") : secret "image-registry-tls" not found Jul 07 22:04:20.751834 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:04:20.751731 2545 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 22:04:20.751834 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:04:20.751800 2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c80e0cce-1240-4a8f-873b-abd71e85b59a-metrics-tls podName:c80e0cce-1240-4a8f-873b-abd71e85b59a nodeName:}" failed. No retries permitted until 2026-07-07 22:04:28.751782989 +0000 UTC m=+51.066005986 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c80e0cce-1240-4a8f-873b-abd71e85b59a-metrics-tls") pod "dns-default-bqv92" (UID: "c80e0cce-1240-4a8f-873b-abd71e85b59a") : secret "dns-default-metrics-tls" not found Jul 07 22:04:21.522216 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:21.522190 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-rhg5r_71f36bfa-4b8a-4572-b522-57c16392aa0b/console-operator/0.log" Jul 07 22:04:21.522423 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:21.522232 2545 generic.go:356] "Generic (PLEG): container finished" podID="71f36bfa-4b8a-4572-b522-57c16392aa0b" containerID="0e48413d0554d0dc7a8365a56b80d9abae8644f59d9366b83ffeb9778d72c0b2" exitCode=255 Jul 07 22:04:21.522423 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:21.522327 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-rhg5r" event={"ID":"71f36bfa-4b8a-4572-b522-57c16392aa0b","Type":"ContainerDied","Data":"0e48413d0554d0dc7a8365a56b80d9abae8644f59d9366b83ffeb9778d72c0b2"} Jul 07 22:04:21.522569 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:21.522551 2545 scope.go:117] "RemoveContainer" containerID="0e48413d0554d0dc7a8365a56b80d9abae8644f59d9366b83ffeb9778d72c0b2" Jul 07 22:04:22.526011 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:22.525982 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-rhg5r_71f36bfa-4b8a-4572-b522-57c16392aa0b/console-operator/1.log" Jul 07 22:04:22.526441 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:22.526425 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-rhg5r_71f36bfa-4b8a-4572-b522-57c16392aa0b/console-operator/0.log" Jul 07 22:04:22.526481 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:22.526460 2545 generic.go:356] "Generic (PLEG): container finished" podID="71f36bfa-4b8a-4572-b522-57c16392aa0b" containerID="e052544069e394735f783ce6acffb255f96b8f703ace2f93a82d75122020f1ee" exitCode=255 Jul 07 22:04:22.526511 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:22.526484 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-rhg5r" event={"ID":"71f36bfa-4b8a-4572-b522-57c16392aa0b","Type":"ContainerDied","Data":"e052544069e394735f783ce6acffb255f96b8f703ace2f93a82d75122020f1ee"} Jul 07 22:04:22.526560 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:22.526537 2545 scope.go:117] "RemoveContainer" containerID="0e48413d0554d0dc7a8365a56b80d9abae8644f59d9366b83ffeb9778d72c0b2" Jul 07 22:04:22.526771 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:22.526757 2545 scope.go:117] "RemoveContainer" containerID="e052544069e394735f783ce6acffb255f96b8f703ace2f93a82d75122020f1ee" Jul 07 22:04:22.526971 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:04:22.526946 2545 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-rhg5r_openshift-console-operator(71f36bfa-4b8a-4572-b522-57c16392aa0b)\"" pod="openshift-console-operator/console-operator-68b757fb67-rhg5r" podUID="71f36bfa-4b8a-4572-b522-57c16392aa0b" Jul 07 22:04:23.130832 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:23.130785 2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-9rmdm"] Jul 07 22:04:23.153322 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:23.153291 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-9rmdm"] Jul 07 22:04:23.153471 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:23.153401 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-9rmdm" Jul 07 22:04:23.155698 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:23.155676 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 07 22:04:23.155811 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:23.155718 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 07 22:04:23.155811 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:23.155735 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 07 22:04:23.155996 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:23.155978 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 07 22:04:23.156967 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:23.156948 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-bwh5s\"" Jul 07 22:04:23.271822 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:23.271792 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/add27e45-747d-4e8d-9c57-4ca7f6268343-crio-socket\") pod \"insights-runtime-extractor-9rmdm\" (UID: \"add27e45-747d-4e8d-9c57-4ca7f6268343\") " pod="openshift-insights/insights-runtime-extractor-9rmdm" Jul 07 22:04:23.272051 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:23.271835 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bbjdj\" (UniqueName: \"kubernetes.io/projected/add27e45-747d-4e8d-9c57-4ca7f6268343-kube-api-access-bbjdj\") pod \"insights-runtime-extractor-9rmdm\" (UID: \"add27e45-747d-4e8d-9c57-4ca7f6268343\") " pod="openshift-insights/insights-runtime-extractor-9rmdm" Jul 07 22:04:23.272051 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:23.271940 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/add27e45-747d-4e8d-9c57-4ca7f6268343-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9rmdm\" (UID: \"add27e45-747d-4e8d-9c57-4ca7f6268343\") " pod="openshift-insights/insights-runtime-extractor-9rmdm" Jul 07 22:04:23.272051 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:23.271967 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/add27e45-747d-4e8d-9c57-4ca7f6268343-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9rmdm\" (UID: \"add27e45-747d-4e8d-9c57-4ca7f6268343\") " pod="openshift-insights/insights-runtime-extractor-9rmdm" Jul 07 22:04:23.272207 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:23.272130 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/add27e45-747d-4e8d-9c57-4ca7f6268343-data-volume\") pod \"insights-runtime-extractor-9rmdm\" (UID: \"add27e45-747d-4e8d-9c57-4ca7f6268343\") " pod="openshift-insights/insights-runtime-extractor-9rmdm" Jul 07 22:04:23.373400 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:23.373361 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/add27e45-747d-4e8d-9c57-4ca7f6268343-data-volume\") pod \"insights-runtime-extractor-9rmdm\" (UID: \"add27e45-747d-4e8d-9c57-4ca7f6268343\") " pod="openshift-insights/insights-runtime-extractor-9rmdm" Jul 07 22:04:23.373400 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:23.373405 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/add27e45-747d-4e8d-9c57-4ca7f6268343-crio-socket\") pod \"insights-runtime-extractor-9rmdm\" (UID: \"add27e45-747d-4e8d-9c57-4ca7f6268343\") " pod="openshift-insights/insights-runtime-extractor-9rmdm" Jul 07 22:04:23.373642 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:23.373429 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bbjdj\" (UniqueName: \"kubernetes.io/projected/add27e45-747d-4e8d-9c57-4ca7f6268343-kube-api-access-bbjdj\") pod \"insights-runtime-extractor-9rmdm\" (UID: \"add27e45-747d-4e8d-9c57-4ca7f6268343\") " pod="openshift-insights/insights-runtime-extractor-9rmdm" Jul 07 22:04:23.373642 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:23.373595 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/add27e45-747d-4e8d-9c57-4ca7f6268343-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9rmdm\" (UID: \"add27e45-747d-4e8d-9c57-4ca7f6268343\") " pod="openshift-insights/insights-runtime-extractor-9rmdm" Jul 07 22:04:23.373642 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:23.373626 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/add27e45-747d-4e8d-9c57-4ca7f6268343-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9rmdm\" (UID: \"add27e45-747d-4e8d-9c57-4ca7f6268343\") " pod="openshift-insights/insights-runtime-extractor-9rmdm" Jul 07 22:04:23.373783 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:23.373597 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/add27e45-747d-4e8d-9c57-4ca7f6268343-crio-socket\") pod \"insights-runtime-extractor-9rmdm\" (UID: \"add27e45-747d-4e8d-9c57-4ca7f6268343\") " pod="openshift-insights/insights-runtime-extractor-9rmdm" Jul 07 22:04:23.373783 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:23.373673 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/add27e45-747d-4e8d-9c57-4ca7f6268343-data-volume\") pod \"insights-runtime-extractor-9rmdm\" (UID: \"add27e45-747d-4e8d-9c57-4ca7f6268343\") " pod="openshift-insights/insights-runtime-extractor-9rmdm" Jul 07 22:04:23.373783 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:04:23.373663 2545 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 22:04:23.373783 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:04:23.373716 2545 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 07 22:04:23.373783 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:04:23.373739 2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/add27e45-747d-4e8d-9c57-4ca7f6268343-insights-runtime-extractor-tls podName:add27e45-747d-4e8d-9c57-4ca7f6268343 nodeName:}" failed. No retries permitted until 2026-07-07 22:04:23.873718901 +0000 UTC m=+46.187941901 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/add27e45-747d-4e8d-9c57-4ca7f6268343-insights-runtime-extractor-tls") pod "insights-runtime-extractor-9rmdm" (UID: "add27e45-747d-4e8d-9c57-4ca7f6268343") : secret "insights-runtime-extractor-tls" not found Jul 07 22:04:23.373783 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:04:23.373759 2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/add27e45-747d-4e8d-9c57-4ca7f6268343-kube-rbac-proxy-cm podName:add27e45-747d-4e8d-9c57-4ca7f6268343 nodeName:}" failed. No retries permitted until 2026-07-07 22:04:23.873746503 +0000 UTC m=+46.187969520 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/add27e45-747d-4e8d-9c57-4ca7f6268343-kube-rbac-proxy-cm") pod "insights-runtime-extractor-9rmdm" (UID: "add27e45-747d-4e8d-9c57-4ca7f6268343") : configmap "kube-rbac-proxy" not found Jul 07 22:04:23.383264 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:23.383211 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bbjdj\" (UniqueName: \"kubernetes.io/projected/add27e45-747d-4e8d-9c57-4ca7f6268343-kube-api-access-bbjdj\") pod \"insights-runtime-extractor-9rmdm\" (UID: \"add27e45-747d-4e8d-9c57-4ca7f6268343\") " pod="openshift-insights/insights-runtime-extractor-9rmdm" Jul 07 22:04:23.474802 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:23.474759 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5398d2ac-a4ba-4c50-bdf4-94de0a0b284e-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-fjt2g\" (UID: \"5398d2ac-a4ba-4c50-bdf4-94de0a0b284e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-fjt2g" Jul 07 22:04:23.474954 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:04:23.474911 2545 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 07 22:04:23.475002 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:04:23.474976 2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5398d2ac-a4ba-4c50-bdf4-94de0a0b284e-samples-operator-tls podName:5398d2ac-a4ba-4c50-bdf4-94de0a0b284e nodeName:}" failed. No retries permitted until 2026-07-07 22:04:31.474960924 +0000 UTC m=+53.789183921 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/5398d2ac-a4ba-4c50-bdf4-94de0a0b284e-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-fjt2g" (UID: "5398d2ac-a4ba-4c50-bdf4-94de0a0b284e") : secret "samples-operator-tls" not found Jul 07 22:04:23.530483 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:23.530455 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-rhg5r_71f36bfa-4b8a-4572-b522-57c16392aa0b/console-operator/1.log" Jul 07 22:04:23.530817 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:23.530772 2545 scope.go:117] "RemoveContainer" containerID="e052544069e394735f783ce6acffb255f96b8f703ace2f93a82d75122020f1ee" Jul 07 22:04:23.530928 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:04:23.530910 2545 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-rhg5r_openshift-console-operator(71f36bfa-4b8a-4572-b522-57c16392aa0b)\"" pod="openshift-console-operator/console-operator-68b757fb67-rhg5r" podUID="71f36bfa-4b8a-4572-b522-57c16392aa0b" Jul 07 22:04:23.575901 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:23.575872 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/84447fdf-4f2d-45fd-9e94-acce197ae75b-metrics-certs\") pod \"router-default-67496c4dcc-nnvc2\" (UID: \"84447fdf-4f2d-45fd-9e94-acce197ae75b\") " pod="openshift-ingress/router-default-67496c4dcc-nnvc2" Jul 07 22:04:23.576047 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:23.575942 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/84447fdf-4f2d-45fd-9e94-acce197ae75b-service-ca-bundle\") pod \"router-default-67496c4dcc-nnvc2\" (UID: \"84447fdf-4f2d-45fd-9e94-acce197ae75b\") " pod="openshift-ingress/router-default-67496c4dcc-nnvc2" Jul 07 22:04:23.576047 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:04:23.576012 2545 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 07 22:04:23.576153 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:04:23.576096 2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/84447fdf-4f2d-45fd-9e94-acce197ae75b-metrics-certs podName:84447fdf-4f2d-45fd-9e94-acce197ae75b nodeName:}" failed. No retries permitted until 2026-07-07 22:04:31.576064328 +0000 UTC m=+53.890287324 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/84447fdf-4f2d-45fd-9e94-acce197ae75b-metrics-certs") pod "router-default-67496c4dcc-nnvc2" (UID: "84447fdf-4f2d-45fd-9e94-acce197ae75b") : secret "router-metrics-certs-default" not found Jul 07 22:04:23.576153 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:04:23.576111 2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/84447fdf-4f2d-45fd-9e94-acce197ae75b-service-ca-bundle podName:84447fdf-4f2d-45fd-9e94-acce197ae75b nodeName:}" failed. No retries permitted until 2026-07-07 22:04:31.576104019 +0000 UTC m=+53.890327016 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/84447fdf-4f2d-45fd-9e94-acce197ae75b-service-ca-bundle") pod "router-default-67496c4dcc-nnvc2" (UID: "84447fdf-4f2d-45fd-9e94-acce197ae75b") : configmap references non-existent config key: service-ca.crt Jul 07 22:04:23.878174 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:23.878128 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/add27e45-747d-4e8d-9c57-4ca7f6268343-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9rmdm\" (UID: \"add27e45-747d-4e8d-9c57-4ca7f6268343\") " pod="openshift-insights/insights-runtime-extractor-9rmdm" Jul 07 22:04:23.878348 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:23.878184 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/add27e45-747d-4e8d-9c57-4ca7f6268343-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9rmdm\" (UID: \"add27e45-747d-4e8d-9c57-4ca7f6268343\") " pod="openshift-insights/insights-runtime-extractor-9rmdm" Jul 07 22:04:23.881443 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:04:23.878584 2545 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 22:04:23.881443 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:04:23.878613 2545 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 07 22:04:23.881443 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:04:23.878683 2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/add27e45-747d-4e8d-9c57-4ca7f6268343-insights-runtime-extractor-tls podName:add27e45-747d-4e8d-9c57-4ca7f6268343 nodeName:}" failed. No retries permitted until 2026-07-07 22:04:24.878663096 +0000 UTC m=+47.192886109 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/add27e45-747d-4e8d-9c57-4ca7f6268343-insights-runtime-extractor-tls") pod "insights-runtime-extractor-9rmdm" (UID: "add27e45-747d-4e8d-9c57-4ca7f6268343") : secret "insights-runtime-extractor-tls" not found Jul 07 22:04:23.881443 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:04:23.878705 2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/add27e45-747d-4e8d-9c57-4ca7f6268343-kube-rbac-proxy-cm podName:add27e45-747d-4e8d-9c57-4ca7f6268343 nodeName:}" failed. No retries permitted until 2026-07-07 22:04:24.878694392 +0000 UTC m=+47.192917389 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/add27e45-747d-4e8d-9c57-4ca7f6268343-kube-rbac-proxy-cm") pod "insights-runtime-extractor-9rmdm" (UID: "add27e45-747d-4e8d-9c57-4ca7f6268343") : configmap "kube-rbac-proxy" not found Jul 07 22:04:24.886390 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:24.886343 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/add27e45-747d-4e8d-9c57-4ca7f6268343-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9rmdm\" (UID: \"add27e45-747d-4e8d-9c57-4ca7f6268343\") " pod="openshift-insights/insights-runtime-extractor-9rmdm" Jul 07 22:04:24.886778 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:04:24.886472 2545 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 07 22:04:24.886778 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:04:24.886533 2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/add27e45-747d-4e8d-9c57-4ca7f6268343-kube-rbac-proxy-cm podName:add27e45-747d-4e8d-9c57-4ca7f6268343 nodeName:}" failed. No retries permitted until 2026-07-07 22:04:26.886518406 +0000 UTC m=+49.200741404 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/add27e45-747d-4e8d-9c57-4ca7f6268343-kube-rbac-proxy-cm") pod "insights-runtime-extractor-9rmdm" (UID: "add27e45-747d-4e8d-9c57-4ca7f6268343") : configmap "kube-rbac-proxy" not found Jul 07 22:04:24.886778 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:24.886477 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/add27e45-747d-4e8d-9c57-4ca7f6268343-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9rmdm\" (UID: \"add27e45-747d-4e8d-9c57-4ca7f6268343\") " pod="openshift-insights/insights-runtime-extractor-9rmdm" Jul 07 22:04:24.886778 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:04:24.886543 2545 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 22:04:24.886778 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:04:24.886632 2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/add27e45-747d-4e8d-9c57-4ca7f6268343-insights-runtime-extractor-tls podName:add27e45-747d-4e8d-9c57-4ca7f6268343 nodeName:}" failed. No retries permitted until 2026-07-07 22:04:26.8866121 +0000 UTC m=+49.200835097 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/add27e45-747d-4e8d-9c57-4ca7f6268343-insights-runtime-extractor-tls") pod "insights-runtime-extractor-9rmdm" (UID: "add27e45-747d-4e8d-9c57-4ca7f6268343") : secret "insights-runtime-extractor-tls" not found Jul 07 22:04:25.129088 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:25.129063 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-9vvqt_7ac39c08-2ce6-4578-8b01-b68587adf750/dns-node-resolver/0.log" Jul 07 22:04:26.132978 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:26.132952 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-cc72r_381be182-aba1-4465-a899-8f217bbeacd2/node-ca/0.log" Jul 07 22:04:26.146699 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:26.146681 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-rhg5r" Jul 07 22:04:26.146801 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:26.146729 2545 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-68b757fb67-rhg5r" Jul 07 22:04:26.147044 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:26.147032 2545 scope.go:117] "RemoveContainer" containerID="e052544069e394735f783ce6acffb255f96b8f703ace2f93a82d75122020f1ee" Jul 07 22:04:26.147213 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:04:26.147194 2545 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-rhg5r_openshift-console-operator(71f36bfa-4b8a-4572-b522-57c16392aa0b)\"" pod="openshift-console-operator/console-operator-68b757fb67-rhg5r" podUID="71f36bfa-4b8a-4572-b522-57c16392aa0b" Jul 07 22:04:26.903843 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:26.903807 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/add27e45-747d-4e8d-9c57-4ca7f6268343-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9rmdm\" (UID: \"add27e45-747d-4e8d-9c57-4ca7f6268343\") " pod="openshift-insights/insights-runtime-extractor-9rmdm" Jul 07 22:04:26.903843 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:26.903847 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/add27e45-747d-4e8d-9c57-4ca7f6268343-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9rmdm\" (UID: \"add27e45-747d-4e8d-9c57-4ca7f6268343\") " pod="openshift-insights/insights-runtime-extractor-9rmdm" Jul 07 22:04:26.904066 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:04:26.903963 2545 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 07 22:04:26.904066 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:04:26.903987 2545 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 22:04:26.904066 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:04:26.904032 2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/add27e45-747d-4e8d-9c57-4ca7f6268343-insights-runtime-extractor-tls podName:add27e45-747d-4e8d-9c57-4ca7f6268343 nodeName:}" failed. No retries permitted until 2026-07-07 22:04:30.904013563 +0000 UTC m=+53.218236580 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/add27e45-747d-4e8d-9c57-4ca7f6268343-insights-runtime-extractor-tls") pod "insights-runtime-extractor-9rmdm" (UID: "add27e45-747d-4e8d-9c57-4ca7f6268343") : secret "insights-runtime-extractor-tls" not found Jul 07 22:04:26.904066 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:04:26.904048 2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/add27e45-747d-4e8d-9c57-4ca7f6268343-kube-rbac-proxy-cm podName:add27e45-747d-4e8d-9c57-4ca7f6268343 nodeName:}" failed. No retries permitted until 2026-07-07 22:04:30.904040433 +0000 UTC m=+53.218263430 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/add27e45-747d-4e8d-9c57-4ca7f6268343-kube-rbac-proxy-cm") pod "insights-runtime-extractor-9rmdm" (UID: "add27e45-747d-4e8d-9c57-4ca7f6268343") : configmap "kube-rbac-proxy" not found Jul 07 22:04:27.130099 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:27.130062 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-wrlkn_230f7860-610d-430a-987b-6dfaa258d4e7/serve-healthcheck-canary/0.log" Jul 07 22:04:28.819634 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:28.819594 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c80e0cce-1240-4a8f-873b-abd71e85b59a-metrics-tls\") pod \"dns-default-bqv92\" (UID: \"c80e0cce-1240-4a8f-873b-abd71e85b59a\") " pod="openshift-dns/dns-default-bqv92" Jul 07 22:04:28.819634 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:28.819641 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/78996163-d007-4f3c-ac3a-44902fe9be66-registry-tls\") pod \"image-registry-b9885dbc7-lx8h7\" (UID: \"78996163-d007-4f3c-ac3a-44902fe9be66\") " pod="openshift-image-registry/image-registry-b9885dbc7-lx8h7" Jul 07 22:04:28.820037 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:04:28.819746 2545 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 07 22:04:28.820037 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:04:28.819808 2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c80e0cce-1240-4a8f-873b-abd71e85b59a-metrics-tls podName:c80e0cce-1240-4a8f-873b-abd71e85b59a nodeName:}" failed. No retries permitted until 2026-07-07 22:04:44.819791675 +0000 UTC m=+67.134014672 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c80e0cce-1240-4a8f-873b-abd71e85b59a-metrics-tls") pod "dns-default-bqv92" (UID: "c80e0cce-1240-4a8f-873b-abd71e85b59a") : secret "dns-default-metrics-tls" not found Jul 07 22:04:28.820037 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:04:28.819814 2545 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 07 22:04:28.820037 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:04:28.819829 2545 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-b9885dbc7-lx8h7: secret "image-registry-tls" not found Jul 07 22:04:28.820037 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:04:28.819875 2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/78996163-d007-4f3c-ac3a-44902fe9be66-registry-tls podName:78996163-d007-4f3c-ac3a-44902fe9be66 nodeName:}" failed. No retries permitted until 2026-07-07 22:04:44.819860341 +0000 UTC m=+67.134083347 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/78996163-d007-4f3c-ac3a-44902fe9be66-registry-tls") pod "image-registry-b9885dbc7-lx8h7" (UID: "78996163-d007-4f3c-ac3a-44902fe9be66") : secret "image-registry-tls" not found Jul 07 22:04:30.935794 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:30.935753 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/add27e45-747d-4e8d-9c57-4ca7f6268343-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9rmdm\" (UID: \"add27e45-747d-4e8d-9c57-4ca7f6268343\") " pod="openshift-insights/insights-runtime-extractor-9rmdm" Jul 07 22:04:30.935794 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:30.935793 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/add27e45-747d-4e8d-9c57-4ca7f6268343-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9rmdm\" (UID: \"add27e45-747d-4e8d-9c57-4ca7f6268343\") " pod="openshift-insights/insights-runtime-extractor-9rmdm" Jul 07 22:04:30.936326 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:04:30.935911 2545 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 07 22:04:30.936326 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:04:30.935980 2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/add27e45-747d-4e8d-9c57-4ca7f6268343-insights-runtime-extractor-tls podName:add27e45-747d-4e8d-9c57-4ca7f6268343 nodeName:}" failed. No retries permitted until 2026-07-07 22:04:38.935966143 +0000 UTC m=+61.250189140 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/add27e45-747d-4e8d-9c57-4ca7f6268343-insights-runtime-extractor-tls") pod "insights-runtime-extractor-9rmdm" (UID: "add27e45-747d-4e8d-9c57-4ca7f6268343") : secret "insights-runtime-extractor-tls" not found Jul 07 22:04:30.936326 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:30.936313 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/add27e45-747d-4e8d-9c57-4ca7f6268343-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-9rmdm\" (UID: \"add27e45-747d-4e8d-9c57-4ca7f6268343\") " pod="openshift-insights/insights-runtime-extractor-9rmdm" Jul 07 22:04:31.541665 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:31.541625 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5398d2ac-a4ba-4c50-bdf4-94de0a0b284e-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-fjt2g\" (UID: \"5398d2ac-a4ba-4c50-bdf4-94de0a0b284e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-fjt2g" Jul 07 22:04:31.543888 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:31.543856 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5398d2ac-a4ba-4c50-bdf4-94de0a0b284e-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-fjt2g\" (UID: \"5398d2ac-a4ba-4c50-bdf4-94de0a0b284e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-fjt2g" Jul 07 22:04:31.642056 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:31.642021 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/84447fdf-4f2d-45fd-9e94-acce197ae75b-metrics-certs\") pod \"router-default-67496c4dcc-nnvc2\" (UID: \"84447fdf-4f2d-45fd-9e94-acce197ae75b\") " pod="openshift-ingress/router-default-67496c4dcc-nnvc2" Jul 07 22:04:31.642266 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:31.642106 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/84447fdf-4f2d-45fd-9e94-acce197ae75b-service-ca-bundle\") pod \"router-default-67496c4dcc-nnvc2\" (UID: \"84447fdf-4f2d-45fd-9e94-acce197ae75b\") " pod="openshift-ingress/router-default-67496c4dcc-nnvc2" Jul 07 22:04:31.642266 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:04:31.642194 2545 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 07 22:04:31.642266 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:04:31.642240 2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/84447fdf-4f2d-45fd-9e94-acce197ae75b-service-ca-bundle podName:84447fdf-4f2d-45fd-9e94-acce197ae75b nodeName:}" failed. No retries permitted until 2026-07-07 22:04:47.64222417 +0000 UTC m=+69.956447168 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/84447fdf-4f2d-45fd-9e94-acce197ae75b-service-ca-bundle") pod "router-default-67496c4dcc-nnvc2" (UID: "84447fdf-4f2d-45fd-9e94-acce197ae75b") : configmap references non-existent config key: service-ca.crt Jul 07 22:04:31.642266 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:04:31.642266 2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/84447fdf-4f2d-45fd-9e94-acce197ae75b-metrics-certs podName:84447fdf-4f2d-45fd-9e94-acce197ae75b nodeName:}" failed. No retries permitted until 2026-07-07 22:04:47.642248966 +0000 UTC m=+69.956471977 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/84447fdf-4f2d-45fd-9e94-acce197ae75b-metrics-certs") pod "router-default-67496c4dcc-nnvc2" (UID: "84447fdf-4f2d-45fd-9e94-acce197ae75b") : secret "router-metrics-certs-default" not found Jul 07 22:04:31.645042 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:31.645012 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-fjt2g" Jul 07 22:04:31.784800 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:31.784770 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-fjt2g"] Jul 07 22:04:32.556300 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:32.556261 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-fjt2g" event={"ID":"5398d2ac-a4ba-4c50-bdf4-94de0a0b284e","Type":"ContainerStarted","Data":"09c9ded340c34388c7185922d747ea7ed48a7fbd7452cf2754066f6eff65581f"} Jul 07 22:04:34.564354 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:34.564317 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-fjt2g" event={"ID":"5398d2ac-a4ba-4c50-bdf4-94de0a0b284e","Type":"ContainerStarted","Data":"2e17aeb9a1b413c40f005be1c5a30dfeb0e23f396022f339e7c3f3540c766bec"} Jul 07 22:04:34.564804 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:34.564378 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-fjt2g" event={"ID":"5398d2ac-a4ba-4c50-bdf4-94de0a0b284e","Type":"ContainerStarted","Data":"ffc9e13327278f28fade456aeddaee8622a6ffeb73a6280f3afe79e0eb5d184f"} Jul 07 22:04:34.583813 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:34.583770 2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-fjt2g" podStartSLOduration=17.843967874 podStartE2EDuration="19.583756534s" totalImagesPullingTime="1.73978866s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 22:04:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:04:34.583013718 +0000 UTC m=+56.897236737" watchObservedRunningTime="2026-07-07 22:04:34.583756534 +0000 UTC m=+56.897979552" Jul 07 22:04:37.306014 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:37.305985 2545 scope.go:117] "RemoveContainer" containerID="e052544069e394735f783ce6acffb255f96b8f703ace2f93a82d75122020f1ee" Jul 07 22:04:37.576206 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:37.576128 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-rhg5r_71f36bfa-4b8a-4572-b522-57c16392aa0b/console-operator/1.log" Jul 07 22:04:37.576379 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:37.576209 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-rhg5r" event={"ID":"71f36bfa-4b8a-4572-b522-57c16392aa0b","Type":"ContainerStarted","Data":"f1248c051729335a23e3892242cd278dc408af5718a2fa108187b4ba0386be48"} Jul 07 22:04:37.576507 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:37.576485 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-rhg5r" Jul 07 22:04:37.591914 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:37.591867 2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-68b757fb67-rhg5r" podStartSLOduration=18.115608243 podStartE2EDuration="22.59185081s" totalImagesPullingTime="4.476242567s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 22:04:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:04:37.591112539 +0000 UTC m=+59.905335558" watchObservedRunningTime="2026-07-07 22:04:37.59185081 +0000 UTC m=+59.906073828" Jul 07 22:04:37.710105 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:37.710065 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-68b757fb67-rhg5r" Jul 07 22:04:38.473488 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:38.473461 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-6h4fg" Jul 07 22:04:38.998424 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:38.998387 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/add27e45-747d-4e8d-9c57-4ca7f6268343-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9rmdm\" (UID: \"add27e45-747d-4e8d-9c57-4ca7f6268343\") " pod="openshift-insights/insights-runtime-extractor-9rmdm" Jul 07 22:04:39.000658 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:39.000639 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/add27e45-747d-4e8d-9c57-4ca7f6268343-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-9rmdm\" (UID: \"add27e45-747d-4e8d-9c57-4ca7f6268343\") " pod="openshift-insights/insights-runtime-extractor-9rmdm" Jul 07 22:04:39.065466 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:39.065434 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-bwh5s\"" Jul 07 22:04:39.073378 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:39.073359 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-9rmdm" Jul 07 22:04:39.213149 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:39.213123 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-9rmdm"] Jul 07 22:04:39.215936 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:04:39.215909 2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podadd27e45_747d_4e8d_9c57_4ca7f6268343.slice/crio-0f016880330d7f54029d3e5dc9d2f110e611bdc477b9f415fc6a1635336b553a WatchSource:0}: Error finding container 0f016880330d7f54029d3e5dc9d2f110e611bdc477b9f415fc6a1635336b553a: Status 404 returned error can't find the container with id 0f016880330d7f54029d3e5dc9d2f110e611bdc477b9f415fc6a1635336b553a Jul 07 22:04:39.585567 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:39.585526 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9rmdm" event={"ID":"add27e45-747d-4e8d-9c57-4ca7f6268343","Type":"ContainerStarted","Data":"0f016880330d7f54029d3e5dc9d2f110e611bdc477b9f415fc6a1635336b553a"} Jul 07 22:04:43.601367 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:43.601329 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9rmdm" event={"ID":"add27e45-747d-4e8d-9c57-4ca7f6268343","Type":"ContainerStarted","Data":"dfa61796789ce7fd089bd63144930f2f0fcfab5a923ac305fc120c7566ec3212"} Jul 07 22:04:43.943029 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:43.942942 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f7c3d30c-9d3a-43c7-9a22-92d1fca6d2ea-metrics-certs\") pod \"network-metrics-daemon-88qj9\" (UID: \"f7c3d30c-9d3a-43c7-9a22-92d1fca6d2ea\") " pod="openshift-multus/network-metrics-daemon-88qj9" Jul 07 22:04:43.945251 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:43.945228 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 07 22:04:43.956309 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:43.956284 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f7c3d30c-9d3a-43c7-9a22-92d1fca6d2ea-metrics-certs\") pod \"network-metrics-daemon-88qj9\" (UID: \"f7c3d30c-9d3a-43c7-9a22-92d1fca6d2ea\") " pod="openshift-multus/network-metrics-daemon-88qj9" Jul 07 22:04:44.043732 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:44.043684 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4vbkc\" (UniqueName: \"kubernetes.io/projected/30024b54-40f6-40b2-9b04-a82f5650ceba-kube-api-access-4vbkc\") pod \"network-check-target-82xqf\" (UID: \"30024b54-40f6-40b2-9b04-a82f5650ceba\") " pod="openshift-network-diagnostics/network-check-target-82xqf" Jul 07 22:04:44.043931 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:44.043755 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/01e846e0-a94f-446e-8ec7-fdeb51040759-original-pull-secret\") pod \"global-pull-secret-syncer-nqfk8\" (UID: \"01e846e0-a94f-446e-8ec7-fdeb51040759\") " pod="kube-system/global-pull-secret-syncer-nqfk8" Jul 07 22:04:44.046032 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:44.046000 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 07 22:04:44.046164 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:44.046117 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 07 22:04:44.055980 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:44.055958 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 07 22:04:44.056193 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:44.056177 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/01e846e0-a94f-446e-8ec7-fdeb51040759-original-pull-secret\") pod \"global-pull-secret-syncer-nqfk8\" (UID: \"01e846e0-a94f-446e-8ec7-fdeb51040759\") " pod="kube-system/global-pull-secret-syncer-nqfk8" Jul 07 22:04:44.066996 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:44.066969 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4vbkc\" (UniqueName: \"kubernetes.io/projected/30024b54-40f6-40b2-9b04-a82f5650ceba-kube-api-access-4vbkc\") pod \"network-check-target-82xqf\" (UID: \"30024b54-40f6-40b2-9b04-a82f5650ceba\") " pod="openshift-network-diagnostics/network-check-target-82xqf" Jul 07 22:04:44.216913 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:44.216829 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nqfk8" Jul 07 22:04:44.231394 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:44.231369 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-hdlmc\"" Jul 07 22:04:44.234926 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:44.234904 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-p2w6w\"" Jul 07 22:04:44.239632 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:44.239591 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-88qj9" Jul 07 22:04:44.243906 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:44.243874 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-82xqf" Jul 07 22:04:44.369631 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:44.369561 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-nqfk8"] Jul 07 22:04:44.373519 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:04:44.373479 2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01e846e0_a94f_446e_8ec7_fdeb51040759.slice/crio-bf2ed3e15283bb8cfcbe619b7ccb02f04b8e71f5d0d0568d7aae7163583d134a WatchSource:0}: Error finding container bf2ed3e15283bb8cfcbe619b7ccb02f04b8e71f5d0d0568d7aae7163583d134a: Status 404 returned error can't find the container with id bf2ed3e15283bb8cfcbe619b7ccb02f04b8e71f5d0d0568d7aae7163583d134a Jul 07 22:04:44.399926 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:44.399895 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-88qj9"] Jul 07 22:04:44.404071 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:04:44.404039 2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7c3d30c_9d3a_43c7_9a22_92d1fca6d2ea.slice/crio-a4090639a7c3ae2e9d65ca80580e9a5ca1b68e9741eff9c278c1dd1b377e58bd WatchSource:0}: Error finding container a4090639a7c3ae2e9d65ca80580e9a5ca1b68e9741eff9c278c1dd1b377e58bd: Status 404 returned error can't find the container with id a4090639a7c3ae2e9d65ca80580e9a5ca1b68e9741eff9c278c1dd1b377e58bd Jul 07 22:04:44.419812 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:44.419786 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-82xqf"] Jul 07 22:04:44.423482 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:04:44.423455 2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30024b54_40f6_40b2_9b04_a82f5650ceba.slice/crio-9490079a7de2f9a6b64d8fc22a6d3cd81e8506c25095a8101edcdc55f503e2c8 WatchSource:0}: Error finding container 9490079a7de2f9a6b64d8fc22a6d3cd81e8506c25095a8101edcdc55f503e2c8: Status 404 returned error can't find the container with id 9490079a7de2f9a6b64d8fc22a6d3cd81e8506c25095a8101edcdc55f503e2c8 Jul 07 22:04:44.606804 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:44.606771 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-88qj9" event={"ID":"f7c3d30c-9d3a-43c7-9a22-92d1fca6d2ea","Type":"ContainerStarted","Data":"a4090639a7c3ae2e9d65ca80580e9a5ca1b68e9741eff9c278c1dd1b377e58bd"} Jul 07 22:04:44.608933 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:44.608904 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9rmdm" event={"ID":"add27e45-747d-4e8d-9c57-4ca7f6268343","Type":"ContainerStarted","Data":"e96415a9847faf04cbf5a6e31390022155f268a23c17b69e5737cb46b880c286"} Jul 07 22:04:44.610831 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:44.610807 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-82xqf" event={"ID":"30024b54-40f6-40b2-9b04-a82f5650ceba","Type":"ContainerStarted","Data":"9490079a7de2f9a6b64d8fc22a6d3cd81e8506c25095a8101edcdc55f503e2c8"} Jul 07 22:04:44.611960 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:44.611936 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-nqfk8" event={"ID":"01e846e0-a94f-446e-8ec7-fdeb51040759","Type":"ContainerStarted","Data":"bf2ed3e15283bb8cfcbe619b7ccb02f04b8e71f5d0d0568d7aae7163583d134a"} Jul 07 22:04:44.850373 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:44.850331 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c80e0cce-1240-4a8f-873b-abd71e85b59a-metrics-tls\") pod \"dns-default-bqv92\" (UID: \"c80e0cce-1240-4a8f-873b-abd71e85b59a\") " pod="openshift-dns/dns-default-bqv92" Jul 07 22:04:44.850574 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:44.850391 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/78996163-d007-4f3c-ac3a-44902fe9be66-registry-tls\") pod \"image-registry-b9885dbc7-lx8h7\" (UID: \"78996163-d007-4f3c-ac3a-44902fe9be66\") " pod="openshift-image-registry/image-registry-b9885dbc7-lx8h7" Jul 07 22:04:44.853410 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:44.853369 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c80e0cce-1240-4a8f-873b-abd71e85b59a-metrics-tls\") pod \"dns-default-bqv92\" (UID: \"c80e0cce-1240-4a8f-873b-abd71e85b59a\") " pod="openshift-dns/dns-default-bqv92" Jul 07 22:04:44.853563 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:44.853487 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/78996163-d007-4f3c-ac3a-44902fe9be66-registry-tls\") pod \"image-registry-b9885dbc7-lx8h7\" (UID: \"78996163-d007-4f3c-ac3a-44902fe9be66\") " pod="openshift-image-registry/image-registry-b9885dbc7-lx8h7" Jul 07 22:04:45.042990 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:45.042777 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-c748h\"" Jul 07 22:04:45.051067 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:45.050780 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-b9885dbc7-lx8h7" Jul 07 22:04:45.061592 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:45.061383 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-kp4p7\"" Jul 07 22:04:45.070358 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:45.070332 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-bqv92" Jul 07 22:04:45.264946 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:45.264890 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-b9885dbc7-lx8h7"] Jul 07 22:04:45.274570 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:04:45.274539 2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78996163_d007_4f3c_ac3a_44902fe9be66.slice/crio-defdd452ad110e1e343aec48f3ac3a959460f3f9dd62bc9875cd4bd076d8262f WatchSource:0}: Error finding container defdd452ad110e1e343aec48f3ac3a959460f3f9dd62bc9875cd4bd076d8262f: Status 404 returned error can't find the container with id defdd452ad110e1e343aec48f3ac3a959460f3f9dd62bc9875cd4bd076d8262f Jul 07 22:04:45.301832 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:45.301808 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-bqv92"] Jul 07 22:04:45.308460 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:04:45.308159 2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc80e0cce_1240_4a8f_873b_abd71e85b59a.slice/crio-91c2b0ec23a8feb7f503b52eb7a6cd2c0cce27280bda1a969b47f1aec900d455 WatchSource:0}: Error finding container 91c2b0ec23a8feb7f503b52eb7a6cd2c0cce27280bda1a969b47f1aec900d455: Status 404 returned error can't find the container with id 91c2b0ec23a8feb7f503b52eb7a6cd2c0cce27280bda1a969b47f1aec900d455 Jul 07 22:04:45.619324 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:45.619179 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-bqv92" event={"ID":"c80e0cce-1240-4a8f-873b-abd71e85b59a","Type":"ContainerStarted","Data":"91c2b0ec23a8feb7f503b52eb7a6cd2c0cce27280bda1a969b47f1aec900d455"} Jul 07 22:04:45.621491 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:45.621408 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-b9885dbc7-lx8h7" event={"ID":"78996163-d007-4f3c-ac3a-44902fe9be66","Type":"ContainerStarted","Data":"d9da8be85010c7a62ffc882ed477e1d973c2f712e10a40e82b5af6e00a69e75e"} Jul 07 22:04:45.621491 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:45.621456 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-b9885dbc7-lx8h7" event={"ID":"78996163-d007-4f3c-ac3a-44902fe9be66","Type":"ContainerStarted","Data":"defdd452ad110e1e343aec48f3ac3a959460f3f9dd62bc9875cd4bd076d8262f"} Jul 07 22:04:45.621788 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:45.621765 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-b9885dbc7-lx8h7" Jul 07 22:04:45.642847 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:45.642500 2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-b9885dbc7-lx8h7" podStartSLOduration=67.642480178 podStartE2EDuration="1m7.642480178s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 22:03:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:04:45.640615251 +0000 UTC m=+67.954838269" watchObservedRunningTime="2026-07-07 22:04:45.642480178 +0000 UTC m=+67.956703199" Jul 07 22:04:46.680830 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:46.679779 2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-685bfb5f64-q7rgl"] Jul 07 22:04:46.688851 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:46.688321 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-q7rgl" Jul 07 22:04:46.691797 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:46.690848 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-f5748\"" Jul 07 22:04:46.691797 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:46.691318 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 07 22:04:46.691797 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:46.691555 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 07 22:04:46.695332 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:46.694887 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-q7rgl"] Jul 07 22:04:46.767711 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:46.767677 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jd87p\" (UniqueName: \"kubernetes.io/projected/f9c45f5f-a6f4-406c-a8f8-3d474a0dce97-kube-api-access-jd87p\") pod \"downloads-685bfb5f64-q7rgl\" (UID: \"f9c45f5f-a6f4-406c-a8f8-3d474a0dce97\") " pod="openshift-console/downloads-685bfb5f64-q7rgl" Jul 07 22:04:46.868944 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:46.868902 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jd87p\" (UniqueName: \"kubernetes.io/projected/f9c45f5f-a6f4-406c-a8f8-3d474a0dce97-kube-api-access-jd87p\") pod \"downloads-685bfb5f64-q7rgl\" (UID: \"f9c45f5f-a6f4-406c-a8f8-3d474a0dce97\") " pod="openshift-console/downloads-685bfb5f64-q7rgl" Jul 07 22:04:46.885283 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:46.885257 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jd87p\" (UniqueName: \"kubernetes.io/projected/f9c45f5f-a6f4-406c-a8f8-3d474a0dce97-kube-api-access-jd87p\") pod \"downloads-685bfb5f64-q7rgl\" (UID: \"f9c45f5f-a6f4-406c-a8f8-3d474a0dce97\") " pod="openshift-console/downloads-685bfb5f64-q7rgl" Jul 07 22:04:47.013461 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:47.013422 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-685bfb5f64-q7rgl" Jul 07 22:04:47.675219 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:47.675178 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/84447fdf-4f2d-45fd-9e94-acce197ae75b-metrics-certs\") pod \"router-default-67496c4dcc-nnvc2\" (UID: \"84447fdf-4f2d-45fd-9e94-acce197ae75b\") " pod="openshift-ingress/router-default-67496c4dcc-nnvc2" Jul 07 22:04:47.675392 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:47.675269 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/84447fdf-4f2d-45fd-9e94-acce197ae75b-service-ca-bundle\") pod \"router-default-67496c4dcc-nnvc2\" (UID: \"84447fdf-4f2d-45fd-9e94-acce197ae75b\") " pod="openshift-ingress/router-default-67496c4dcc-nnvc2" Jul 07 22:04:47.676057 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:47.676007 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/84447fdf-4f2d-45fd-9e94-acce197ae75b-service-ca-bundle\") pod \"router-default-67496c4dcc-nnvc2\" (UID: \"84447fdf-4f2d-45fd-9e94-acce197ae75b\") " pod="openshift-ingress/router-default-67496c4dcc-nnvc2" Jul 07 22:04:47.678120 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:47.678050 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/84447fdf-4f2d-45fd-9e94-acce197ae75b-metrics-certs\") pod \"router-default-67496c4dcc-nnvc2\" (UID: \"84447fdf-4f2d-45fd-9e94-acce197ae75b\") " pod="openshift-ingress/router-default-67496c4dcc-nnvc2" Jul 07 22:04:47.954071 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:47.953989 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-747vt\"" Jul 07 22:04:47.962186 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:47.962160 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-67496c4dcc-nnvc2" Jul 07 22:04:48.978008 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:48.977929 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-685bfb5f64-q7rgl"] Jul 07 22:04:50.240105 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:04:50.240060 2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9c45f5f_a6f4_406c_a8f8_3d474a0dce97.slice/crio-52ecb670380dcf08f4b647a9bbc3fe2f3dac97b1866c62e86ca9048270c5e275 WatchSource:0}: Error finding container 52ecb670380dcf08f4b647a9bbc3fe2f3dac97b1866c62e86ca9048270c5e275: Status 404 returned error can't find the container with id 52ecb670380dcf08f4b647a9bbc3fe2f3dac97b1866c62e86ca9048270c5e275 Jul 07 22:04:50.417112 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:50.415918 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-67496c4dcc-nnvc2"] Jul 07 22:04:50.420950 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:04:50.420907 2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84447fdf_4f2d_45fd_9e94_acce197ae75b.slice/crio-f31f496fb956725a41238b9d24d75019cbcde02ded47465266e1bf280dc789a8 WatchSource:0}: Error finding container f31f496fb956725a41238b9d24d75019cbcde02ded47465266e1bf280dc789a8: Status 404 returned error can't find the container with id f31f496fb956725a41238b9d24d75019cbcde02ded47465266e1bf280dc789a8 Jul 07 22:04:50.643657 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:50.643626 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-88qj9" event={"ID":"f7c3d30c-9d3a-43c7-9a22-92d1fca6d2ea","Type":"ContainerStarted","Data":"478ed146d1a83953b090d92a1e6d9e3fc9feef720dcbe005d8fd62a6f94d6ff6"} Jul 07 22:04:50.643830 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:50.643681 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-88qj9" event={"ID":"f7c3d30c-9d3a-43c7-9a22-92d1fca6d2ea","Type":"ContainerStarted","Data":"94159b13304b4731e9cd83d8566cbbd963ca30713bf1d0f1f647c19e40f5e770"} Jul 07 22:04:50.645772 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:50.645746 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-9rmdm" event={"ID":"add27e45-747d-4e8d-9c57-4ca7f6268343","Type":"ContainerStarted","Data":"394c03c89bdbcc467c5e4731ffe2756670fb933a895fe8d49b8cf1a2f00c31d6"} Jul 07 22:04:50.647827 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:50.647584 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-bqv92" event={"ID":"c80e0cce-1240-4a8f-873b-abd71e85b59a","Type":"ContainerStarted","Data":"5ebcf3f3a2aec8b1b631aa6104be342077d1eb68d207fb19811de9ac8d74cc32"} Jul 07 22:04:50.647827 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:50.647631 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-bqv92" event={"ID":"c80e0cce-1240-4a8f-873b-abd71e85b59a","Type":"ContainerStarted","Data":"d89f2ae09ff968601f4b6b2d46e1359069891d74ee04bd85d47de6f411c7a805"} Jul 07 22:04:50.647827 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:50.647719 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-bqv92" Jul 07 22:04:50.649135 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:50.649109 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-82xqf" event={"ID":"30024b54-40f6-40b2-9b04-a82f5650ceba","Type":"ContainerStarted","Data":"d88b6c5b67a52c715142a51a739a30bb9f3c5b15a7de9e652c0405eccd45e1dd"} Jul 07 22:04:50.649267 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:50.649244 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-82xqf" Jul 07 22:04:50.650638 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:50.650613 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-nqfk8" event={"ID":"01e846e0-a94f-446e-8ec7-fdeb51040759","Type":"ContainerStarted","Data":"67aba2614cb20e031daf7987753860f764c310dae9266b27ec2a40d8903b30be"} Jul 07 22:04:50.652007 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:50.651984 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-67496c4dcc-nnvc2" event={"ID":"84447fdf-4f2d-45fd-9e94-acce197ae75b","Type":"ContainerStarted","Data":"59cdf94136f744dc29028e19a06feec029e70a0998586635d2ade39a6943408c"} Jul 07 22:04:50.652135 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:50.652023 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-67496c4dcc-nnvc2" event={"ID":"84447fdf-4f2d-45fd-9e94-acce197ae75b","Type":"ContainerStarted","Data":"f31f496fb956725a41238b9d24d75019cbcde02ded47465266e1bf280dc789a8"} Jul 07 22:04:50.652993 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:50.652972 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-q7rgl" event={"ID":"f9c45f5f-a6f4-406c-a8f8-3d474a0dce97","Type":"ContainerStarted","Data":"52ecb670380dcf08f4b647a9bbc3fe2f3dac97b1866c62e86ca9048270c5e275"} Jul 07 22:04:50.661444 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:50.661407 2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-88qj9" podStartSLOduration=66.841996379 podStartE2EDuration="1m12.66139367s" totalImagesPullingTime="5.819397291s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 22:03:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:04:50.659995507 +0000 UTC m=+72.974218527" watchObservedRunningTime="2026-07-07 22:04:50.66139367 +0000 UTC m=+72.975616689" Jul 07 22:04:50.679817 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:50.679765 2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-9rmdm" podStartSLOduration=16.751887078 podStartE2EDuration="27.679745927s" totalImagesPullingTime="10.927858849s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 22:04:23 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:04:50.678869835 +0000 UTC m=+72.993092866" watchObservedRunningTime="2026-07-07 22:04:50.679745927 +0000 UTC m=+72.993968948" Jul 07 22:04:50.693056 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:50.693009 2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-nqfk8" podStartSLOduration=66.792814986 podStartE2EDuration="1m12.692995017s" totalImagesPullingTime="5.900180031s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 22:03:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:04:50.692594186 +0000 UTC m=+73.006817205" watchObservedRunningTime="2026-07-07 22:04:50.692995017 +0000 UTC m=+73.007218035" Jul 07 22:04:50.707751 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:50.707709 2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-bqv92" podStartSLOduration=33.758092436 podStartE2EDuration="38.707684982s" totalImagesPullingTime="4.949592546s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 22:04:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:04:50.707465387 +0000 UTC m=+73.021688407" watchObservedRunningTime="2026-07-07 22:04:50.707684982 +0000 UTC m=+73.021907984" Jul 07 22:04:50.725233 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:50.725180 2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-67496c4dcc-nnvc2" podStartSLOduration=35.725165245 podStartE2EDuration="35.725165245s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 22:04:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:04:50.724781337 +0000 UTC m=+73.039004359" watchObservedRunningTime="2026-07-07 22:04:50.725165245 +0000 UTC m=+73.039388264" Jul 07 22:04:50.740725 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:50.740671 2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-82xqf" podStartSLOduration=66.913244362 podStartE2EDuration="1m12.740653351s" totalImagesPullingTime="5.827408989s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 22:03:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:04:50.739970917 +0000 UTC m=+73.054193937" watchObservedRunningTime="2026-07-07 22:04:50.740653351 +0000 UTC m=+73.054876372" Jul 07 22:04:50.962743 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:50.962655 2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-67496c4dcc-nnvc2" Jul 07 22:04:50.965774 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:50.965748 2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-67496c4dcc-nnvc2" Jul 07 22:04:51.657136 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:51.657107 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-67496c4dcc-nnvc2" Jul 07 22:04:51.658197 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:51.658173 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-67496c4dcc-nnvc2" Jul 07 22:04:53.594167 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:53.594129 2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-76cf759c89-6j67d"] Jul 07 22:04:53.623129 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:53.623097 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-76cf759c89-6j67d"] Jul 07 22:04:53.623279 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:53.623181 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76cf759c89-6j67d" Jul 07 22:04:53.626057 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:53.626017 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 07 22:04:53.626224 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:53.626108 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 07 22:04:53.626224 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:53.626122 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 07 22:04:53.627065 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:53.626879 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 07 22:04:53.627065 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:53.626879 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 07 22:04:53.627065 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:53.626925 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-hd2gj\"" Jul 07 22:04:53.631769 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:53.631551 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 07 22:04:53.723628 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:53.723590 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8af4f328-8dbc-41eb-8e14-a82d34bbb3eb-trusted-ca-bundle\") pod \"console-76cf759c89-6j67d\" (UID: \"8af4f328-8dbc-41eb-8e14-a82d34bbb3eb\") " pod="openshift-console/console-76cf759c89-6j67d" Jul 07 22:04:53.723628 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:53.723633 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8af4f328-8dbc-41eb-8e14-a82d34bbb3eb-oauth-serving-cert\") pod \"console-76cf759c89-6j67d\" (UID: \"8af4f328-8dbc-41eb-8e14-a82d34bbb3eb\") " pod="openshift-console/console-76cf759c89-6j67d" Jul 07 22:04:53.723886 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:53.723680 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8af4f328-8dbc-41eb-8e14-a82d34bbb3eb-service-ca\") pod \"console-76cf759c89-6j67d\" (UID: \"8af4f328-8dbc-41eb-8e14-a82d34bbb3eb\") " pod="openshift-console/console-76cf759c89-6j67d" Jul 07 22:04:53.723886 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:53.723801 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8af4f328-8dbc-41eb-8e14-a82d34bbb3eb-console-serving-cert\") pod \"console-76cf759c89-6j67d\" (UID: \"8af4f328-8dbc-41eb-8e14-a82d34bbb3eb\") " pod="openshift-console/console-76cf759c89-6j67d" Jul 07 22:04:53.723886 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:53.723850 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8af4f328-8dbc-41eb-8e14-a82d34bbb3eb-console-config\") pod \"console-76cf759c89-6j67d\" (UID: \"8af4f328-8dbc-41eb-8e14-a82d34bbb3eb\") " pod="openshift-console/console-76cf759c89-6j67d" Jul 07 22:04:53.724026 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:53.723924 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nf2rm\" (UniqueName: \"kubernetes.io/projected/8af4f328-8dbc-41eb-8e14-a82d34bbb3eb-kube-api-access-nf2rm\") pod \"console-76cf759c89-6j67d\" (UID: \"8af4f328-8dbc-41eb-8e14-a82d34bbb3eb\") " pod="openshift-console/console-76cf759c89-6j67d" Jul 07 22:04:53.724026 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:53.724003 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8af4f328-8dbc-41eb-8e14-a82d34bbb3eb-console-oauth-config\") pod \"console-76cf759c89-6j67d\" (UID: \"8af4f328-8dbc-41eb-8e14-a82d34bbb3eb\") " pod="openshift-console/console-76cf759c89-6j67d" Jul 07 22:04:53.824768 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:53.824727 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nf2rm\" (UniqueName: \"kubernetes.io/projected/8af4f328-8dbc-41eb-8e14-a82d34bbb3eb-kube-api-access-nf2rm\") pod \"console-76cf759c89-6j67d\" (UID: \"8af4f328-8dbc-41eb-8e14-a82d34bbb3eb\") " pod="openshift-console/console-76cf759c89-6j67d" Jul 07 22:04:53.824953 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:53.824784 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8af4f328-8dbc-41eb-8e14-a82d34bbb3eb-console-oauth-config\") pod \"console-76cf759c89-6j67d\" (UID: \"8af4f328-8dbc-41eb-8e14-a82d34bbb3eb\") " pod="openshift-console/console-76cf759c89-6j67d" Jul 07 22:04:53.824953 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:53.824829 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8af4f328-8dbc-41eb-8e14-a82d34bbb3eb-trusted-ca-bundle\") pod \"console-76cf759c89-6j67d\" (UID: \"8af4f328-8dbc-41eb-8e14-a82d34bbb3eb\") " pod="openshift-console/console-76cf759c89-6j67d" Jul 07 22:04:53.824953 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:53.824851 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8af4f328-8dbc-41eb-8e14-a82d34bbb3eb-oauth-serving-cert\") pod \"console-76cf759c89-6j67d\" (UID: \"8af4f328-8dbc-41eb-8e14-a82d34bbb3eb\") " pod="openshift-console/console-76cf759c89-6j67d" Jul 07 22:04:53.824953 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:53.824874 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8af4f328-8dbc-41eb-8e14-a82d34bbb3eb-service-ca\") pod \"console-76cf759c89-6j67d\" (UID: \"8af4f328-8dbc-41eb-8e14-a82d34bbb3eb\") " pod="openshift-console/console-76cf759c89-6j67d" Jul 07 22:04:53.824953 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:53.824902 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8af4f328-8dbc-41eb-8e14-a82d34bbb3eb-console-serving-cert\") pod \"console-76cf759c89-6j67d\" (UID: \"8af4f328-8dbc-41eb-8e14-a82d34bbb3eb\") " pod="openshift-console/console-76cf759c89-6j67d" Jul 07 22:04:53.824953 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:53.824926 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8af4f328-8dbc-41eb-8e14-a82d34bbb3eb-console-config\") pod \"console-76cf759c89-6j67d\" (UID: \"8af4f328-8dbc-41eb-8e14-a82d34bbb3eb\") " pod="openshift-console/console-76cf759c89-6j67d" Jul 07 22:04:53.825708 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:53.825673 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8af4f328-8dbc-41eb-8e14-a82d34bbb3eb-console-config\") pod \"console-76cf759c89-6j67d\" (UID: \"8af4f328-8dbc-41eb-8e14-a82d34bbb3eb\") " pod="openshift-console/console-76cf759c89-6j67d" Jul 07 22:04:53.825822 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:53.825673 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8af4f328-8dbc-41eb-8e14-a82d34bbb3eb-oauth-serving-cert\") pod \"console-76cf759c89-6j67d\" (UID: \"8af4f328-8dbc-41eb-8e14-a82d34bbb3eb\") " pod="openshift-console/console-76cf759c89-6j67d" Jul 07 22:04:53.825892 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:53.825868 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8af4f328-8dbc-41eb-8e14-a82d34bbb3eb-trusted-ca-bundle\") pod \"console-76cf759c89-6j67d\" (UID: \"8af4f328-8dbc-41eb-8e14-a82d34bbb3eb\") " pod="openshift-console/console-76cf759c89-6j67d" Jul 07 22:04:53.826233 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:53.826202 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8af4f328-8dbc-41eb-8e14-a82d34bbb3eb-service-ca\") pod \"console-76cf759c89-6j67d\" (UID: \"8af4f328-8dbc-41eb-8e14-a82d34bbb3eb\") " pod="openshift-console/console-76cf759c89-6j67d" Jul 07 22:04:53.827569 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:53.827545 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8af4f328-8dbc-41eb-8e14-a82d34bbb3eb-console-oauth-config\") pod \"console-76cf759c89-6j67d\" (UID: \"8af4f328-8dbc-41eb-8e14-a82d34bbb3eb\") " pod="openshift-console/console-76cf759c89-6j67d" Jul 07 22:04:53.827845 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:53.827819 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8af4f328-8dbc-41eb-8e14-a82d34bbb3eb-console-serving-cert\") pod \"console-76cf759c89-6j67d\" (UID: \"8af4f328-8dbc-41eb-8e14-a82d34bbb3eb\") " pod="openshift-console/console-76cf759c89-6j67d" Jul 07 22:04:53.832402 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:53.832380 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nf2rm\" (UniqueName: \"kubernetes.io/projected/8af4f328-8dbc-41eb-8e14-a82d34bbb3eb-kube-api-access-nf2rm\") pod \"console-76cf759c89-6j67d\" (UID: \"8af4f328-8dbc-41eb-8e14-a82d34bbb3eb\") " pod="openshift-console/console-76cf759c89-6j67d" Jul 07 22:04:53.935334 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:53.935247 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76cf759c89-6j67d" Jul 07 22:04:54.095898 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:54.095875 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-76cf759c89-6j67d"] Jul 07 22:04:54.098623 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:04:54.098599 2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8af4f328_8dbc_41eb_8e14_a82d34bbb3eb.slice/crio-eecf326490234f372c94d9c9cb9c16aab5d421255e5605fcaa280756c0874a79 WatchSource:0}: Error finding container eecf326490234f372c94d9c9cb9c16aab5d421255e5605fcaa280756c0874a79: Status 404 returned error can't find the container with id eecf326490234f372c94d9c9cb9c16aab5d421255e5605fcaa280756c0874a79 Jul 07 22:04:54.672574 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:54.672484 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76cf759c89-6j67d" event={"ID":"8af4f328-8dbc-41eb-8e14-a82d34bbb3eb","Type":"ContainerStarted","Data":"eecf326490234f372c94d9c9cb9c16aab5d421255e5605fcaa280756c0874a79"} Jul 07 22:04:55.165023 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:55.164987 2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-vh2rt"] Jul 07 22:04:55.167209 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:55.167183 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-vh2rt" Jul 07 22:04:55.169522 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:55.169499 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 07 22:04:55.169743 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:55.169729 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-f4c96\"" Jul 07 22:04:55.177898 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:55.177875 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-vh2rt"] Jul 07 22:04:55.236439 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:55.236399 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/d81a9215-297f-45fa-bc60-1e7993111c96-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-vh2rt\" (UID: \"d81a9215-297f-45fa-bc60-1e7993111c96\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-vh2rt" Jul 07 22:04:55.336948 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:55.336895 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/d81a9215-297f-45fa-bc60-1e7993111c96-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-vh2rt\" (UID: \"d81a9215-297f-45fa-bc60-1e7993111c96\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-vh2rt" Jul 07 22:04:55.337155 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:04:55.337061 2545 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found Jul 07 22:04:55.337226 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:04:55.337161 2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d81a9215-297f-45fa-bc60-1e7993111c96-tls-certificates podName:d81a9215-297f-45fa-bc60-1e7993111c96 nodeName:}" failed. No retries permitted until 2026-07-07 22:04:55.837139023 +0000 UTC m=+78.151362023 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/d81a9215-297f-45fa-bc60-1e7993111c96-tls-certificates") pod "prometheus-operator-admission-webhook-d559d66d5-vh2rt" (UID: "d81a9215-297f-45fa-bc60-1e7993111c96") : secret "prometheus-operator-admission-webhook-tls" not found Jul 07 22:04:55.841347 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:55.841301 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/d81a9215-297f-45fa-bc60-1e7993111c96-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-vh2rt\" (UID: \"d81a9215-297f-45fa-bc60-1e7993111c96\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-vh2rt" Jul 07 22:04:55.844420 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:55.844386 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/d81a9215-297f-45fa-bc60-1e7993111c96-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-vh2rt\" (UID: \"d81a9215-297f-45fa-bc60-1e7993111c96\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-vh2rt" Jul 07 22:04:56.078327 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:56.078292 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-vh2rt" Jul 07 22:04:57.338013 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:57.337972 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-vh2rt"] Jul 07 22:04:57.341665 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:04:57.341636 2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd81a9215_297f_45fa_bc60_1e7993111c96.slice/crio-5ce41d8cd9d116225ff5b3f88c84189598f2156c9857d753cdfc48513ef0c937 WatchSource:0}: Error finding container 5ce41d8cd9d116225ff5b3f88c84189598f2156c9857d753cdfc48513ef0c937: Status 404 returned error can't find the container with id 5ce41d8cd9d116225ff5b3f88c84189598f2156c9857d753cdfc48513ef0c937 Jul 07 22:04:57.687495 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:57.687452 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76cf759c89-6j67d" event={"ID":"8af4f328-8dbc-41eb-8e14-a82d34bbb3eb","Type":"ContainerStarted","Data":"d4e368eb43dfce5d732f4892864cfc625280e229e9269121579d17b1622562db"} Jul 07 22:04:57.688653 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:57.688627 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-vh2rt" event={"ID":"d81a9215-297f-45fa-bc60-1e7993111c96","Type":"ContainerStarted","Data":"5ce41d8cd9d116225ff5b3f88c84189598f2156c9857d753cdfc48513ef0c937"} Jul 07 22:04:57.705255 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:57.705203 2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-76cf759c89-6j67d" podStartSLOduration=1.565959154 podStartE2EDuration="4.705187337s" totalImagesPullingTime="3.139228183s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 22:04:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:04:57.704144412 +0000 UTC m=+80.018367433" watchObservedRunningTime="2026-07-07 22:04:57.705187337 +0000 UTC m=+80.019410433" Jul 07 22:04:58.695007 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:58.694912 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-vh2rt" event={"ID":"d81a9215-297f-45fa-bc60-1e7993111c96","Type":"ContainerStarted","Data":"f5a33ab464302abfcbe86f5c94c7d57e3d98415d508eb5ead6d9ba60ced583b5"} Jul 07 22:04:58.710979 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:58.710929 2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-vh2rt" podStartSLOduration=2.626913105 podStartE2EDuration="3.71091195s" totalImagesPullingTime="1.083998845s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 22:04:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:04:58.710040296 +0000 UTC m=+81.024263317" watchObservedRunningTime="2026-07-07 22:04:58.71091195 +0000 UTC m=+81.025134968" Jul 07 22:04:59.699461 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:59.699116 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-vh2rt" Jul 07 22:04:59.704615 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:04:59.704583 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-vh2rt" Jul 07 22:05:00.221719 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:00.221684 2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-tvxfb"] Jul 07 22:05:00.224837 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:00.224812 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-tvxfb" Jul 07 22:05:00.227575 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:00.227370 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 07 22:05:00.227575 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:00.227382 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-nqlsp\"" Jul 07 22:05:00.227575 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:00.227450 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 07 22:05:00.227575 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:00.227368 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 07 22:05:00.227575 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:00.227372 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 07 22:05:00.227575 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:00.227523 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 07 22:05:00.233845 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:00.233817 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-tvxfb"] Jul 07 22:05:00.282341 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:00.282309 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hlkr8\" (UniqueName: \"kubernetes.io/projected/7a0aec9a-a95c-4742-9c09-54c6f03ea2bb-kube-api-access-hlkr8\") pod \"prometheus-operator-567f7b8b7c-tvxfb\" (UID: \"7a0aec9a-a95c-4742-9c09-54c6f03ea2bb\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-tvxfb" Jul 07 22:05:00.282523 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:00.282359 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/7a0aec9a-a95c-4742-9c09-54c6f03ea2bb-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-tvxfb\" (UID: \"7a0aec9a-a95c-4742-9c09-54c6f03ea2bb\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-tvxfb" Jul 07 22:05:00.282523 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:00.282391 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7a0aec9a-a95c-4742-9c09-54c6f03ea2bb-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-tvxfb\" (UID: \"7a0aec9a-a95c-4742-9c09-54c6f03ea2bb\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-tvxfb" Jul 07 22:05:00.282523 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:00.282450 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7a0aec9a-a95c-4742-9c09-54c6f03ea2bb-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-tvxfb\" (UID: \"7a0aec9a-a95c-4742-9c09-54c6f03ea2bb\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-tvxfb" Jul 07 22:05:00.383066 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:00.383028 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hlkr8\" (UniqueName: \"kubernetes.io/projected/7a0aec9a-a95c-4742-9c09-54c6f03ea2bb-kube-api-access-hlkr8\") pod \"prometheus-operator-567f7b8b7c-tvxfb\" (UID: \"7a0aec9a-a95c-4742-9c09-54c6f03ea2bb\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-tvxfb" Jul 07 22:05:00.383261 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:00.383118 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/7a0aec9a-a95c-4742-9c09-54c6f03ea2bb-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-tvxfb\" (UID: \"7a0aec9a-a95c-4742-9c09-54c6f03ea2bb\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-tvxfb" Jul 07 22:05:00.383261 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:00.383152 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7a0aec9a-a95c-4742-9c09-54c6f03ea2bb-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-tvxfb\" (UID: \"7a0aec9a-a95c-4742-9c09-54c6f03ea2bb\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-tvxfb" Jul 07 22:05:00.383261 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:00.383193 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7a0aec9a-a95c-4742-9c09-54c6f03ea2bb-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-tvxfb\" (UID: \"7a0aec9a-a95c-4742-9c09-54c6f03ea2bb\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-tvxfb" Jul 07 22:05:00.383261 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:05:00.383253 2545 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found Jul 07 22:05:00.383521 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:05:00.383322 2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7a0aec9a-a95c-4742-9c09-54c6f03ea2bb-prometheus-operator-tls podName:7a0aec9a-a95c-4742-9c09-54c6f03ea2bb nodeName:}" failed. No retries permitted until 2026-07-07 22:05:00.883301626 +0000 UTC m=+83.197524636 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/7a0aec9a-a95c-4742-9c09-54c6f03ea2bb-prometheus-operator-tls") pod "prometheus-operator-567f7b8b7c-tvxfb" (UID: "7a0aec9a-a95c-4742-9c09-54c6f03ea2bb") : secret "prometheus-operator-tls" not found Jul 07 22:05:00.383936 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:00.383907 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7a0aec9a-a95c-4742-9c09-54c6f03ea2bb-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-tvxfb\" (UID: \"7a0aec9a-a95c-4742-9c09-54c6f03ea2bb\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-tvxfb" Jul 07 22:05:00.385964 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:00.385943 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7a0aec9a-a95c-4742-9c09-54c6f03ea2bb-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-tvxfb\" (UID: \"7a0aec9a-a95c-4742-9c09-54c6f03ea2bb\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-tvxfb" Jul 07 22:05:00.391575 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:00.391529 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hlkr8\" (UniqueName: \"kubernetes.io/projected/7a0aec9a-a95c-4742-9c09-54c6f03ea2bb-kube-api-access-hlkr8\") pod \"prometheus-operator-567f7b8b7c-tvxfb\" (UID: \"7a0aec9a-a95c-4742-9c09-54c6f03ea2bb\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-tvxfb" Jul 07 22:05:00.659426 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:00.659350 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-bqv92" Jul 07 22:05:00.886895 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:00.886857 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/7a0aec9a-a95c-4742-9c09-54c6f03ea2bb-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-tvxfb\" (UID: \"7a0aec9a-a95c-4742-9c09-54c6f03ea2bb\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-tvxfb" Jul 07 22:05:00.889297 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:00.889279 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/7a0aec9a-a95c-4742-9c09-54c6f03ea2bb-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-tvxfb\" (UID: \"7a0aec9a-a95c-4742-9c09-54c6f03ea2bb\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-tvxfb" Jul 07 22:05:01.139291 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:01.139254 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-tvxfb" Jul 07 22:05:01.268123 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:01.268090 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-tvxfb"] Jul 07 22:05:01.270900 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:05:01.270868 2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a0aec9a_a95c_4742_9c09_54c6f03ea2bb.slice/crio-b80abb18a50024150bb953823ba31e372bd2f34e5fa801358ace314b9f43e6ce WatchSource:0}: Error finding container b80abb18a50024150bb953823ba31e372bd2f34e5fa801358ace314b9f43e6ce: Status 404 returned error can't find the container with id b80abb18a50024150bb953823ba31e372bd2f34e5fa801358ace314b9f43e6ce Jul 07 22:05:01.708669 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:01.708636 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-tvxfb" event={"ID":"7a0aec9a-a95c-4742-9c09-54c6f03ea2bb","Type":"ContainerStarted","Data":"b80abb18a50024150bb953823ba31e372bd2f34e5fa801358ace314b9f43e6ce"} Jul 07 22:05:03.935818 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:03.935777 2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-76cf759c89-6j67d" Jul 07 22:05:03.936254 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:03.935858 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-76cf759c89-6j67d" Jul 07 22:05:03.941501 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:03.941474 2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-76cf759c89-6j67d" Jul 07 22:05:04.726181 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:04.726142 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-76cf759c89-6j67d" Jul 07 22:05:05.056205 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:05.056122 2545 patch_prober.go:28] interesting pod/image-registry-b9885dbc7-lx8h7 container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 07 22:05:05.056205 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:05.056183 2545 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-b9885dbc7-lx8h7" podUID="78996163-d007-4f3c-ac3a-44902fe9be66" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 07 22:05:06.633179 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:06.633150 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-b9885dbc7-lx8h7" Jul 07 22:05:09.135147 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:09.135115 2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-b9885dbc7-lx8h7"] Jul 07 22:05:10.747591 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:10.747550 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-tvxfb" event={"ID":"7a0aec9a-a95c-4742-9c09-54c6f03ea2bb","Type":"ContainerStarted","Data":"95e3d7f37aafd5bba0c84322dd0a7b41d01fda7d1024669e76fa842a3eb7f3a2"} Jul 07 22:05:10.748055 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:10.747621 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-tvxfb" event={"ID":"7a0aec9a-a95c-4742-9c09-54c6f03ea2bb","Type":"ContainerStarted","Data":"127d4734a75fb0e8c5c6db9ace50c49bd29f6c98c51ce5c02f49b0deb51a5551"} Jul 07 22:05:10.749921 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:10.749884 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-685bfb5f64-q7rgl" event={"ID":"f9c45f5f-a6f4-406c-a8f8-3d474a0dce97","Type":"ContainerStarted","Data":"35ad8af2c5e98fa5c913886f3c07881fe5c2fd7f30761f06950977d873543bee"} Jul 07 22:05:10.750101 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:10.750061 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-685bfb5f64-q7rgl" Jul 07 22:05:10.763997 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:10.763956 2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-tvxfb" podStartSLOduration=2.139590999 podStartE2EDuration="10.763939524s" totalImagesPullingTime="8.624348525s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 22:05:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:05:10.763609507 +0000 UTC m=+93.077832556" watchObservedRunningTime="2026-07-07 22:05:10.763939524 +0000 UTC m=+93.078162547" Jul 07 22:05:10.766011 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:10.765985 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-685bfb5f64-q7rgl" Jul 07 22:05:10.779705 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:10.779652 2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-685bfb5f64-q7rgl" podStartSLOduration=5.117251169 podStartE2EDuration="24.779637535s" totalImagesPullingTime="19.662386366s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 22:04:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:05:10.778186483 +0000 UTC m=+93.092409503" watchObservedRunningTime="2026-07-07 22:05:10.779637535 +0000 UTC m=+93.093860553" Jul 07 22:05:12.554727 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:12.554685 2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-nkc2m"] Jul 07 22:05:12.582837 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:12.582097 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-nkc2m"] Jul 07 22:05:12.582837 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:12.582156 2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-j629n"] Jul 07 22:05:12.582837 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:12.582207 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-nkc2m" Jul 07 22:05:12.584488 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:12.584464 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 07 22:05:12.584616 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:12.584528 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-77sqk\"" Jul 07 22:05:12.584799 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:12.584781 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 07 22:05:12.600459 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:12.600432 2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-9hl2n"] Jul 07 22:05:12.600600 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:12.600567 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-j629n" Jul 07 22:05:12.603574 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:12.603198 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 07 22:05:12.603574 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:12.603237 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 07 22:05:12.603574 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:12.603261 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 07 22:05:12.603574 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:12.603296 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-tnstn\"" Jul 07 22:05:12.620564 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:12.620545 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-j629n"] Jul 07 22:05:12.620674 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:12.620661 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-9hl2n" Jul 07 22:05:12.623224 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:12.623201 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 07 22:05:12.623345 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:12.623252 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-c9wff\"" Jul 07 22:05:12.623433 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:12.623359 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 07 22:05:12.623542 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:12.623473 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 07 22:05:12.690631 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:12.690594 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/60b9212f-09a8-4beb-a1f5-c242e7cd135c-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-j629n\" (UID: \"60b9212f-09a8-4beb-a1f5-c242e7cd135c\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-j629n" Jul 07 22:05:12.690631 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:12.690639 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/60b9212f-09a8-4beb-a1f5-c242e7cd135c-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-j629n\" (UID: \"60b9212f-09a8-4beb-a1f5-c242e7cd135c\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-j629n" Jul 07 22:05:12.690931 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:12.690675 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ptz65\" (UniqueName: \"kubernetes.io/projected/60b9212f-09a8-4beb-a1f5-c242e7cd135c-kube-api-access-ptz65\") pod \"kube-state-metrics-7c9dbb8b7d-j629n\" (UID: \"60b9212f-09a8-4beb-a1f5-c242e7cd135c\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-j629n" Jul 07 22:05:12.690931 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:12.690728 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cpgqp\" (UniqueName: \"kubernetes.io/projected/1e0d5872-f2ba-4d37-b3ad-80f1d502b936-kube-api-access-cpgqp\") pod \"openshift-state-metrics-54cf96dfcf-nkc2m\" (UID: \"1e0d5872-f2ba-4d37-b3ad-80f1d502b936\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-nkc2m" Jul 07 22:05:12.690931 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:12.690769 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1e0d5872-f2ba-4d37-b3ad-80f1d502b936-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-nkc2m\" (UID: \"1e0d5872-f2ba-4d37-b3ad-80f1d502b936\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-nkc2m" Jul 07 22:05:12.690931 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:12.690797 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/60b9212f-09a8-4beb-a1f5-c242e7cd135c-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-j629n\" (UID: \"60b9212f-09a8-4beb-a1f5-c242e7cd135c\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-j629n" Jul 07 22:05:12.690931 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:12.690836 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1e0d5872-f2ba-4d37-b3ad-80f1d502b936-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-nkc2m\" (UID: \"1e0d5872-f2ba-4d37-b3ad-80f1d502b936\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-nkc2m" Jul 07 22:05:12.690931 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:12.690865 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/60b9212f-09a8-4beb-a1f5-c242e7cd135c-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-j629n\" (UID: \"60b9212f-09a8-4beb-a1f5-c242e7cd135c\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-j629n" Jul 07 22:05:12.690931 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:12.690893 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/1e0d5872-f2ba-4d37-b3ad-80f1d502b936-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-nkc2m\" (UID: \"1e0d5872-f2ba-4d37-b3ad-80f1d502b936\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-nkc2m" Jul 07 22:05:12.690931 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:12.690923 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/60b9212f-09a8-4beb-a1f5-c242e7cd135c-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-j629n\" (UID: \"60b9212f-09a8-4beb-a1f5-c242e7cd135c\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-j629n" Jul 07 22:05:12.791546 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:12.791515 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/60b9212f-09a8-4beb-a1f5-c242e7cd135c-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-j629n\" (UID: \"60b9212f-09a8-4beb-a1f5-c242e7cd135c\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-j629n" Jul 07 22:05:12.791546 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:12.791550 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/60b9212f-09a8-4beb-a1f5-c242e7cd135c-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-j629n\" (UID: \"60b9212f-09a8-4beb-a1f5-c242e7cd135c\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-j629n" Jul 07 22:05:12.791794 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:12.791584 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4ae9b323-7862-4003-a018-71b5e4d4705a-root\") pod \"node-exporter-9hl2n\" (UID: \"4ae9b323-7862-4003-a018-71b5e4d4705a\") " pod="openshift-monitoring/node-exporter-9hl2n" Jul 07 22:05:12.791794 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:12.791607 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ptz65\" (UniqueName: \"kubernetes.io/projected/60b9212f-09a8-4beb-a1f5-c242e7cd135c-kube-api-access-ptz65\") pod \"kube-state-metrics-7c9dbb8b7d-j629n\" (UID: \"60b9212f-09a8-4beb-a1f5-c242e7cd135c\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-j629n" Jul 07 22:05:12.791794 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:12.791629 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4ae9b323-7862-4003-a018-71b5e4d4705a-sys\") pod \"node-exporter-9hl2n\" (UID: \"4ae9b323-7862-4003-a018-71b5e4d4705a\") " pod="openshift-monitoring/node-exporter-9hl2n" Jul 07 22:05:12.791954 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:12.791797 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4ae9b323-7862-4003-a018-71b5e4d4705a-metrics-client-ca\") pod \"node-exporter-9hl2n\" (UID: \"4ae9b323-7862-4003-a018-71b5e4d4705a\") " pod="openshift-monitoring/node-exporter-9hl2n" Jul 07 22:05:12.791954 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:12.791864 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4ae9b323-7862-4003-a018-71b5e4d4705a-node-exporter-wtmp\") pod \"node-exporter-9hl2n\" (UID: \"4ae9b323-7862-4003-a018-71b5e4d4705a\") " pod="openshift-monitoring/node-exporter-9hl2n" Jul 07 22:05:12.791954 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:12.791901 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4ae9b323-7862-4003-a018-71b5e4d4705a-node-exporter-accelerators-collector-config\") pod \"node-exporter-9hl2n\" (UID: \"4ae9b323-7862-4003-a018-71b5e4d4705a\") " pod="openshift-monitoring/node-exporter-9hl2n" Jul 07 22:05:12.792128 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:12.791964 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/60b9212f-09a8-4beb-a1f5-c242e7cd135c-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-j629n\" (UID: \"60b9212f-09a8-4beb-a1f5-c242e7cd135c\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-j629n" Jul 07 22:05:12.792128 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:12.791987 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cpgqp\" (UniqueName: \"kubernetes.io/projected/1e0d5872-f2ba-4d37-b3ad-80f1d502b936-kube-api-access-cpgqp\") pod \"openshift-state-metrics-54cf96dfcf-nkc2m\" (UID: \"1e0d5872-f2ba-4d37-b3ad-80f1d502b936\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-nkc2m" Jul 07 22:05:12.792128 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:12.792058 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1e0d5872-f2ba-4d37-b3ad-80f1d502b936-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-nkc2m\" (UID: \"1e0d5872-f2ba-4d37-b3ad-80f1d502b936\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-nkc2m" Jul 07 22:05:12.792291 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:12.792167 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/60b9212f-09a8-4beb-a1f5-c242e7cd135c-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-j629n\" (UID: \"60b9212f-09a8-4beb-a1f5-c242e7cd135c\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-j629n" Jul 07 22:05:12.792291 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:12.792225 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1e0d5872-f2ba-4d37-b3ad-80f1d502b936-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-nkc2m\" (UID: \"1e0d5872-f2ba-4d37-b3ad-80f1d502b936\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-nkc2m" Jul 07 22:05:12.792291 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:12.792256 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/60b9212f-09a8-4beb-a1f5-c242e7cd135c-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-j629n\" (UID: \"60b9212f-09a8-4beb-a1f5-c242e7cd135c\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-j629n" Jul 07 22:05:12.793196 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:12.792881 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/60b9212f-09a8-4beb-a1f5-c242e7cd135c-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-j629n\" (UID: \"60b9212f-09a8-4beb-a1f5-c242e7cd135c\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-j629n" Jul 07 22:05:12.793196 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:12.792884 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1e0d5872-f2ba-4d37-b3ad-80f1d502b936-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-nkc2m\" (UID: \"1e0d5872-f2ba-4d37-b3ad-80f1d502b936\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-nkc2m" Jul 07 22:05:12.793196 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:12.792927 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4ae9b323-7862-4003-a018-71b5e4d4705a-node-exporter-textfile\") pod \"node-exporter-9hl2n\" (UID: \"4ae9b323-7862-4003-a018-71b5e4d4705a\") " pod="openshift-monitoring/node-exporter-9hl2n" Jul 07 22:05:12.793196 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:12.792970 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-26nvt\" (UniqueName: \"kubernetes.io/projected/4ae9b323-7862-4003-a018-71b5e4d4705a-kube-api-access-26nvt\") pod \"node-exporter-9hl2n\" (UID: \"4ae9b323-7862-4003-a018-71b5e4d4705a\") " pod="openshift-monitoring/node-exporter-9hl2n" Jul 07 22:05:12.793196 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:12.793026 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4ae9b323-7862-4003-a018-71b5e4d4705a-node-exporter-tls\") pod \"node-exporter-9hl2n\" (UID: \"4ae9b323-7862-4003-a018-71b5e4d4705a\") " pod="openshift-monitoring/node-exporter-9hl2n" Jul 07 22:05:12.793196 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:12.793053 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4ae9b323-7862-4003-a018-71b5e4d4705a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-9hl2n\" (UID: \"4ae9b323-7862-4003-a018-71b5e4d4705a\") " pod="openshift-monitoring/node-exporter-9hl2n" Jul 07 22:05:12.793196 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:12.793114 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/1e0d5872-f2ba-4d37-b3ad-80f1d502b936-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-nkc2m\" (UID: \"1e0d5872-f2ba-4d37-b3ad-80f1d502b936\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-nkc2m" Jul 07 22:05:12.793196 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:12.793188 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/60b9212f-09a8-4beb-a1f5-c242e7cd135c-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-j629n\" (UID: \"60b9212f-09a8-4beb-a1f5-c242e7cd135c\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-j629n" Jul 07 22:05:12.793804 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:12.793782 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/60b9212f-09a8-4beb-a1f5-c242e7cd135c-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-j629n\" (UID: \"60b9212f-09a8-4beb-a1f5-c242e7cd135c\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-j629n" Jul 07 22:05:12.794774 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:12.794741 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/60b9212f-09a8-4beb-a1f5-c242e7cd135c-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-j629n\" (UID: \"60b9212f-09a8-4beb-a1f5-c242e7cd135c\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-j629n" Jul 07 22:05:12.795885 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:12.795842 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1e0d5872-f2ba-4d37-b3ad-80f1d502b936-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-nkc2m\" (UID: \"1e0d5872-f2ba-4d37-b3ad-80f1d502b936\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-nkc2m" Jul 07 22:05:12.795997 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:12.795896 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/60b9212f-09a8-4beb-a1f5-c242e7cd135c-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-j629n\" (UID: \"60b9212f-09a8-4beb-a1f5-c242e7cd135c\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-j629n" Jul 07 22:05:12.795997 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:12.795977 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/1e0d5872-f2ba-4d37-b3ad-80f1d502b936-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-nkc2m\" (UID: \"1e0d5872-f2ba-4d37-b3ad-80f1d502b936\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-nkc2m" Jul 07 22:05:12.800557 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:12.800533 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ptz65\" (UniqueName: \"kubernetes.io/projected/60b9212f-09a8-4beb-a1f5-c242e7cd135c-kube-api-access-ptz65\") pod \"kube-state-metrics-7c9dbb8b7d-j629n\" (UID: \"60b9212f-09a8-4beb-a1f5-c242e7cd135c\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-j629n" Jul 07 22:05:12.800671 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:12.800542 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cpgqp\" (UniqueName: \"kubernetes.io/projected/1e0d5872-f2ba-4d37-b3ad-80f1d502b936-kube-api-access-cpgqp\") pod \"openshift-state-metrics-54cf96dfcf-nkc2m\" (UID: \"1e0d5872-f2ba-4d37-b3ad-80f1d502b936\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-nkc2m" Jul 07 22:05:12.894146 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:12.894044 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-nkc2m" Jul 07 22:05:12.894308 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:12.894238 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4ae9b323-7862-4003-a018-71b5e4d4705a-node-exporter-textfile\") pod \"node-exporter-9hl2n\" (UID: \"4ae9b323-7862-4003-a018-71b5e4d4705a\") " pod="openshift-monitoring/node-exporter-9hl2n" Jul 07 22:05:12.894308 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:12.894280 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-26nvt\" (UniqueName: \"kubernetes.io/projected/4ae9b323-7862-4003-a018-71b5e4d4705a-kube-api-access-26nvt\") pod \"node-exporter-9hl2n\" (UID: \"4ae9b323-7862-4003-a018-71b5e4d4705a\") " pod="openshift-monitoring/node-exporter-9hl2n" Jul 07 22:05:12.894435 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:12.894307 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4ae9b323-7862-4003-a018-71b5e4d4705a-node-exporter-tls\") pod \"node-exporter-9hl2n\" (UID: \"4ae9b323-7862-4003-a018-71b5e4d4705a\") " pod="openshift-monitoring/node-exporter-9hl2n" Jul 07 22:05:12.894435 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:12.894333 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4ae9b323-7862-4003-a018-71b5e4d4705a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-9hl2n\" (UID: \"4ae9b323-7862-4003-a018-71b5e4d4705a\") " pod="openshift-monitoring/node-exporter-9hl2n" Jul 07 22:05:12.894435 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:12.894385 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4ae9b323-7862-4003-a018-71b5e4d4705a-root\") pod \"node-exporter-9hl2n\" (UID: \"4ae9b323-7862-4003-a018-71b5e4d4705a\") " pod="openshift-monitoring/node-exporter-9hl2n" Jul 07 22:05:12.894435 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:12.894412 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4ae9b323-7862-4003-a018-71b5e4d4705a-sys\") pod \"node-exporter-9hl2n\" (UID: \"4ae9b323-7862-4003-a018-71b5e4d4705a\") " pod="openshift-monitoring/node-exporter-9hl2n" Jul 07 22:05:12.894632 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:12.894437 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4ae9b323-7862-4003-a018-71b5e4d4705a-metrics-client-ca\") pod \"node-exporter-9hl2n\" (UID: \"4ae9b323-7862-4003-a018-71b5e4d4705a\") " pod="openshift-monitoring/node-exporter-9hl2n" Jul 07 22:05:12.894632 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:12.894465 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4ae9b323-7862-4003-a018-71b5e4d4705a-node-exporter-wtmp\") pod \"node-exporter-9hl2n\" (UID: \"4ae9b323-7862-4003-a018-71b5e4d4705a\") " pod="openshift-monitoring/node-exporter-9hl2n" Jul 07 22:05:12.894632 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:12.894492 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4ae9b323-7862-4003-a018-71b5e4d4705a-node-exporter-accelerators-collector-config\") pod \"node-exporter-9hl2n\" (UID: \"4ae9b323-7862-4003-a018-71b5e4d4705a\") " pod="openshift-monitoring/node-exporter-9hl2n" Jul 07 22:05:12.894793 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:12.894628 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4ae9b323-7862-4003-a018-71b5e4d4705a-node-exporter-textfile\") pod \"node-exporter-9hl2n\" (UID: \"4ae9b323-7862-4003-a018-71b5e4d4705a\") " pod="openshift-monitoring/node-exporter-9hl2n" Jul 07 22:05:12.894793 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:12.894697 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4ae9b323-7862-4003-a018-71b5e4d4705a-root\") pod \"node-exporter-9hl2n\" (UID: \"4ae9b323-7862-4003-a018-71b5e4d4705a\") " pod="openshift-monitoring/node-exporter-9hl2n" Jul 07 22:05:12.894793 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:12.894729 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4ae9b323-7862-4003-a018-71b5e4d4705a-sys\") pod \"node-exporter-9hl2n\" (UID: \"4ae9b323-7862-4003-a018-71b5e4d4705a\") " pod="openshift-monitoring/node-exporter-9hl2n" Jul 07 22:05:12.894926 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:12.894817 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4ae9b323-7862-4003-a018-71b5e4d4705a-node-exporter-wtmp\") pod \"node-exporter-9hl2n\" (UID: \"4ae9b323-7862-4003-a018-71b5e4d4705a\") " pod="openshift-monitoring/node-exporter-9hl2n" Jul 07 22:05:12.895435 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:12.895413 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4ae9b323-7862-4003-a018-71b5e4d4705a-node-exporter-accelerators-collector-config\") pod \"node-exporter-9hl2n\" (UID: \"4ae9b323-7862-4003-a018-71b5e4d4705a\") " pod="openshift-monitoring/node-exporter-9hl2n" Jul 07 22:05:12.895563 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:12.895469 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4ae9b323-7862-4003-a018-71b5e4d4705a-metrics-client-ca\") pod \"node-exporter-9hl2n\" (UID: \"4ae9b323-7862-4003-a018-71b5e4d4705a\") " pod="openshift-monitoring/node-exporter-9hl2n" Jul 07 22:05:12.897408 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:12.897386 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4ae9b323-7862-4003-a018-71b5e4d4705a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-9hl2n\" (UID: \"4ae9b323-7862-4003-a018-71b5e4d4705a\") " pod="openshift-monitoring/node-exporter-9hl2n" Jul 07 22:05:12.897525 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:12.897472 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4ae9b323-7862-4003-a018-71b5e4d4705a-node-exporter-tls\") pod \"node-exporter-9hl2n\" (UID: \"4ae9b323-7862-4003-a018-71b5e4d4705a\") " pod="openshift-monitoring/node-exporter-9hl2n" Jul 07 22:05:12.902155 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:12.902130 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-26nvt\" (UniqueName: \"kubernetes.io/projected/4ae9b323-7862-4003-a018-71b5e4d4705a-kube-api-access-26nvt\") pod \"node-exporter-9hl2n\" (UID: \"4ae9b323-7862-4003-a018-71b5e4d4705a\") " pod="openshift-monitoring/node-exporter-9hl2n" Jul 07 22:05:12.912060 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:12.912030 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-j629n" Jul 07 22:05:12.930866 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:12.930843 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-9hl2n" Jul 07 22:05:12.976613 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:05:12.976574 2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ae9b323_7862_4003_a018_71b5e4d4705a.slice/crio-ecc4766118f8f18ad1c09c0b3a437e05b7b7e5b3e554b06e8751ef499857409a WatchSource:0}: Error finding container ecc4766118f8f18ad1c09c0b3a437e05b7b7e5b3e554b06e8751ef499857409a: Status 404 returned error can't find the container with id ecc4766118f8f18ad1c09c0b3a437e05b7b7e5b3e554b06e8751ef499857409a Jul 07 22:05:13.088022 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:13.087911 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-nkc2m"] Jul 07 22:05:13.091053 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:05:13.091019 2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e0d5872_f2ba_4d37_b3ad_80f1d502b936.slice/crio-90858194beb8044b5e3652418520729c9c0a6352908afb11eb45c105f0d93c9b WatchSource:0}: Error finding container 90858194beb8044b5e3652418520729c9c0a6352908afb11eb45c105f0d93c9b: Status 404 returned error can't find the container with id 90858194beb8044b5e3652418520729c9c0a6352908afb11eb45c105f0d93c9b Jul 07 22:05:13.108408 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:13.108386 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-j629n"] Jul 07 22:05:13.110500 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:05:13.110464 2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60b9212f_09a8_4beb_a1f5_c242e7cd135c.slice/crio-475bfb3cdfc98a3445575979a3a4fdbdda902169a1466982e7b645e1a7a3313b WatchSource:0}: Error finding container 475bfb3cdfc98a3445575979a3a4fdbdda902169a1466982e7b645e1a7a3313b: Status 404 returned error can't find the container with id 475bfb3cdfc98a3445575979a3a4fdbdda902169a1466982e7b645e1a7a3313b Jul 07 22:05:13.768060 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:13.768017 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9hl2n" event={"ID":"4ae9b323-7862-4003-a018-71b5e4d4705a","Type":"ContainerStarted","Data":"ecc4766118f8f18ad1c09c0b3a437e05b7b7e5b3e554b06e8751ef499857409a"} Jul 07 22:05:13.771791 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:13.771760 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-nkc2m" event={"ID":"1e0d5872-f2ba-4d37-b3ad-80f1d502b936","Type":"ContainerStarted","Data":"aa63eb8da41c443803c0016692fe37f3dccac92fad0cea6c3d2cde93c8c8580d"} Jul 07 22:05:13.771969 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:13.771822 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-nkc2m" event={"ID":"1e0d5872-f2ba-4d37-b3ad-80f1d502b936","Type":"ContainerStarted","Data":"16b09f7f0d79e8049ffb250cec2c14678e78873b200a851c369596a9b814ab4b"} Jul 07 22:05:13.771969 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:13.771851 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-nkc2m" event={"ID":"1e0d5872-f2ba-4d37-b3ad-80f1d502b936","Type":"ContainerStarted","Data":"90858194beb8044b5e3652418520729c9c0a6352908afb11eb45c105f0d93c9b"} Jul 07 22:05:13.774049 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:13.774024 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-j629n" event={"ID":"60b9212f-09a8-4beb-a1f5-c242e7cd135c","Type":"ContainerStarted","Data":"475bfb3cdfc98a3445575979a3a4fdbdda902169a1466982e7b645e1a7a3313b"} Jul 07 22:05:16.790987 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:16.790942 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-j629n" event={"ID":"60b9212f-09a8-4beb-a1f5-c242e7cd135c","Type":"ContainerStarted","Data":"d542251abd9c0734c0dfa9d37b2182efa10220379f4dce6b78aea1a2fc5edf8c"} Jul 07 22:05:16.791512 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:16.791024 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-j629n" event={"ID":"60b9212f-09a8-4beb-a1f5-c242e7cd135c","Type":"ContainerStarted","Data":"f755e3cd9dfe55829181a3b247f55763e49b03a6dcc42287d5fa7a8929e84446"} Jul 07 22:05:16.791512 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:16.791059 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-j629n" event={"ID":"60b9212f-09a8-4beb-a1f5-c242e7cd135c","Type":"ContainerStarted","Data":"707478e2bff291cb4fc12be5a93a604f1cd37844a97f4aa498efbb07f3be0002"} Jul 07 22:05:16.792818 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:16.792789 2545 generic.go:356] "Generic (PLEG): container finished" podID="4ae9b323-7862-4003-a018-71b5e4d4705a" containerID="c402844cd6447544703795f3337f3d2b45a514958fa0b7f08a6d7c8142d87c03" exitCode=0 Jul 07 22:05:16.792949 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:16.792880 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9hl2n" event={"ID":"4ae9b323-7862-4003-a018-71b5e4d4705a","Type":"ContainerDied","Data":"c402844cd6447544703795f3337f3d2b45a514958fa0b7f08a6d7c8142d87c03"} Jul 07 22:05:16.795387 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:16.795364 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-nkc2m" event={"ID":"1e0d5872-f2ba-4d37-b3ad-80f1d502b936","Type":"ContainerStarted","Data":"930f6b3057cd6a8c5398e9c4345e8df9573fb21da3f8bf6c43c1a5c76a9c2c9c"} Jul 07 22:05:16.814293 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:16.814244 2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-j629n" podStartSLOduration=2.07098821 podStartE2EDuration="4.81422915s" totalImagesPullingTime="2.74324094s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 22:05:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:05:16.808866877 +0000 UTC m=+99.123089912" watchObservedRunningTime="2026-07-07 22:05:16.81422915 +0000 UTC m=+99.128452170" Jul 07 22:05:16.847739 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:16.847686 2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-nkc2m" podStartSLOduration=2.386455527 podStartE2EDuration="4.847669761s" totalImagesPullingTime="2.461214234s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 22:05:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:05:16.847495543 +0000 UTC m=+99.161718562" watchObservedRunningTime="2026-07-07 22:05:16.847669761 +0000 UTC m=+99.161892789" Jul 07 22:05:16.870374 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:16.870348 2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-7c9bdddb8d-tlwzc"] Jul 07 22:05:16.908033 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:16.908007 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-7c9bdddb8d-tlwzc"] Jul 07 22:05:16.908239 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:16.908161 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-7c9bdddb8d-tlwzc" Jul 07 22:05:16.911368 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:16.910897 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 07 22:05:16.911368 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:16.910953 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 07 22:05:16.911368 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:16.910996 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-8mobg5dcltcf7\"" Jul 07 22:05:16.911368 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:16.911266 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-gdv8x\"" Jul 07 22:05:16.911650 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:16.911549 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 07 22:05:16.911811 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:16.911791 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 07 22:05:17.041785 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:17.041724 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kcvs6\" (UniqueName: \"kubernetes.io/projected/4ded1363-3a10-48da-9736-4c181ae38c2e-kube-api-access-kcvs6\") pod \"metrics-server-7c9bdddb8d-tlwzc\" (UID: \"4ded1363-3a10-48da-9736-4c181ae38c2e\") " pod="openshift-monitoring/metrics-server-7c9bdddb8d-tlwzc" Jul 07 22:05:17.041785 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:17.041763 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4ded1363-3a10-48da-9736-4c181ae38c2e-client-ca-bundle\") pod \"metrics-server-7c9bdddb8d-tlwzc\" (UID: \"4ded1363-3a10-48da-9736-4c181ae38c2e\") " pod="openshift-monitoring/metrics-server-7c9bdddb8d-tlwzc" Jul 07 22:05:17.041969 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:17.041863 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/4ded1363-3a10-48da-9736-4c181ae38c2e-secret-metrics-server-client-certs\") pod \"metrics-server-7c9bdddb8d-tlwzc\" (UID: \"4ded1363-3a10-48da-9736-4c181ae38c2e\") " pod="openshift-monitoring/metrics-server-7c9bdddb8d-tlwzc" Jul 07 22:05:17.041969 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:17.041942 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/4ded1363-3a10-48da-9736-4c181ae38c2e-secret-metrics-server-tls\") pod \"metrics-server-7c9bdddb8d-tlwzc\" (UID: \"4ded1363-3a10-48da-9736-4c181ae38c2e\") " pod="openshift-monitoring/metrics-server-7c9bdddb8d-tlwzc" Jul 07 22:05:17.041969 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:17.041965 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/4ded1363-3a10-48da-9736-4c181ae38c2e-metrics-server-audit-profiles\") pod \"metrics-server-7c9bdddb8d-tlwzc\" (UID: \"4ded1363-3a10-48da-9736-4c181ae38c2e\") " pod="openshift-monitoring/metrics-server-7c9bdddb8d-tlwzc" Jul 07 22:05:17.042161 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:17.041995 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/4ded1363-3a10-48da-9736-4c181ae38c2e-audit-log\") pod \"metrics-server-7c9bdddb8d-tlwzc\" (UID: \"4ded1363-3a10-48da-9736-4c181ae38c2e\") " pod="openshift-monitoring/metrics-server-7c9bdddb8d-tlwzc" Jul 07 22:05:17.042161 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:17.042029 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4ded1363-3a10-48da-9736-4c181ae38c2e-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7c9bdddb8d-tlwzc\" (UID: \"4ded1363-3a10-48da-9736-4c181ae38c2e\") " pod="openshift-monitoring/metrics-server-7c9bdddb8d-tlwzc" Jul 07 22:05:17.143322 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:17.143293 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/4ded1363-3a10-48da-9736-4c181ae38c2e-secret-metrics-server-client-certs\") pod \"metrics-server-7c9bdddb8d-tlwzc\" (UID: \"4ded1363-3a10-48da-9736-4c181ae38c2e\") " pod="openshift-monitoring/metrics-server-7c9bdddb8d-tlwzc" Jul 07 22:05:17.143471 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:17.143357 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/4ded1363-3a10-48da-9736-4c181ae38c2e-secret-metrics-server-tls\") pod \"metrics-server-7c9bdddb8d-tlwzc\" (UID: \"4ded1363-3a10-48da-9736-4c181ae38c2e\") " pod="openshift-monitoring/metrics-server-7c9bdddb8d-tlwzc" Jul 07 22:05:17.143544 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:17.143468 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/4ded1363-3a10-48da-9736-4c181ae38c2e-metrics-server-audit-profiles\") pod \"metrics-server-7c9bdddb8d-tlwzc\" (UID: \"4ded1363-3a10-48da-9736-4c181ae38c2e\") " pod="openshift-monitoring/metrics-server-7c9bdddb8d-tlwzc" Jul 07 22:05:17.143544 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:17.143520 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/4ded1363-3a10-48da-9736-4c181ae38c2e-audit-log\") pod \"metrics-server-7c9bdddb8d-tlwzc\" (UID: \"4ded1363-3a10-48da-9736-4c181ae38c2e\") " pod="openshift-monitoring/metrics-server-7c9bdddb8d-tlwzc" Jul 07 22:05:17.143649 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:17.143551 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4ded1363-3a10-48da-9736-4c181ae38c2e-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7c9bdddb8d-tlwzc\" (UID: \"4ded1363-3a10-48da-9736-4c181ae38c2e\") " pod="openshift-monitoring/metrics-server-7c9bdddb8d-tlwzc" Jul 07 22:05:17.143649 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:17.143603 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kcvs6\" (UniqueName: \"kubernetes.io/projected/4ded1363-3a10-48da-9736-4c181ae38c2e-kube-api-access-kcvs6\") pod \"metrics-server-7c9bdddb8d-tlwzc\" (UID: \"4ded1363-3a10-48da-9736-4c181ae38c2e\") " pod="openshift-monitoring/metrics-server-7c9bdddb8d-tlwzc" Jul 07 22:05:17.143649 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:17.143640 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4ded1363-3a10-48da-9736-4c181ae38c2e-client-ca-bundle\") pod \"metrics-server-7c9bdddb8d-tlwzc\" (UID: \"4ded1363-3a10-48da-9736-4c181ae38c2e\") " pod="openshift-monitoring/metrics-server-7c9bdddb8d-tlwzc" Jul 07 22:05:17.144630 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:17.144604 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4ded1363-3a10-48da-9736-4c181ae38c2e-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-7c9bdddb8d-tlwzc\" (UID: \"4ded1363-3a10-48da-9736-4c181ae38c2e\") " pod="openshift-monitoring/metrics-server-7c9bdddb8d-tlwzc" Jul 07 22:05:17.144804 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:17.144659 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/4ded1363-3a10-48da-9736-4c181ae38c2e-metrics-server-audit-profiles\") pod \"metrics-server-7c9bdddb8d-tlwzc\" (UID: \"4ded1363-3a10-48da-9736-4c181ae38c2e\") " pod="openshift-monitoring/metrics-server-7c9bdddb8d-tlwzc" Jul 07 22:05:17.144999 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:17.144907 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/4ded1363-3a10-48da-9736-4c181ae38c2e-audit-log\") pod \"metrics-server-7c9bdddb8d-tlwzc\" (UID: \"4ded1363-3a10-48da-9736-4c181ae38c2e\") " pod="openshift-monitoring/metrics-server-7c9bdddb8d-tlwzc" Jul 07 22:05:17.146389 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:17.146363 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/4ded1363-3a10-48da-9736-4c181ae38c2e-secret-metrics-server-client-certs\") pod \"metrics-server-7c9bdddb8d-tlwzc\" (UID: \"4ded1363-3a10-48da-9736-4c181ae38c2e\") " pod="openshift-monitoring/metrics-server-7c9bdddb8d-tlwzc" Jul 07 22:05:17.146548 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:17.146508 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4ded1363-3a10-48da-9736-4c181ae38c2e-client-ca-bundle\") pod \"metrics-server-7c9bdddb8d-tlwzc\" (UID: \"4ded1363-3a10-48da-9736-4c181ae38c2e\") " pod="openshift-monitoring/metrics-server-7c9bdddb8d-tlwzc" Jul 07 22:05:17.146653 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:17.146574 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/4ded1363-3a10-48da-9736-4c181ae38c2e-secret-metrics-server-tls\") pod \"metrics-server-7c9bdddb8d-tlwzc\" (UID: \"4ded1363-3a10-48da-9736-4c181ae38c2e\") " pod="openshift-monitoring/metrics-server-7c9bdddb8d-tlwzc" Jul 07 22:05:17.151476 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:17.151453 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kcvs6\" (UniqueName: \"kubernetes.io/projected/4ded1363-3a10-48da-9736-4c181ae38c2e-kube-api-access-kcvs6\") pod \"metrics-server-7c9bdddb8d-tlwzc\" (UID: \"4ded1363-3a10-48da-9736-4c181ae38c2e\") " pod="openshift-monitoring/metrics-server-7c9bdddb8d-tlwzc" Jul 07 22:05:17.219639 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:17.219602 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-7c9bdddb8d-tlwzc" Jul 07 22:05:17.373040 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:17.371795 2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-cd89d45bb-tkgqk"] Jul 07 22:05:17.397351 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:17.396590 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-cd89d45bb-tkgqk"] Jul 07 22:05:17.397351 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:17.396725 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-cd89d45bb-tkgqk" Jul 07 22:05:17.426026 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:17.425996 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-7c9bdddb8d-tlwzc"] Jul 07 22:05:17.430368 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:05:17.430342 2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ded1363_3a10_48da_9736_4c181ae38c2e.slice/crio-ce5fcf5dae6edc9d72b9518c2c1276b2d6628af9368023af637166f8fe7eae4d WatchSource:0}: Error finding container ce5fcf5dae6edc9d72b9518c2c1276b2d6628af9368023af637166f8fe7eae4d: Status 404 returned error can't find the container with id ce5fcf5dae6edc9d72b9518c2c1276b2d6628af9368023af637166f8fe7eae4d Jul 07 22:05:17.547277 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:17.547242 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/00d83737-fc9e-4153-9c39-88c8848df8a0-console-serving-cert\") pod \"console-cd89d45bb-tkgqk\" (UID: \"00d83737-fc9e-4153-9c39-88c8848df8a0\") " pod="openshift-console/console-cd89d45bb-tkgqk" Jul 07 22:05:17.547463 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:17.547331 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/00d83737-fc9e-4153-9c39-88c8848df8a0-console-config\") pod \"console-cd89d45bb-tkgqk\" (UID: \"00d83737-fc9e-4153-9c39-88c8848df8a0\") " pod="openshift-console/console-cd89d45bb-tkgqk" Jul 07 22:05:17.547463 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:17.547382 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/00d83737-fc9e-4153-9c39-88c8848df8a0-trusted-ca-bundle\") pod \"console-cd89d45bb-tkgqk\" (UID: \"00d83737-fc9e-4153-9c39-88c8848df8a0\") " pod="openshift-console/console-cd89d45bb-tkgqk" Jul 07 22:05:17.547463 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:17.547426 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lch97\" (UniqueName: \"kubernetes.io/projected/00d83737-fc9e-4153-9c39-88c8848df8a0-kube-api-access-lch97\") pod \"console-cd89d45bb-tkgqk\" (UID: \"00d83737-fc9e-4153-9c39-88c8848df8a0\") " pod="openshift-console/console-cd89d45bb-tkgqk" Jul 07 22:05:17.547638 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:17.547513 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/00d83737-fc9e-4153-9c39-88c8848df8a0-console-oauth-config\") pod \"console-cd89d45bb-tkgqk\" (UID: \"00d83737-fc9e-4153-9c39-88c8848df8a0\") " pod="openshift-console/console-cd89d45bb-tkgqk" Jul 07 22:05:17.547638 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:17.547574 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/00d83737-fc9e-4153-9c39-88c8848df8a0-service-ca\") pod \"console-cd89d45bb-tkgqk\" (UID: \"00d83737-fc9e-4153-9c39-88c8848df8a0\") " pod="openshift-console/console-cd89d45bb-tkgqk" Jul 07 22:05:17.547638 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:17.547628 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/00d83737-fc9e-4153-9c39-88c8848df8a0-oauth-serving-cert\") pod \"console-cd89d45bb-tkgqk\" (UID: \"00d83737-fc9e-4153-9c39-88c8848df8a0\") " pod="openshift-console/console-cd89d45bb-tkgqk" Jul 07 22:05:17.649123 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:17.649012 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/00d83737-fc9e-4153-9c39-88c8848df8a0-trusted-ca-bundle\") pod \"console-cd89d45bb-tkgqk\" (UID: \"00d83737-fc9e-4153-9c39-88c8848df8a0\") " pod="openshift-console/console-cd89d45bb-tkgqk" Jul 07 22:05:17.649123 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:17.649072 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lch97\" (UniqueName: \"kubernetes.io/projected/00d83737-fc9e-4153-9c39-88c8848df8a0-kube-api-access-lch97\") pod \"console-cd89d45bb-tkgqk\" (UID: \"00d83737-fc9e-4153-9c39-88c8848df8a0\") " pod="openshift-console/console-cd89d45bb-tkgqk" Jul 07 22:05:17.649410 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:17.649131 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/00d83737-fc9e-4153-9c39-88c8848df8a0-console-oauth-config\") pod \"console-cd89d45bb-tkgqk\" (UID: \"00d83737-fc9e-4153-9c39-88c8848df8a0\") " pod="openshift-console/console-cd89d45bb-tkgqk" Jul 07 22:05:17.649410 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:17.649162 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/00d83737-fc9e-4153-9c39-88c8848df8a0-service-ca\") pod \"console-cd89d45bb-tkgqk\" (UID: \"00d83737-fc9e-4153-9c39-88c8848df8a0\") " pod="openshift-console/console-cd89d45bb-tkgqk" Jul 07 22:05:17.649410 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:17.649199 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/00d83737-fc9e-4153-9c39-88c8848df8a0-oauth-serving-cert\") pod \"console-cd89d45bb-tkgqk\" (UID: \"00d83737-fc9e-4153-9c39-88c8848df8a0\") " pod="openshift-console/console-cd89d45bb-tkgqk" Jul 07 22:05:17.649410 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:17.649232 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/00d83737-fc9e-4153-9c39-88c8848df8a0-console-serving-cert\") pod \"console-cd89d45bb-tkgqk\" (UID: \"00d83737-fc9e-4153-9c39-88c8848df8a0\") " pod="openshift-console/console-cd89d45bb-tkgqk" Jul 07 22:05:17.649410 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:17.649279 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/00d83737-fc9e-4153-9c39-88c8848df8a0-console-config\") pod \"console-cd89d45bb-tkgqk\" (UID: \"00d83737-fc9e-4153-9c39-88c8848df8a0\") " pod="openshift-console/console-cd89d45bb-tkgqk" Jul 07 22:05:17.650067 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:17.650004 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/00d83737-fc9e-4153-9c39-88c8848df8a0-service-ca\") pod \"console-cd89d45bb-tkgqk\" (UID: \"00d83737-fc9e-4153-9c39-88c8848df8a0\") " pod="openshift-console/console-cd89d45bb-tkgqk" Jul 07 22:05:17.650225 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:17.650111 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/00d83737-fc9e-4153-9c39-88c8848df8a0-console-config\") pod \"console-cd89d45bb-tkgqk\" (UID: \"00d83737-fc9e-4153-9c39-88c8848df8a0\") " pod="openshift-console/console-cd89d45bb-tkgqk" Jul 07 22:05:17.650225 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:17.650198 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/00d83737-fc9e-4153-9c39-88c8848df8a0-oauth-serving-cert\") pod \"console-cd89d45bb-tkgqk\" (UID: \"00d83737-fc9e-4153-9c39-88c8848df8a0\") " pod="openshift-console/console-cd89d45bb-tkgqk" Jul 07 22:05:17.651117 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:17.651096 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/00d83737-fc9e-4153-9c39-88c8848df8a0-trusted-ca-bundle\") pod \"console-cd89d45bb-tkgqk\" (UID: \"00d83737-fc9e-4153-9c39-88c8848df8a0\") " pod="openshift-console/console-cd89d45bb-tkgqk" Jul 07 22:05:17.652017 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:17.651996 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/00d83737-fc9e-4153-9c39-88c8848df8a0-console-oauth-config\") pod \"console-cd89d45bb-tkgqk\" (UID: \"00d83737-fc9e-4153-9c39-88c8848df8a0\") " pod="openshift-console/console-cd89d45bb-tkgqk" Jul 07 22:05:17.652213 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:17.652192 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/00d83737-fc9e-4153-9c39-88c8848df8a0-console-serving-cert\") pod \"console-cd89d45bb-tkgqk\" (UID: \"00d83737-fc9e-4153-9c39-88c8848df8a0\") " pod="openshift-console/console-cd89d45bb-tkgqk" Jul 07 22:05:17.664263 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:17.664236 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lch97\" (UniqueName: \"kubernetes.io/projected/00d83737-fc9e-4153-9c39-88c8848df8a0-kube-api-access-lch97\") pod \"console-cd89d45bb-tkgqk\" (UID: \"00d83737-fc9e-4153-9c39-88c8848df8a0\") " pod="openshift-console/console-cd89d45bb-tkgqk" Jul 07 22:05:17.711100 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:17.711040 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-cd89d45bb-tkgqk" Jul 07 22:05:17.770004 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:17.769825 2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-78cf548768-7mq45"] Jul 07 22:05:17.810208 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:17.810176 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-78cf548768-7mq45"] Jul 07 22:05:17.812011 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:17.810705 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-78cf548768-7mq45" Jul 07 22:05:17.813474 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:17.813012 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9hl2n" event={"ID":"4ae9b323-7862-4003-a018-71b5e4d4705a","Type":"ContainerStarted","Data":"551cce03a606c210bf8583785b8ebfc8db9cfe438191600f9abac11c89c1d368"} Jul 07 22:05:17.813474 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:17.813066 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9hl2n" event={"ID":"4ae9b323-7862-4003-a018-71b5e4d4705a","Type":"ContainerStarted","Data":"06fee5b522fee74e86517f917764915911a25f59863c7afe7a632833dc869780"} Jul 07 22:05:17.813935 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:17.813728 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 07 22:05:17.813935 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:17.813793 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-xzkhs\"" Jul 07 22:05:17.814198 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:17.814148 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 07 22:05:17.816451 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:17.814618 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 07 22:05:17.816451 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:17.814651 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 07 22:05:17.816451 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:17.814782 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 07 22:05:17.816451 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:17.815802 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-7c9bdddb8d-tlwzc" event={"ID":"4ded1363-3a10-48da-9736-4c181ae38c2e","Type":"ContainerStarted","Data":"ce5fcf5dae6edc9d72b9518c2c1276b2d6628af9368023af637166f8fe7eae4d"} Jul 07 22:05:17.819570 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:17.819459 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 07 22:05:17.891028 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:17.890963 2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-9hl2n" podStartSLOduration=2.864256363 podStartE2EDuration="5.890945704s" totalImagesPullingTime="2.876542764s" totalInitContainerRuntime="150.146577ms" isStatefulPod=true podCreationTimestamp="2026-07-07 22:05:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:05:17.850317016 +0000 UTC m=+100.164540035" watchObservedRunningTime="2026-07-07 22:05:17.890945704 +0000 UTC m=+100.205168723" Jul 07 22:05:17.892410 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:17.892385 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-cd89d45bb-tkgqk"] Jul 07 22:05:17.895786 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:05:17.895757 2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00d83737_fc9e_4153_9c39_88c8848df8a0.slice/crio-ebb581f0393d58f6e446dee63ef650ae61f8065cc86baf5bca30209ecff7f052 WatchSource:0}: Error finding container ebb581f0393d58f6e446dee63ef650ae61f8065cc86baf5bca30209ecff7f052: Status 404 returned error can't find the container with id ebb581f0393d58f6e446dee63ef650ae61f8065cc86baf5bca30209ecff7f052 Jul 07 22:05:17.954641 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:17.952535 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/efbefc08-120c-46e7-a5f4-f5e95cadbb2c-secret-telemeter-client\") pod \"telemeter-client-78cf548768-7mq45\" (UID: \"efbefc08-120c-46e7-a5f4-f5e95cadbb2c\") " pod="openshift-monitoring/telemeter-client-78cf548768-7mq45" Jul 07 22:05:17.954641 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:17.952597 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/efbefc08-120c-46e7-a5f4-f5e95cadbb2c-serving-certs-ca-bundle\") pod \"telemeter-client-78cf548768-7mq45\" (UID: \"efbefc08-120c-46e7-a5f4-f5e95cadbb2c\") " pod="openshift-monitoring/telemeter-client-78cf548768-7mq45" Jul 07 22:05:17.954641 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:17.952659 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/efbefc08-120c-46e7-a5f4-f5e95cadbb2c-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-78cf548768-7mq45\" (UID: \"efbefc08-120c-46e7-a5f4-f5e95cadbb2c\") " pod="openshift-monitoring/telemeter-client-78cf548768-7mq45" Jul 07 22:05:17.954641 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:17.952687 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7npcq\" (UniqueName: \"kubernetes.io/projected/efbefc08-120c-46e7-a5f4-f5e95cadbb2c-kube-api-access-7npcq\") pod \"telemeter-client-78cf548768-7mq45\" (UID: \"efbefc08-120c-46e7-a5f4-f5e95cadbb2c\") " pod="openshift-monitoring/telemeter-client-78cf548768-7mq45" Jul 07 22:05:17.954641 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:17.952732 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/efbefc08-120c-46e7-a5f4-f5e95cadbb2c-telemeter-trusted-ca-bundle\") pod \"telemeter-client-78cf548768-7mq45\" (UID: \"efbefc08-120c-46e7-a5f4-f5e95cadbb2c\") " pod="openshift-monitoring/telemeter-client-78cf548768-7mq45" Jul 07 22:05:17.954641 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:17.952806 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/efbefc08-120c-46e7-a5f4-f5e95cadbb2c-federate-client-tls\") pod \"telemeter-client-78cf548768-7mq45\" (UID: \"efbefc08-120c-46e7-a5f4-f5e95cadbb2c\") " pod="openshift-monitoring/telemeter-client-78cf548768-7mq45" Jul 07 22:05:17.954641 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:17.952924 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/efbefc08-120c-46e7-a5f4-f5e95cadbb2c-telemeter-client-tls\") pod \"telemeter-client-78cf548768-7mq45\" (UID: \"efbefc08-120c-46e7-a5f4-f5e95cadbb2c\") " pod="openshift-monitoring/telemeter-client-78cf548768-7mq45" Jul 07 22:05:17.954641 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:17.952959 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/efbefc08-120c-46e7-a5f4-f5e95cadbb2c-metrics-client-ca\") pod \"telemeter-client-78cf548768-7mq45\" (UID: \"efbefc08-120c-46e7-a5f4-f5e95cadbb2c\") " pod="openshift-monitoring/telemeter-client-78cf548768-7mq45" Jul 07 22:05:18.054029 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:18.053972 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/efbefc08-120c-46e7-a5f4-f5e95cadbb2c-telemeter-client-tls\") pod \"telemeter-client-78cf548768-7mq45\" (UID: \"efbefc08-120c-46e7-a5f4-f5e95cadbb2c\") " pod="openshift-monitoring/telemeter-client-78cf548768-7mq45" Jul 07 22:05:18.054029 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:18.054024 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/efbefc08-120c-46e7-a5f4-f5e95cadbb2c-metrics-client-ca\") pod \"telemeter-client-78cf548768-7mq45\" (UID: \"efbefc08-120c-46e7-a5f4-f5e95cadbb2c\") " pod="openshift-monitoring/telemeter-client-78cf548768-7mq45" Jul 07 22:05:18.054573 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:18.054127 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/efbefc08-120c-46e7-a5f4-f5e95cadbb2c-secret-telemeter-client\") pod \"telemeter-client-78cf548768-7mq45\" (UID: \"efbefc08-120c-46e7-a5f4-f5e95cadbb2c\") " pod="openshift-monitoring/telemeter-client-78cf548768-7mq45" Jul 07 22:05:18.054573 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:18.054159 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/efbefc08-120c-46e7-a5f4-f5e95cadbb2c-serving-certs-ca-bundle\") pod \"telemeter-client-78cf548768-7mq45\" (UID: \"efbefc08-120c-46e7-a5f4-f5e95cadbb2c\") " pod="openshift-monitoring/telemeter-client-78cf548768-7mq45" Jul 07 22:05:18.054573 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:18.054223 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/efbefc08-120c-46e7-a5f4-f5e95cadbb2c-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-78cf548768-7mq45\" (UID: \"efbefc08-120c-46e7-a5f4-f5e95cadbb2c\") " pod="openshift-monitoring/telemeter-client-78cf548768-7mq45" Jul 07 22:05:18.054573 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:18.054250 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7npcq\" (UniqueName: \"kubernetes.io/projected/efbefc08-120c-46e7-a5f4-f5e95cadbb2c-kube-api-access-7npcq\") pod \"telemeter-client-78cf548768-7mq45\" (UID: \"efbefc08-120c-46e7-a5f4-f5e95cadbb2c\") " pod="openshift-monitoring/telemeter-client-78cf548768-7mq45" Jul 07 22:05:18.054573 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:18.054277 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/efbefc08-120c-46e7-a5f4-f5e95cadbb2c-telemeter-trusted-ca-bundle\") pod \"telemeter-client-78cf548768-7mq45\" (UID: \"efbefc08-120c-46e7-a5f4-f5e95cadbb2c\") " pod="openshift-monitoring/telemeter-client-78cf548768-7mq45" Jul 07 22:05:18.054573 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:18.054319 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/efbefc08-120c-46e7-a5f4-f5e95cadbb2c-federate-client-tls\") pod \"telemeter-client-78cf548768-7mq45\" (UID: \"efbefc08-120c-46e7-a5f4-f5e95cadbb2c\") " pod="openshift-monitoring/telemeter-client-78cf548768-7mq45" Jul 07 22:05:18.058492 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:18.057395 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/efbefc08-120c-46e7-a5f4-f5e95cadbb2c-federate-client-tls\") pod \"telemeter-client-78cf548768-7mq45\" (UID: \"efbefc08-120c-46e7-a5f4-f5e95cadbb2c\") " pod="openshift-monitoring/telemeter-client-78cf548768-7mq45" Jul 07 22:05:18.058492 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:18.057635 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/efbefc08-120c-46e7-a5f4-f5e95cadbb2c-metrics-client-ca\") pod \"telemeter-client-78cf548768-7mq45\" (UID: \"efbefc08-120c-46e7-a5f4-f5e95cadbb2c\") " pod="openshift-monitoring/telemeter-client-78cf548768-7mq45" Jul 07 22:05:18.058492 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:18.058442 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/efbefc08-120c-46e7-a5f4-f5e95cadbb2c-serving-certs-ca-bundle\") pod \"telemeter-client-78cf548768-7mq45\" (UID: \"efbefc08-120c-46e7-a5f4-f5e95cadbb2c\") " pod="openshift-monitoring/telemeter-client-78cf548768-7mq45" Jul 07 22:05:18.058792 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:18.058743 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/efbefc08-120c-46e7-a5f4-f5e95cadbb2c-telemeter-trusted-ca-bundle\") pod \"telemeter-client-78cf548768-7mq45\" (UID: \"efbefc08-120c-46e7-a5f4-f5e95cadbb2c\") " pod="openshift-monitoring/telemeter-client-78cf548768-7mq45" Jul 07 22:05:18.061005 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:18.060939 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/efbefc08-120c-46e7-a5f4-f5e95cadbb2c-telemeter-client-tls\") pod \"telemeter-client-78cf548768-7mq45\" (UID: \"efbefc08-120c-46e7-a5f4-f5e95cadbb2c\") " pod="openshift-monitoring/telemeter-client-78cf548768-7mq45" Jul 07 22:05:18.061427 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:18.061168 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/efbefc08-120c-46e7-a5f4-f5e95cadbb2c-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-78cf548768-7mq45\" (UID: \"efbefc08-120c-46e7-a5f4-f5e95cadbb2c\") " pod="openshift-monitoring/telemeter-client-78cf548768-7mq45" Jul 07 22:05:18.061427 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:18.061388 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/efbefc08-120c-46e7-a5f4-f5e95cadbb2c-secret-telemeter-client\") pod \"telemeter-client-78cf548768-7mq45\" (UID: \"efbefc08-120c-46e7-a5f4-f5e95cadbb2c\") " pod="openshift-monitoring/telemeter-client-78cf548768-7mq45" Jul 07 22:05:18.065576 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:18.065217 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7npcq\" (UniqueName: \"kubernetes.io/projected/efbefc08-120c-46e7-a5f4-f5e95cadbb2c-kube-api-access-7npcq\") pod \"telemeter-client-78cf548768-7mq45\" (UID: \"efbefc08-120c-46e7-a5f4-f5e95cadbb2c\") " pod="openshift-monitoring/telemeter-client-78cf548768-7mq45" Jul 07 22:05:18.123404 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:18.123371 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-78cf548768-7mq45" Jul 07 22:05:18.376163 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:18.376063 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-78cf548768-7mq45"] Jul 07 22:05:18.379769 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:05:18.379739 2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podefbefc08_120c_46e7_a5f4_f5e95cadbb2c.slice/crio-869fd7b67bb05643ade0a1fa4e05621488b5b76e37e39d332d317fe88a23a0c6 WatchSource:0}: Error finding container 869fd7b67bb05643ade0a1fa4e05621488b5b76e37e39d332d317fe88a23a0c6: Status 404 returned error can't find the container with id 869fd7b67bb05643ade0a1fa4e05621488b5b76e37e39d332d317fe88a23a0c6 Jul 07 22:05:18.744452 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:18.744298 2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 07 22:05:18.766697 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:18.764438 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:05:18.771213 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:18.770172 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 07 22:05:18.775764 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:18.775737 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 07 22:05:18.776228 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:18.776208 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-5bvn9pkr8at1j\"" Jul 07 22:05:18.776607 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:18.776589 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 07 22:05:18.780659 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:18.780475 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 07 22:05:18.786420 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:18.786401 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 07 22:05:18.786570 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:18.786547 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 07 22:05:18.786797 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:18.786782 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 07 22:05:18.787069 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:18.787051 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 07 22:05:18.791323 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:18.787062 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 07 22:05:18.791548 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:18.787179 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-68nc9\"" Jul 07 22:05:18.791643 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:18.787243 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 07 22:05:18.791742 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:18.787411 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 07 22:05:18.791822 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:18.787728 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 07 22:05:18.791915 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:18.787975 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 07 22:05:18.792017 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:18.789149 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 07 22:05:18.792110 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:18.790258 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 07 22:05:18.825405 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:18.825362 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-78cf548768-7mq45" event={"ID":"efbefc08-120c-46e7-a5f4-f5e95cadbb2c","Type":"ContainerStarted","Data":"869fd7b67bb05643ade0a1fa4e05621488b5b76e37e39d332d317fe88a23a0c6"} Jul 07 22:05:18.829615 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:18.829580 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-cd89d45bb-tkgqk" event={"ID":"00d83737-fc9e-4153-9c39-88c8848df8a0","Type":"ContainerStarted","Data":"b83b9f1e5d79e97c26c874afeed6da905c3185ef46bb8c5fcde876f175b09e65"} Jul 07 22:05:18.829744 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:18.829625 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-cd89d45bb-tkgqk" event={"ID":"00d83737-fc9e-4153-9c39-88c8848df8a0","Type":"ContainerStarted","Data":"ebb581f0393d58f6e446dee63ef650ae61f8065cc86baf5bca30209ecff7f052"} Jul 07 22:05:18.847190 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:18.847129 2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-cd89d45bb-tkgqk" podStartSLOduration=1.847109264 podStartE2EDuration="1.847109264s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 22:05:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:05:18.846532242 +0000 UTC m=+101.160755264" watchObservedRunningTime="2026-07-07 22:05:18.847109264 +0000 UTC m=+101.161332285" Jul 07 22:05:18.862550 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:18.862519 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/339d6ea1-7785-4db8-bf7c-abb7bab436d0-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"339d6ea1-7785-4db8-bf7c-abb7bab436d0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:05:18.862704 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:18.862564 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/339d6ea1-7785-4db8-bf7c-abb7bab436d0-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"339d6ea1-7785-4db8-bf7c-abb7bab436d0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:05:18.862704 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:18.862586 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/339d6ea1-7785-4db8-bf7c-abb7bab436d0-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"339d6ea1-7785-4db8-bf7c-abb7bab436d0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:05:18.862704 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:18.862610 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/339d6ea1-7785-4db8-bf7c-abb7bab436d0-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"339d6ea1-7785-4db8-bf7c-abb7bab436d0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:05:18.862704 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:18.862631 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/339d6ea1-7785-4db8-bf7c-abb7bab436d0-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"339d6ea1-7785-4db8-bf7c-abb7bab436d0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:05:18.862704 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:18.862685 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6vwfd\" (UniqueName: \"kubernetes.io/projected/339d6ea1-7785-4db8-bf7c-abb7bab436d0-kube-api-access-6vwfd\") pod \"prometheus-k8s-0\" (UID: \"339d6ea1-7785-4db8-bf7c-abb7bab436d0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:05:18.862952 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:18.862712 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/339d6ea1-7785-4db8-bf7c-abb7bab436d0-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"339d6ea1-7785-4db8-bf7c-abb7bab436d0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:05:18.862952 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:18.862741 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/339d6ea1-7785-4db8-bf7c-abb7bab436d0-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"339d6ea1-7785-4db8-bf7c-abb7bab436d0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:05:18.862952 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:18.862759 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/339d6ea1-7785-4db8-bf7c-abb7bab436d0-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"339d6ea1-7785-4db8-bf7c-abb7bab436d0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:05:18.862952 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:18.862818 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/339d6ea1-7785-4db8-bf7c-abb7bab436d0-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"339d6ea1-7785-4db8-bf7c-abb7bab436d0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:05:18.862952 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:18.862890 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/339d6ea1-7785-4db8-bf7c-abb7bab436d0-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"339d6ea1-7785-4db8-bf7c-abb7bab436d0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:05:18.862952 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:18.862918 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/339d6ea1-7785-4db8-bf7c-abb7bab436d0-web-config\") pod \"prometheus-k8s-0\" (UID: \"339d6ea1-7785-4db8-bf7c-abb7bab436d0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:05:18.862952 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:18.862951 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/339d6ea1-7785-4db8-bf7c-abb7bab436d0-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"339d6ea1-7785-4db8-bf7c-abb7bab436d0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:05:18.863223 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:18.862976 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/339d6ea1-7785-4db8-bf7c-abb7bab436d0-config\") pod \"prometheus-k8s-0\" (UID: \"339d6ea1-7785-4db8-bf7c-abb7bab436d0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:05:18.863223 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:18.863001 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/339d6ea1-7785-4db8-bf7c-abb7bab436d0-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"339d6ea1-7785-4db8-bf7c-abb7bab436d0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:05:18.863223 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:18.863025 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/339d6ea1-7785-4db8-bf7c-abb7bab436d0-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"339d6ea1-7785-4db8-bf7c-abb7bab436d0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:05:18.863223 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:18.863051 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/339d6ea1-7785-4db8-bf7c-abb7bab436d0-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"339d6ea1-7785-4db8-bf7c-abb7bab436d0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:05:18.863223 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:18.863094 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/339d6ea1-7785-4db8-bf7c-abb7bab436d0-config-out\") pod \"prometheus-k8s-0\" (UID: \"339d6ea1-7785-4db8-bf7c-abb7bab436d0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:05:18.863223 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:18.863122 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/339d6ea1-7785-4db8-bf7c-abb7bab436d0-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"339d6ea1-7785-4db8-bf7c-abb7bab436d0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:05:18.863223 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:18.863147 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/339d6ea1-7785-4db8-bf7c-abb7bab436d0-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"339d6ea1-7785-4db8-bf7c-abb7bab436d0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:05:18.964438 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:18.964399 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/339d6ea1-7785-4db8-bf7c-abb7bab436d0-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"339d6ea1-7785-4db8-bf7c-abb7bab436d0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:05:18.964689 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:18.964527 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/339d6ea1-7785-4db8-bf7c-abb7bab436d0-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"339d6ea1-7785-4db8-bf7c-abb7bab436d0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:05:18.964689 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:18.964557 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/339d6ea1-7785-4db8-bf7c-abb7bab436d0-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"339d6ea1-7785-4db8-bf7c-abb7bab436d0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:05:18.964689 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:18.964611 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/339d6ea1-7785-4db8-bf7c-abb7bab436d0-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"339d6ea1-7785-4db8-bf7c-abb7bab436d0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:05:18.964689 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:18.964635 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/339d6ea1-7785-4db8-bf7c-abb7bab436d0-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"339d6ea1-7785-4db8-bf7c-abb7bab436d0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:05:18.964689 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:18.964679 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6vwfd\" (UniqueName: \"kubernetes.io/projected/339d6ea1-7785-4db8-bf7c-abb7bab436d0-kube-api-access-6vwfd\") pod \"prometheus-k8s-0\" (UID: \"339d6ea1-7785-4db8-bf7c-abb7bab436d0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:05:18.964955 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:18.964703 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/339d6ea1-7785-4db8-bf7c-abb7bab436d0-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"339d6ea1-7785-4db8-bf7c-abb7bab436d0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:05:18.964955 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:18.964738 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/339d6ea1-7785-4db8-bf7c-abb7bab436d0-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"339d6ea1-7785-4db8-bf7c-abb7bab436d0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:05:18.964955 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:18.964763 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/339d6ea1-7785-4db8-bf7c-abb7bab436d0-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"339d6ea1-7785-4db8-bf7c-abb7bab436d0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:05:18.964955 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:18.964815 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/339d6ea1-7785-4db8-bf7c-abb7bab436d0-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"339d6ea1-7785-4db8-bf7c-abb7bab436d0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:05:18.964955 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:18.964846 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/339d6ea1-7785-4db8-bf7c-abb7bab436d0-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"339d6ea1-7785-4db8-bf7c-abb7bab436d0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:05:18.964955 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:18.964873 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/339d6ea1-7785-4db8-bf7c-abb7bab436d0-web-config\") pod \"prometheus-k8s-0\" (UID: \"339d6ea1-7785-4db8-bf7c-abb7bab436d0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:05:18.964955 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:18.964918 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/339d6ea1-7785-4db8-bf7c-abb7bab436d0-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"339d6ea1-7785-4db8-bf7c-abb7bab436d0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:05:18.964955 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:18.964944 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/339d6ea1-7785-4db8-bf7c-abb7bab436d0-config\") pod \"prometheus-k8s-0\" (UID: \"339d6ea1-7785-4db8-bf7c-abb7bab436d0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:05:18.965389 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:18.964974 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/339d6ea1-7785-4db8-bf7c-abb7bab436d0-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"339d6ea1-7785-4db8-bf7c-abb7bab436d0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:05:18.965389 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:18.965000 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/339d6ea1-7785-4db8-bf7c-abb7bab436d0-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"339d6ea1-7785-4db8-bf7c-abb7bab436d0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:05:18.965389 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:18.965028 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/339d6ea1-7785-4db8-bf7c-abb7bab436d0-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"339d6ea1-7785-4db8-bf7c-abb7bab436d0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:05:18.965389 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:18.965100 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/339d6ea1-7785-4db8-bf7c-abb7bab436d0-config-out\") pod \"prometheus-k8s-0\" (UID: \"339d6ea1-7785-4db8-bf7c-abb7bab436d0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:05:18.965389 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:18.965131 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/339d6ea1-7785-4db8-bf7c-abb7bab436d0-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"339d6ea1-7785-4db8-bf7c-abb7bab436d0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:05:18.965389 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:18.965156 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/339d6ea1-7785-4db8-bf7c-abb7bab436d0-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"339d6ea1-7785-4db8-bf7c-abb7bab436d0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:05:18.969863 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:18.966919 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/339d6ea1-7785-4db8-bf7c-abb7bab436d0-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"339d6ea1-7785-4db8-bf7c-abb7bab436d0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:05:18.969863 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:18.967254 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/339d6ea1-7785-4db8-bf7c-abb7bab436d0-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"339d6ea1-7785-4db8-bf7c-abb7bab436d0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:05:18.969863 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:18.968762 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/339d6ea1-7785-4db8-bf7c-abb7bab436d0-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"339d6ea1-7785-4db8-bf7c-abb7bab436d0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:05:18.975700 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:18.970063 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/339d6ea1-7785-4db8-bf7c-abb7bab436d0-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"339d6ea1-7785-4db8-bf7c-abb7bab436d0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:05:18.975700 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:18.970869 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/339d6ea1-7785-4db8-bf7c-abb7bab436d0-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"339d6ea1-7785-4db8-bf7c-abb7bab436d0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:05:18.975700 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:18.971882 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/339d6ea1-7785-4db8-bf7c-abb7bab436d0-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"339d6ea1-7785-4db8-bf7c-abb7bab436d0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:05:18.975700 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:18.972138 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/339d6ea1-7785-4db8-bf7c-abb7bab436d0-web-config\") pod \"prometheus-k8s-0\" (UID: \"339d6ea1-7785-4db8-bf7c-abb7bab436d0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:05:18.975700 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:18.972167 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/339d6ea1-7785-4db8-bf7c-abb7bab436d0-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"339d6ea1-7785-4db8-bf7c-abb7bab436d0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:05:18.975700 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:18.972381 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/339d6ea1-7785-4db8-bf7c-abb7bab436d0-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"339d6ea1-7785-4db8-bf7c-abb7bab436d0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:05:18.975700 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:18.972598 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/339d6ea1-7785-4db8-bf7c-abb7bab436d0-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"339d6ea1-7785-4db8-bf7c-abb7bab436d0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:05:18.975700 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:18.972754 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/339d6ea1-7785-4db8-bf7c-abb7bab436d0-config-out\") pod \"prometheus-k8s-0\" (UID: \"339d6ea1-7785-4db8-bf7c-abb7bab436d0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:05:18.975700 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:18.973315 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/339d6ea1-7785-4db8-bf7c-abb7bab436d0-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"339d6ea1-7785-4db8-bf7c-abb7bab436d0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:05:18.975700 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:18.973595 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/339d6ea1-7785-4db8-bf7c-abb7bab436d0-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"339d6ea1-7785-4db8-bf7c-abb7bab436d0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:05:18.975700 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:18.974447 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/339d6ea1-7785-4db8-bf7c-abb7bab436d0-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"339d6ea1-7785-4db8-bf7c-abb7bab436d0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:05:18.976315 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:18.975760 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/339d6ea1-7785-4db8-bf7c-abb7bab436d0-config\") pod \"prometheus-k8s-0\" (UID: \"339d6ea1-7785-4db8-bf7c-abb7bab436d0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:05:18.977108 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:18.977058 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/339d6ea1-7785-4db8-bf7c-abb7bab436d0-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"339d6ea1-7785-4db8-bf7c-abb7bab436d0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:05:18.977233 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:18.977139 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/339d6ea1-7785-4db8-bf7c-abb7bab436d0-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"339d6ea1-7785-4db8-bf7c-abb7bab436d0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:05:18.977306 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:18.977233 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/339d6ea1-7785-4db8-bf7c-abb7bab436d0-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"339d6ea1-7785-4db8-bf7c-abb7bab436d0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:05:18.977306 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:18.977240 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/339d6ea1-7785-4db8-bf7c-abb7bab436d0-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"339d6ea1-7785-4db8-bf7c-abb7bab436d0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:05:18.979301 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:18.978767 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6vwfd\" (UniqueName: \"kubernetes.io/projected/339d6ea1-7785-4db8-bf7c-abb7bab436d0-kube-api-access-6vwfd\") pod \"prometheus-k8s-0\" (UID: \"339d6ea1-7785-4db8-bf7c-abb7bab436d0\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:05:19.106312 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:19.106228 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:05:19.363447 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:19.363424 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 07 22:05:19.366437 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:05:19.366408 2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod339d6ea1_7785_4db8_bf7c_abb7bab436d0.slice/crio-2b412524f978524da1adc2c9bf6a108dd8f631f547bfdae3b455f19b80867758 WatchSource:0}: Error finding container 2b412524f978524da1adc2c9bf6a108dd8f631f547bfdae3b455f19b80867758: Status 404 returned error can't find the container with id 2b412524f978524da1adc2c9bf6a108dd8f631f547bfdae3b455f19b80867758 Jul 07 22:05:19.836616 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:19.836578 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-7c9bdddb8d-tlwzc" event={"ID":"4ded1363-3a10-48da-9736-4c181ae38c2e","Type":"ContainerStarted","Data":"3541455d2fda5f62cdaa6595fae7968ce9871f3f7d07adb50ef5532151fa16d4"} Jul 07 22:05:19.838177 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:19.838145 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"339d6ea1-7785-4db8-bf7c-abb7bab436d0","Type":"ContainerStarted","Data":"2b412524f978524da1adc2c9bf6a108dd8f631f547bfdae3b455f19b80867758"} Jul 07 22:05:19.854209 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:19.854160 2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-7c9bdddb8d-tlwzc" podStartSLOduration=1.754165975 podStartE2EDuration="3.854145726s" totalImagesPullingTime="2.099979751s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 22:05:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:05:19.85313773 +0000 UTC m=+102.167360750" watchObservedRunningTime="2026-07-07 22:05:19.854145726 +0000 UTC m=+102.168368745" Jul 07 22:05:21.660226 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:21.660201 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-82xqf" Jul 07 22:05:21.848695 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:21.848671 2545 generic.go:356] "Generic (PLEG): container finished" podID="339d6ea1-7785-4db8-bf7c-abb7bab436d0" containerID="50f90db7427dcc606161d816baafb551b61b4ba7bd0269f8602f0cb5f487e98a" exitCode=0 Jul 07 22:05:21.848823 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:21.848766 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"339d6ea1-7785-4db8-bf7c-abb7bab436d0","Type":"ContainerDied","Data":"50f90db7427dcc606161d816baafb551b61b4ba7bd0269f8602f0cb5f487e98a"} Jul 07 22:05:21.850860 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:21.850841 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-78cf548768-7mq45" event={"ID":"efbefc08-120c-46e7-a5f4-f5e95cadbb2c","Type":"ContainerStarted","Data":"7779d2783682f3b57442cf73328c7abc61c4ea79ee43a7ee0dcf08508ce21480"} Jul 07 22:05:21.850929 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:21.850883 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-78cf548768-7mq45" event={"ID":"efbefc08-120c-46e7-a5f4-f5e95cadbb2c","Type":"ContainerStarted","Data":"076e041c28ec5f21ba3c1025657de6d7980429ac54511e1a0b460f85fb8fa0f9"} Jul 07 22:05:22.857456 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:22.857416 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-78cf548768-7mq45" event={"ID":"efbefc08-120c-46e7-a5f4-f5e95cadbb2c","Type":"ContainerStarted","Data":"5573637673e43651b2042255167dfa19aa86aa14b32dfec19bcae42b9de21342"} Jul 07 22:05:22.880600 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:22.880552 2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-78cf548768-7mq45" podStartSLOduration=2.720050108 podStartE2EDuration="5.880536368s" totalImagesPullingTime="3.16048626s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 22:05:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:05:22.878532511 +0000 UTC m=+105.192755530" watchObservedRunningTime="2026-07-07 22:05:22.880536368 +0000 UTC m=+105.194759387" Jul 07 22:05:23.551543 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:23.551511 2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-cd89d45bb-tkgqk"] Jul 07 22:05:26.884486 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:26.884450 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"339d6ea1-7785-4db8-bf7c-abb7bab436d0","Type":"ContainerStarted","Data":"7678a150e9417b60c7a1b869ca512c4ac29376fbed1a6a87b480b99ef718f25a"} Jul 07 22:05:26.884926 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:26.884522 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"339d6ea1-7785-4db8-bf7c-abb7bab436d0","Type":"ContainerStarted","Data":"ebf01e0fe3f06fbf6fd0892b4301e945d6e6178762aad6d617c199af85fb175f"} Jul 07 22:05:27.711612 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:27.711569 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-cd89d45bb-tkgqk" Jul 07 22:05:29.898908 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:29.898869 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"339d6ea1-7785-4db8-bf7c-abb7bab436d0","Type":"ContainerStarted","Data":"7b5c73d70a552d14ce9ae431a09082d40041ce08819a017a56402b12b61341c1"} Jul 07 22:05:29.899426 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:29.898947 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"339d6ea1-7785-4db8-bf7c-abb7bab436d0","Type":"ContainerStarted","Data":"06d9867ab81be5bb8cb0bf881ce8cb5c90a48d6b2ea14205bd5b2baa28316c9d"} Jul 07 22:05:29.899426 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:29.898990 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"339d6ea1-7785-4db8-bf7c-abb7bab436d0","Type":"ContainerStarted","Data":"9592cb6a48ac6ce6982c5755e4dba7922655b0ede33649191e3a176cf5e2281f"} Jul 07 22:05:29.899426 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:29.899038 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"339d6ea1-7785-4db8-bf7c-abb7bab436d0","Type":"ContainerStarted","Data":"3686690db1cb9ec53c0c26bbe035b0c4ce6a061b408902fe64a237991ee20e00"} Jul 07 22:05:29.931372 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:29.931290 2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.146561571 podStartE2EDuration="11.931272868s" totalImagesPullingTime="8.760024276s" totalInitContainerRuntime="24.687021ms" isStatefulPod=false podCreationTimestamp="2026-07-07 22:05:18 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:05:29.928665029 +0000 UTC m=+112.242888048" watchObservedRunningTime="2026-07-07 22:05:29.931272868 +0000 UTC m=+112.245495894" Jul 07 22:05:32.911599 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:32.911563 2545 generic.go:356] "Generic (PLEG): container finished" podID="01d309f5-5c78-4908-a057-7dcb433b7f3b" containerID="9259d71890afb80eb4a436ca6a2ead118eb9030887fa3ae72ac6d0c92903136a" exitCode=0 Jul 07 22:05:32.912059 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:32.911621 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wt5nz" event={"ID":"01d309f5-5c78-4908-a057-7dcb433b7f3b","Type":"ContainerDied","Data":"9259d71890afb80eb4a436ca6a2ead118eb9030887fa3ae72ac6d0c92903136a"} Jul 07 22:05:32.912059 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:32.911961 2545 scope.go:117] "RemoveContainer" containerID="9259d71890afb80eb4a436ca6a2ead118eb9030887fa3ae72ac6d0c92903136a" Jul 07 22:05:33.916904 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:33.916870 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-wt5nz" event={"ID":"01d309f5-5c78-4908-a057-7dcb433b7f3b","Type":"ContainerStarted","Data":"2b5c71e1d99adadab05261c21f70c5f29a213e359c115e960e3c9b77d70617fd"} Jul 07 22:05:34.094978 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:34.094944 2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-76cf759c89-6j67d"] Jul 07 22:05:34.107580 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:34.107554 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:05:34.155479 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:34.155364 2545 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-b9885dbc7-lx8h7" podUID="78996163-d007-4f3c-ac3a-44902fe9be66" containerName="registry" containerID="cri-o://d9da8be85010c7a62ffc882ed477e1d973c2f712e10a40e82b5af6e00a69e75e" gracePeriod=30 Jul 07 22:05:34.445516 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:34.445448 2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-b9885dbc7-lx8h7" Jul 07 22:05:34.525175 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:34.525141 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/78996163-d007-4f3c-ac3a-44902fe9be66-installation-pull-secrets\") pod \"78996163-d007-4f3c-ac3a-44902fe9be66\" (UID: \"78996163-d007-4f3c-ac3a-44902fe9be66\") " Jul 07 22:05:34.525372 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:34.525187 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/78996163-d007-4f3c-ac3a-44902fe9be66-registry-tls\") pod \"78996163-d007-4f3c-ac3a-44902fe9be66\" (UID: \"78996163-d007-4f3c-ac3a-44902fe9be66\") " Jul 07 22:05:34.525372 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:34.525213 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9xtqm\" (UniqueName: \"kubernetes.io/projected/78996163-d007-4f3c-ac3a-44902fe9be66-kube-api-access-9xtqm\") pod \"78996163-d007-4f3c-ac3a-44902fe9be66\" (UID: \"78996163-d007-4f3c-ac3a-44902fe9be66\") " Jul 07 22:05:34.525372 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:34.525245 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/78996163-d007-4f3c-ac3a-44902fe9be66-bound-sa-token\") pod \"78996163-d007-4f3c-ac3a-44902fe9be66\" (UID: \"78996163-d007-4f3c-ac3a-44902fe9be66\") " Jul 07 22:05:34.525372 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:34.525281 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/78996163-d007-4f3c-ac3a-44902fe9be66-ca-trust-extracted\") pod \"78996163-d007-4f3c-ac3a-44902fe9be66\" (UID: \"78996163-d007-4f3c-ac3a-44902fe9be66\") " Jul 07 22:05:34.525372 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:34.525361 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/78996163-d007-4f3c-ac3a-44902fe9be66-trusted-ca\") pod \"78996163-d007-4f3c-ac3a-44902fe9be66\" (UID: \"78996163-d007-4f3c-ac3a-44902fe9be66\") " Jul 07 22:05:34.525630 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:34.525405 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/78996163-d007-4f3c-ac3a-44902fe9be66-registry-certificates\") pod \"78996163-d007-4f3c-ac3a-44902fe9be66\" (UID: \"78996163-d007-4f3c-ac3a-44902fe9be66\") " Jul 07 22:05:34.525630 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:34.525441 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/78996163-d007-4f3c-ac3a-44902fe9be66-image-registry-private-configuration\") pod \"78996163-d007-4f3c-ac3a-44902fe9be66\" (UID: \"78996163-d007-4f3c-ac3a-44902fe9be66\") " Jul 07 22:05:34.526040 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:34.526007 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/78996163-d007-4f3c-ac3a-44902fe9be66-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "78996163-d007-4f3c-ac3a-44902fe9be66" (UID: "78996163-d007-4f3c-ac3a-44902fe9be66"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 22:05:34.526200 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:34.526052 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/78996163-d007-4f3c-ac3a-44902fe9be66-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "78996163-d007-4f3c-ac3a-44902fe9be66" (UID: "78996163-d007-4f3c-ac3a-44902fe9be66"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 22:05:34.527855 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:34.527821 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/78996163-d007-4f3c-ac3a-44902fe9be66-kube-api-access-9xtqm" (OuterVolumeSpecName: "kube-api-access-9xtqm") pod "78996163-d007-4f3c-ac3a-44902fe9be66" (UID: "78996163-d007-4f3c-ac3a-44902fe9be66"). InnerVolumeSpecName "kube-api-access-9xtqm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 22:05:34.528047 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:34.528011 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/78996163-d007-4f3c-ac3a-44902fe9be66-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "78996163-d007-4f3c-ac3a-44902fe9be66" (UID: "78996163-d007-4f3c-ac3a-44902fe9be66"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 22:05:34.528305 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:34.528281 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/78996163-d007-4f3c-ac3a-44902fe9be66-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "78996163-d007-4f3c-ac3a-44902fe9be66" (UID: "78996163-d007-4f3c-ac3a-44902fe9be66"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 22:05:34.528402 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:34.528376 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/78996163-d007-4f3c-ac3a-44902fe9be66-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "78996163-d007-4f3c-ac3a-44902fe9be66" (UID: "78996163-d007-4f3c-ac3a-44902fe9be66"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 22:05:34.528454 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:34.528403 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/78996163-d007-4f3c-ac3a-44902fe9be66-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "78996163-d007-4f3c-ac3a-44902fe9be66" (UID: "78996163-d007-4f3c-ac3a-44902fe9be66"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 22:05:34.533787 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:34.533761 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78996163-d007-4f3c-ac3a-44902fe9be66-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "78996163-d007-4f3c-ac3a-44902fe9be66" (UID: "78996163-d007-4f3c-ac3a-44902fe9be66"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 22:05:34.627001 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:34.626967 2545 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9xtqm\" (UniqueName: \"kubernetes.io/projected/78996163-d007-4f3c-ac3a-44902fe9be66-kube-api-access-9xtqm\") on node \"ip-10-0-131-55.ec2.internal\" DevicePath \"\"" Jul 07 22:05:34.627001 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:34.626996 2545 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/78996163-d007-4f3c-ac3a-44902fe9be66-bound-sa-token\") on node \"ip-10-0-131-55.ec2.internal\" DevicePath \"\"" Jul 07 22:05:34.627001 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:34.627005 2545 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/78996163-d007-4f3c-ac3a-44902fe9be66-ca-trust-extracted\") on node \"ip-10-0-131-55.ec2.internal\" DevicePath \"\"" Jul 07 22:05:34.627253 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:34.627014 2545 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/78996163-d007-4f3c-ac3a-44902fe9be66-trusted-ca\") on node \"ip-10-0-131-55.ec2.internal\" DevicePath \"\"" Jul 07 22:05:34.627253 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:34.627023 2545 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/78996163-d007-4f3c-ac3a-44902fe9be66-registry-certificates\") on node \"ip-10-0-131-55.ec2.internal\" DevicePath \"\"" Jul 07 22:05:34.627253 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:34.627033 2545 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/78996163-d007-4f3c-ac3a-44902fe9be66-image-registry-private-configuration\") on node \"ip-10-0-131-55.ec2.internal\" DevicePath \"\"" Jul 07 22:05:34.627253 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:34.627042 2545 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/78996163-d007-4f3c-ac3a-44902fe9be66-installation-pull-secrets\") on node \"ip-10-0-131-55.ec2.internal\" DevicePath \"\"" Jul 07 22:05:34.627253 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:34.627051 2545 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/78996163-d007-4f3c-ac3a-44902fe9be66-registry-tls\") on node \"ip-10-0-131-55.ec2.internal\" DevicePath \"\"" Jul 07 22:05:34.922427 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:34.922391 2545 generic.go:356] "Generic (PLEG): container finished" podID="78996163-d007-4f3c-ac3a-44902fe9be66" containerID="d9da8be85010c7a62ffc882ed477e1d973c2f712e10a40e82b5af6e00a69e75e" exitCode=0 Jul 07 22:05:34.922841 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:34.922436 2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-b9885dbc7-lx8h7" Jul 07 22:05:34.922841 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:34.922475 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-b9885dbc7-lx8h7" event={"ID":"78996163-d007-4f3c-ac3a-44902fe9be66","Type":"ContainerDied","Data":"d9da8be85010c7a62ffc882ed477e1d973c2f712e10a40e82b5af6e00a69e75e"} Jul 07 22:05:34.922841 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:34.922526 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-b9885dbc7-lx8h7" event={"ID":"78996163-d007-4f3c-ac3a-44902fe9be66","Type":"ContainerDied","Data":"defdd452ad110e1e343aec48f3ac3a959460f3f9dd62bc9875cd4bd076d8262f"} Jul 07 22:05:34.922841 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:34.922546 2545 scope.go:117] "RemoveContainer" containerID="d9da8be85010c7a62ffc882ed477e1d973c2f712e10a40e82b5af6e00a69e75e" Jul 07 22:05:34.931827 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:34.931810 2545 scope.go:117] "RemoveContainer" containerID="d9da8be85010c7a62ffc882ed477e1d973c2f712e10a40e82b5af6e00a69e75e" Jul 07 22:05:34.932102 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:05:34.932057 2545 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d9da8be85010c7a62ffc882ed477e1d973c2f712e10a40e82b5af6e00a69e75e\": container with ID starting with d9da8be85010c7a62ffc882ed477e1d973c2f712e10a40e82b5af6e00a69e75e not found: ID does not exist" containerID="d9da8be85010c7a62ffc882ed477e1d973c2f712e10a40e82b5af6e00a69e75e" Jul 07 22:05:34.932165 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:34.932123 2545 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d9da8be85010c7a62ffc882ed477e1d973c2f712e10a40e82b5af6e00a69e75e"} err="failed to get container status \"d9da8be85010c7a62ffc882ed477e1d973c2f712e10a40e82b5af6e00a69e75e\": rpc error: code = NotFound desc = could not find container \"d9da8be85010c7a62ffc882ed477e1d973c2f712e10a40e82b5af6e00a69e75e\": container with ID starting with d9da8be85010c7a62ffc882ed477e1d973c2f712e10a40e82b5af6e00a69e75e not found: ID does not exist" Jul 07 22:05:34.942842 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:34.942820 2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-b9885dbc7-lx8h7"] Jul 07 22:05:34.946947 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:34.946927 2545 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-b9885dbc7-lx8h7"] Jul 07 22:05:36.310897 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:36.310855 2545 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="78996163-d007-4f3c-ac3a-44902fe9be66" path="/var/lib/kubelet/pods/78996163-d007-4f3c-ac3a-44902fe9be66/volumes" Jul 07 22:05:37.220619 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:37.220581 2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-7c9bdddb8d-tlwzc" Jul 07 22:05:37.220792 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:37.220667 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-7c9bdddb8d-tlwzc" Jul 07 22:05:48.583248 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:48.583187 2545 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-cd89d45bb-tkgqk" podUID="00d83737-fc9e-4153-9c39-88c8848df8a0" containerName="console" containerID="cri-o://b83b9f1e5d79e97c26c874afeed6da905c3185ef46bb8c5fcde876f175b09e65" gracePeriod=15 Jul 07 22:05:48.864035 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:48.863974 2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-cd89d45bb-tkgqk" Jul 07 22:05:48.953895 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:48.953864 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lch97\" (UniqueName: \"kubernetes.io/projected/00d83737-fc9e-4153-9c39-88c8848df8a0-kube-api-access-lch97\") pod \"00d83737-fc9e-4153-9c39-88c8848df8a0\" (UID: \"00d83737-fc9e-4153-9c39-88c8848df8a0\") " Jul 07 22:05:48.954061 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:48.953908 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/00d83737-fc9e-4153-9c39-88c8848df8a0-service-ca\") pod \"00d83737-fc9e-4153-9c39-88c8848df8a0\" (UID: \"00d83737-fc9e-4153-9c39-88c8848df8a0\") " Jul 07 22:05:48.954061 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:48.953946 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/00d83737-fc9e-4153-9c39-88c8848df8a0-console-oauth-config\") pod \"00d83737-fc9e-4153-9c39-88c8848df8a0\" (UID: \"00d83737-fc9e-4153-9c39-88c8848df8a0\") " Jul 07 22:05:48.954061 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:48.954009 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/00d83737-fc9e-4153-9c39-88c8848df8a0-console-config\") pod \"00d83737-fc9e-4153-9c39-88c8848df8a0\" (UID: \"00d83737-fc9e-4153-9c39-88c8848df8a0\") " Jul 07 22:05:48.954061 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:48.954053 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/00d83737-fc9e-4153-9c39-88c8848df8a0-console-serving-cert\") pod \"00d83737-fc9e-4153-9c39-88c8848df8a0\" (UID: \"00d83737-fc9e-4153-9c39-88c8848df8a0\") " Jul 07 22:05:48.954284 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:48.954101 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/00d83737-fc9e-4153-9c39-88c8848df8a0-trusted-ca-bundle\") pod \"00d83737-fc9e-4153-9c39-88c8848df8a0\" (UID: \"00d83737-fc9e-4153-9c39-88c8848df8a0\") " Jul 07 22:05:48.954284 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:48.954173 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/00d83737-fc9e-4153-9c39-88c8848df8a0-oauth-serving-cert\") pod \"00d83737-fc9e-4153-9c39-88c8848df8a0\" (UID: \"00d83737-fc9e-4153-9c39-88c8848df8a0\") " Jul 07 22:05:48.954425 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:48.954401 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/00d83737-fc9e-4153-9c39-88c8848df8a0-service-ca" (OuterVolumeSpecName: "service-ca") pod "00d83737-fc9e-4153-9c39-88c8848df8a0" (UID: "00d83737-fc9e-4153-9c39-88c8848df8a0"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 22:05:48.954506 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:48.954487 2545 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/00d83737-fc9e-4153-9c39-88c8848df8a0-service-ca\") on node \"ip-10-0-131-55.ec2.internal\" DevicePath \"\"" Jul 07 22:05:48.954675 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:48.954650 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/00d83737-fc9e-4153-9c39-88c8848df8a0-console-config" (OuterVolumeSpecName: "console-config") pod "00d83737-fc9e-4153-9c39-88c8848df8a0" (UID: "00d83737-fc9e-4153-9c39-88c8848df8a0"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 22:05:48.954829 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:48.954804 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/00d83737-fc9e-4153-9c39-88c8848df8a0-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "00d83737-fc9e-4153-9c39-88c8848df8a0" (UID: "00d83737-fc9e-4153-9c39-88c8848df8a0"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 22:05:48.954829 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:48.954802 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/00d83737-fc9e-4153-9c39-88c8848df8a0-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "00d83737-fc9e-4153-9c39-88c8848df8a0" (UID: "00d83737-fc9e-4153-9c39-88c8848df8a0"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 22:05:48.956334 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:48.956303 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/00d83737-fc9e-4153-9c39-88c8848df8a0-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "00d83737-fc9e-4153-9c39-88c8848df8a0" (UID: "00d83737-fc9e-4153-9c39-88c8848df8a0"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 22:05:48.956421 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:48.956362 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/00d83737-fc9e-4153-9c39-88c8848df8a0-kube-api-access-lch97" (OuterVolumeSpecName: "kube-api-access-lch97") pod "00d83737-fc9e-4153-9c39-88c8848df8a0" (UID: "00d83737-fc9e-4153-9c39-88c8848df8a0"). InnerVolumeSpecName "kube-api-access-lch97". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 22:05:48.956421 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:48.956408 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/00d83737-fc9e-4153-9c39-88c8848df8a0-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "00d83737-fc9e-4153-9c39-88c8848df8a0" (UID: "00d83737-fc9e-4153-9c39-88c8848df8a0"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 22:05:48.980658 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:48.980622 2545 generic.go:356] "Generic (PLEG): container finished" podID="00d83737-fc9e-4153-9c39-88c8848df8a0" containerID="b83b9f1e5d79e97c26c874afeed6da905c3185ef46bb8c5fcde876f175b09e65" exitCode=0 Jul 07 22:05:48.980789 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:48.980668 2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-cd89d45bb-tkgqk" Jul 07 22:05:48.980789 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:48.980702 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-cd89d45bb-tkgqk" event={"ID":"00d83737-fc9e-4153-9c39-88c8848df8a0","Type":"ContainerDied","Data":"b83b9f1e5d79e97c26c874afeed6da905c3185ef46bb8c5fcde876f175b09e65"} Jul 07 22:05:48.980789 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:48.980748 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-cd89d45bb-tkgqk" event={"ID":"00d83737-fc9e-4153-9c39-88c8848df8a0","Type":"ContainerDied","Data":"ebb581f0393d58f6e446dee63ef650ae61f8065cc86baf5bca30209ecff7f052"} Jul 07 22:05:48.980789 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:48.980767 2545 scope.go:117] "RemoveContainer" containerID="b83b9f1e5d79e97c26c874afeed6da905c3185ef46bb8c5fcde876f175b09e65" Jul 07 22:05:48.989113 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:48.989090 2545 scope.go:117] "RemoveContainer" containerID="b83b9f1e5d79e97c26c874afeed6da905c3185ef46bb8c5fcde876f175b09e65" Jul 07 22:05:48.989364 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:05:48.989347 2545 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b83b9f1e5d79e97c26c874afeed6da905c3185ef46bb8c5fcde876f175b09e65\": container with ID starting with b83b9f1e5d79e97c26c874afeed6da905c3185ef46bb8c5fcde876f175b09e65 not found: ID does not exist" containerID="b83b9f1e5d79e97c26c874afeed6da905c3185ef46bb8c5fcde876f175b09e65" Jul 07 22:05:48.989416 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:48.989371 2545 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b83b9f1e5d79e97c26c874afeed6da905c3185ef46bb8c5fcde876f175b09e65"} err="failed to get container status \"b83b9f1e5d79e97c26c874afeed6da905c3185ef46bb8c5fcde876f175b09e65\": rpc error: code = NotFound desc = could not find container \"b83b9f1e5d79e97c26c874afeed6da905c3185ef46bb8c5fcde876f175b09e65\": container with ID starting with b83b9f1e5d79e97c26c874afeed6da905c3185ef46bb8c5fcde876f175b09e65 not found: ID does not exist" Jul 07 22:05:49.001849 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:49.001826 2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-cd89d45bb-tkgqk"] Jul 07 22:05:49.005312 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:49.005289 2545 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-cd89d45bb-tkgqk"] Jul 07 22:05:49.055459 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:49.055430 2545 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lch97\" (UniqueName: \"kubernetes.io/projected/00d83737-fc9e-4153-9c39-88c8848df8a0-kube-api-access-lch97\") on node \"ip-10-0-131-55.ec2.internal\" DevicePath \"\"" Jul 07 22:05:49.055459 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:49.055455 2545 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/00d83737-fc9e-4153-9c39-88c8848df8a0-console-oauth-config\") on node \"ip-10-0-131-55.ec2.internal\" DevicePath \"\"" Jul 07 22:05:49.055600 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:49.055465 2545 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/00d83737-fc9e-4153-9c39-88c8848df8a0-console-config\") on node \"ip-10-0-131-55.ec2.internal\" DevicePath \"\"" Jul 07 22:05:49.055600 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:49.055474 2545 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/00d83737-fc9e-4153-9c39-88c8848df8a0-console-serving-cert\") on node \"ip-10-0-131-55.ec2.internal\" DevicePath \"\"" Jul 07 22:05:49.055600 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:49.055482 2545 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/00d83737-fc9e-4153-9c39-88c8848df8a0-trusted-ca-bundle\") on node \"ip-10-0-131-55.ec2.internal\" DevicePath \"\"" Jul 07 22:05:49.055600 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:49.055491 2545 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/00d83737-fc9e-4153-9c39-88c8848df8a0-oauth-serving-cert\") on node \"ip-10-0-131-55.ec2.internal\" DevicePath \"\"" Jul 07 22:05:50.314895 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:50.314860 2545 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="00d83737-fc9e-4153-9c39-88c8848df8a0" path="/var/lib/kubelet/pods/00d83737-fc9e-4153-9c39-88c8848df8a0/volumes" Jul 07 22:05:57.225801 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:57.225769 2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-7c9bdddb8d-tlwzc" Jul 07 22:05:57.229671 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:57.229649 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-7c9bdddb8d-tlwzc" Jul 07 22:05:59.117190 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:59.117144 2545 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-76cf759c89-6j67d" podUID="8af4f328-8dbc-41eb-8e14-a82d34bbb3eb" containerName="console" containerID="cri-o://d4e368eb43dfce5d732f4892864cfc625280e229e9269121579d17b1622562db" gracePeriod=15 Jul 07 22:05:59.402404 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:59.402378 2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76cf759c89-6j67d" Jul 07 22:05:59.547888 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:59.547854 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8af4f328-8dbc-41eb-8e14-a82d34bbb3eb-trusted-ca-bundle\") pod \"8af4f328-8dbc-41eb-8e14-a82d34bbb3eb\" (UID: \"8af4f328-8dbc-41eb-8e14-a82d34bbb3eb\") " Jul 07 22:05:59.548090 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:59.547976 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8af4f328-8dbc-41eb-8e14-a82d34bbb3eb-console-serving-cert\") pod \"8af4f328-8dbc-41eb-8e14-a82d34bbb3eb\" (UID: \"8af4f328-8dbc-41eb-8e14-a82d34bbb3eb\") " Jul 07 22:05:59.548090 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:59.548011 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8af4f328-8dbc-41eb-8e14-a82d34bbb3eb-console-config\") pod \"8af4f328-8dbc-41eb-8e14-a82d34bbb3eb\" (UID: \"8af4f328-8dbc-41eb-8e14-a82d34bbb3eb\") " Jul 07 22:05:59.548090 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:59.548044 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nf2rm\" (UniqueName: \"kubernetes.io/projected/8af4f328-8dbc-41eb-8e14-a82d34bbb3eb-kube-api-access-nf2rm\") pod \"8af4f328-8dbc-41eb-8e14-a82d34bbb3eb\" (UID: \"8af4f328-8dbc-41eb-8e14-a82d34bbb3eb\") " Jul 07 22:05:59.548090 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:59.548073 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8af4f328-8dbc-41eb-8e14-a82d34bbb3eb-service-ca\") pod \"8af4f328-8dbc-41eb-8e14-a82d34bbb3eb\" (UID: \"8af4f328-8dbc-41eb-8e14-a82d34bbb3eb\") " Jul 07 22:05:59.548308 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:59.548120 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8af4f328-8dbc-41eb-8e14-a82d34bbb3eb-oauth-serving-cert\") pod \"8af4f328-8dbc-41eb-8e14-a82d34bbb3eb\" (UID: \"8af4f328-8dbc-41eb-8e14-a82d34bbb3eb\") " Jul 07 22:05:59.548308 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:59.548175 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8af4f328-8dbc-41eb-8e14-a82d34bbb3eb-console-oauth-config\") pod \"8af4f328-8dbc-41eb-8e14-a82d34bbb3eb\" (UID: \"8af4f328-8dbc-41eb-8e14-a82d34bbb3eb\") " Jul 07 22:05:59.548308 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:59.548236 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8af4f328-8dbc-41eb-8e14-a82d34bbb3eb-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "8af4f328-8dbc-41eb-8e14-a82d34bbb3eb" (UID: "8af4f328-8dbc-41eb-8e14-a82d34bbb3eb"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 22:05:59.548465 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:59.548347 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8af4f328-8dbc-41eb-8e14-a82d34bbb3eb-console-config" (OuterVolumeSpecName: "console-config") pod "8af4f328-8dbc-41eb-8e14-a82d34bbb3eb" (UID: "8af4f328-8dbc-41eb-8e14-a82d34bbb3eb"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 22:05:59.548524 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:59.548494 2545 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8af4f328-8dbc-41eb-8e14-a82d34bbb3eb-trusted-ca-bundle\") on node \"ip-10-0-131-55.ec2.internal\" DevicePath \"\"" Jul 07 22:05:59.548524 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:59.548514 2545 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8af4f328-8dbc-41eb-8e14-a82d34bbb3eb-console-config\") on node \"ip-10-0-131-55.ec2.internal\" DevicePath \"\"" Jul 07 22:05:59.548635 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:59.548581 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8af4f328-8dbc-41eb-8e14-a82d34bbb3eb-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "8af4f328-8dbc-41eb-8e14-a82d34bbb3eb" (UID: "8af4f328-8dbc-41eb-8e14-a82d34bbb3eb"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 22:05:59.548635 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:59.548595 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8af4f328-8dbc-41eb-8e14-a82d34bbb3eb-service-ca" (OuterVolumeSpecName: "service-ca") pod "8af4f328-8dbc-41eb-8e14-a82d34bbb3eb" (UID: "8af4f328-8dbc-41eb-8e14-a82d34bbb3eb"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 22:05:59.550437 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:59.550394 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8af4f328-8dbc-41eb-8e14-a82d34bbb3eb-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "8af4f328-8dbc-41eb-8e14-a82d34bbb3eb" (UID: "8af4f328-8dbc-41eb-8e14-a82d34bbb3eb"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 22:05:59.550572 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:59.550484 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8af4f328-8dbc-41eb-8e14-a82d34bbb3eb-kube-api-access-nf2rm" (OuterVolumeSpecName: "kube-api-access-nf2rm") pod "8af4f328-8dbc-41eb-8e14-a82d34bbb3eb" (UID: "8af4f328-8dbc-41eb-8e14-a82d34bbb3eb"). InnerVolumeSpecName "kube-api-access-nf2rm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 22:05:59.550644 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:59.550601 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8af4f328-8dbc-41eb-8e14-a82d34bbb3eb-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "8af4f328-8dbc-41eb-8e14-a82d34bbb3eb" (UID: "8af4f328-8dbc-41eb-8e14-a82d34bbb3eb"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 22:05:59.649631 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:59.649551 2545 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8af4f328-8dbc-41eb-8e14-a82d34bbb3eb-console-serving-cert\") on node \"ip-10-0-131-55.ec2.internal\" DevicePath \"\"" Jul 07 22:05:59.649631 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:59.649593 2545 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nf2rm\" (UniqueName: \"kubernetes.io/projected/8af4f328-8dbc-41eb-8e14-a82d34bbb3eb-kube-api-access-nf2rm\") on node \"ip-10-0-131-55.ec2.internal\" DevicePath \"\"" Jul 07 22:05:59.649631 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:59.649610 2545 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8af4f328-8dbc-41eb-8e14-a82d34bbb3eb-service-ca\") on node \"ip-10-0-131-55.ec2.internal\" DevicePath \"\"" Jul 07 22:05:59.649631 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:59.649624 2545 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8af4f328-8dbc-41eb-8e14-a82d34bbb3eb-oauth-serving-cert\") on node \"ip-10-0-131-55.ec2.internal\" DevicePath \"\"" Jul 07 22:05:59.649858 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:05:59.649637 2545 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8af4f328-8dbc-41eb-8e14-a82d34bbb3eb-console-oauth-config\") on node \"ip-10-0-131-55.ec2.internal\" DevicePath \"\"" Jul 07 22:06:00.031259 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:06:00.031221 2545 generic.go:356] "Generic (PLEG): container finished" podID="8af4f328-8dbc-41eb-8e14-a82d34bbb3eb" containerID="d4e368eb43dfce5d732f4892864cfc625280e229e9269121579d17b1622562db" exitCode=0 Jul 07 22:06:00.031452 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:06:00.031267 2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76cf759c89-6j67d" Jul 07 22:06:00.031452 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:06:00.031304 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76cf759c89-6j67d" event={"ID":"8af4f328-8dbc-41eb-8e14-a82d34bbb3eb","Type":"ContainerDied","Data":"d4e368eb43dfce5d732f4892864cfc625280e229e9269121579d17b1622562db"} Jul 07 22:06:00.031452 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:06:00.031370 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76cf759c89-6j67d" event={"ID":"8af4f328-8dbc-41eb-8e14-a82d34bbb3eb","Type":"ContainerDied","Data":"eecf326490234f372c94d9c9cb9c16aab5d421255e5605fcaa280756c0874a79"} Jul 07 22:06:00.031452 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:06:00.031399 2545 scope.go:117] "RemoveContainer" containerID="d4e368eb43dfce5d732f4892864cfc625280e229e9269121579d17b1622562db" Jul 07 22:06:00.042919 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:06:00.042637 2545 scope.go:117] "RemoveContainer" containerID="d4e368eb43dfce5d732f4892864cfc625280e229e9269121579d17b1622562db" Jul 07 22:06:00.043561 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:06:00.043498 2545 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d4e368eb43dfce5d732f4892864cfc625280e229e9269121579d17b1622562db\": container with ID starting with d4e368eb43dfce5d732f4892864cfc625280e229e9269121579d17b1622562db not found: ID does not exist" containerID="d4e368eb43dfce5d732f4892864cfc625280e229e9269121579d17b1622562db" Jul 07 22:06:00.043561 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:06:00.043541 2545 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d4e368eb43dfce5d732f4892864cfc625280e229e9269121579d17b1622562db"} err="failed to get container status \"d4e368eb43dfce5d732f4892864cfc625280e229e9269121579d17b1622562db\": rpc error: code = NotFound desc = could not find container \"d4e368eb43dfce5d732f4892864cfc625280e229e9269121579d17b1622562db\": container with ID starting with d4e368eb43dfce5d732f4892864cfc625280e229e9269121579d17b1622562db not found: ID does not exist" Jul 07 22:06:00.057155 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:06:00.057127 2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-76cf759c89-6j67d"] Jul 07 22:06:00.060989 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:06:00.060962 2545 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-76cf759c89-6j67d"] Jul 07 22:06:00.313544 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:06:00.313466 2545 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8af4f328-8dbc-41eb-8e14-a82d34bbb3eb" path="/var/lib/kubelet/pods/8af4f328-8dbc-41eb-8e14-a82d34bbb3eb/volumes" Jul 07 22:06:19.107570 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:06:19.107537 2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:06:19.122772 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:06:19.122746 2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:06:20.129790 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:06:20.129763 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 07 22:08:38.125194 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:08:38.125165 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-rhg5r_71f36bfa-4b8a-4572-b522-57c16392aa0b/console-operator/1.log" Jul 07 22:08:38.128687 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:08:38.128661 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-rhg5r_71f36bfa-4b8a-4572-b522-57c16392aa0b/console-operator/1.log" Jul 07 22:08:38.135989 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:08:38.135966 2545 kubelet.go:1648] "Image garbage collection succeeded" Jul 07 22:08:58.095522 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:08:58.095472 2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-654d77bd8f-dnshd"] Jul 07 22:08:58.097333 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:08:58.095953 2545 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="00d83737-fc9e-4153-9c39-88c8848df8a0" containerName="console" Jul 07 22:08:58.097333 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:08:58.095970 2545 state_mem.go:107] "Deleted CPUSet assignment" podUID="00d83737-fc9e-4153-9c39-88c8848df8a0" containerName="console" Jul 07 22:08:58.097333 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:08:58.095989 2545 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="78996163-d007-4f3c-ac3a-44902fe9be66" containerName="registry" Jul 07 22:08:58.097333 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:08:58.095998 2545 state_mem.go:107] "Deleted CPUSet assignment" podUID="78996163-d007-4f3c-ac3a-44902fe9be66" containerName="registry" Jul 07 22:08:58.097333 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:08:58.096011 2545 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8af4f328-8dbc-41eb-8e14-a82d34bbb3eb" containerName="console" Jul 07 22:08:58.097333 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:08:58.096016 2545 state_mem.go:107] "Deleted CPUSet assignment" podUID="8af4f328-8dbc-41eb-8e14-a82d34bbb3eb" containerName="console" Jul 07 22:08:58.097333 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:08:58.096072 2545 memory_manager.go:368] "RemoveStaleState removing state" podUID="8af4f328-8dbc-41eb-8e14-a82d34bbb3eb" containerName="console" Jul 07 22:08:58.097333 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:08:58.096099 2545 memory_manager.go:368] "RemoveStaleState removing state" podUID="78996163-d007-4f3c-ac3a-44902fe9be66" containerName="registry" Jul 07 22:08:58.097333 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:08:58.096106 2545 memory_manager.go:368] "RemoveStaleState removing state" podUID="00d83737-fc9e-4153-9c39-88c8848df8a0" containerName="console" Jul 07 22:08:58.098545 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:08:58.098517 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-654d77bd8f-dnshd" Jul 07 22:08:58.101431 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:08:58.101411 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-4nflv\"" Jul 07 22:08:58.102504 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:08:58.102391 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 07 22:08:58.102504 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:08:58.102404 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 07 22:08:58.102504 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:08:58.102436 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 07 22:08:58.102504 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:08:58.102445 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 07 22:08:58.103775 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:08:58.103753 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 07 22:08:58.114102 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:08:58.114062 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-654d77bd8f-dnshd"] Jul 07 22:08:58.239746 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:08:58.239703 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/8e1d7f55-e188-4c00-84c2-024a2d313383-manager-config\") pod \"lws-controller-manager-654d77bd8f-dnshd\" (UID: \"8e1d7f55-e188-4c00-84c2-024a2d313383\") " pod="openshift-lws-operator/lws-controller-manager-654d77bd8f-dnshd" Jul 07 22:08:58.239746 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:08:58.239746 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/8e1d7f55-e188-4c00-84c2-024a2d313383-metrics-cert\") pod \"lws-controller-manager-654d77bd8f-dnshd\" (UID: \"8e1d7f55-e188-4c00-84c2-024a2d313383\") " pod="openshift-lws-operator/lws-controller-manager-654d77bd8f-dnshd" Jul 07 22:08:58.239956 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:08:58.239791 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8e1d7f55-e188-4c00-84c2-024a2d313383-cert\") pod \"lws-controller-manager-654d77bd8f-dnshd\" (UID: \"8e1d7f55-e188-4c00-84c2-024a2d313383\") " pod="openshift-lws-operator/lws-controller-manager-654d77bd8f-dnshd" Jul 07 22:08:58.239956 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:08:58.239881 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t7xm9\" (UniqueName: \"kubernetes.io/projected/8e1d7f55-e188-4c00-84c2-024a2d313383-kube-api-access-t7xm9\") pod \"lws-controller-manager-654d77bd8f-dnshd\" (UID: \"8e1d7f55-e188-4c00-84c2-024a2d313383\") " pod="openshift-lws-operator/lws-controller-manager-654d77bd8f-dnshd" Jul 07 22:08:58.341131 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:08:58.341100 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8e1d7f55-e188-4c00-84c2-024a2d313383-cert\") pod \"lws-controller-manager-654d77bd8f-dnshd\" (UID: \"8e1d7f55-e188-4c00-84c2-024a2d313383\") " pod="openshift-lws-operator/lws-controller-manager-654d77bd8f-dnshd" Jul 07 22:08:58.341310 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:08:58.341180 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t7xm9\" (UniqueName: \"kubernetes.io/projected/8e1d7f55-e188-4c00-84c2-024a2d313383-kube-api-access-t7xm9\") pod \"lws-controller-manager-654d77bd8f-dnshd\" (UID: \"8e1d7f55-e188-4c00-84c2-024a2d313383\") " pod="openshift-lws-operator/lws-controller-manager-654d77bd8f-dnshd" Jul 07 22:08:58.341310 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:08:58.341217 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/8e1d7f55-e188-4c00-84c2-024a2d313383-manager-config\") pod \"lws-controller-manager-654d77bd8f-dnshd\" (UID: \"8e1d7f55-e188-4c00-84c2-024a2d313383\") " pod="openshift-lws-operator/lws-controller-manager-654d77bd8f-dnshd" Jul 07 22:08:58.341310 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:08:58.341237 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/8e1d7f55-e188-4c00-84c2-024a2d313383-metrics-cert\") pod \"lws-controller-manager-654d77bd8f-dnshd\" (UID: \"8e1d7f55-e188-4c00-84c2-024a2d313383\") " pod="openshift-lws-operator/lws-controller-manager-654d77bd8f-dnshd" Jul 07 22:08:58.341866 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:08:58.341842 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/8e1d7f55-e188-4c00-84c2-024a2d313383-manager-config\") pod \"lws-controller-manager-654d77bd8f-dnshd\" (UID: \"8e1d7f55-e188-4c00-84c2-024a2d313383\") " pod="openshift-lws-operator/lws-controller-manager-654d77bd8f-dnshd" Jul 07 22:08:58.343804 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:08:58.343777 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8e1d7f55-e188-4c00-84c2-024a2d313383-cert\") pod \"lws-controller-manager-654d77bd8f-dnshd\" (UID: \"8e1d7f55-e188-4c00-84c2-024a2d313383\") " pod="openshift-lws-operator/lws-controller-manager-654d77bd8f-dnshd" Jul 07 22:08:58.343897 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:08:58.343778 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/8e1d7f55-e188-4c00-84c2-024a2d313383-metrics-cert\") pod \"lws-controller-manager-654d77bd8f-dnshd\" (UID: \"8e1d7f55-e188-4c00-84c2-024a2d313383\") " pod="openshift-lws-operator/lws-controller-manager-654d77bd8f-dnshd" Jul 07 22:08:58.355119 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:08:58.355040 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t7xm9\" (UniqueName: \"kubernetes.io/projected/8e1d7f55-e188-4c00-84c2-024a2d313383-kube-api-access-t7xm9\") pod \"lws-controller-manager-654d77bd8f-dnshd\" (UID: \"8e1d7f55-e188-4c00-84c2-024a2d313383\") " pod="openshift-lws-operator/lws-controller-manager-654d77bd8f-dnshd" Jul 07 22:08:58.410892 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:08:58.410835 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-654d77bd8f-dnshd" Jul 07 22:08:58.543874 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:08:58.543851 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-654d77bd8f-dnshd"] Jul 07 22:08:58.546056 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:08:58.546029 2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e1d7f55_e188_4c00_84c2_024a2d313383.slice/crio-4cb4df0b2ac178ae3f3e419a4dadff0cfd8e6699cbfa7f0ea2d1f385e040faef WatchSource:0}: Error finding container 4cb4df0b2ac178ae3f3e419a4dadff0cfd8e6699cbfa7f0ea2d1f385e040faef: Status 404 returned error can't find the container with id 4cb4df0b2ac178ae3f3e419a4dadff0cfd8e6699cbfa7f0ea2d1f385e040faef Jul 07 22:08:58.547672 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:08:58.547655 2545 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 22:08:58.737512 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:08:58.737481 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-654d77bd8f-dnshd" event={"ID":"8e1d7f55-e188-4c00-84c2-024a2d313383","Type":"ContainerStarted","Data":"4cb4df0b2ac178ae3f3e419a4dadff0cfd8e6699cbfa7f0ea2d1f385e040faef"} Jul 07 22:09:00.264378 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:09:00.264343 2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-597db99f65-jjzbb"] Jul 07 22:09:00.271336 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:09:00.271312 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-597db99f65-jjzbb" Jul 07 22:09:00.274328 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:09:00.274306 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 07 22:09:00.274619 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:09:00.274597 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 07 22:09:00.274725 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:09:00.274657 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 07 22:09:00.274725 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:09:00.274713 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 07 22:09:00.274890 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:09:00.274869 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-wbtfm\"" Jul 07 22:09:00.286158 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:09:00.286068 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-597db99f65-jjzbb"] Jul 07 22:09:00.459645 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:09:00.459605 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/559369c5-4d6c-427a-90c7-2c49e47827ac-webhook-cert\") pod \"opendatahub-operator-controller-manager-597db99f65-jjzbb\" (UID: \"559369c5-4d6c-427a-90c7-2c49e47827ac\") " pod="opendatahub/opendatahub-operator-controller-manager-597db99f65-jjzbb" Jul 07 22:09:00.459822 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:09:00.459720 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/559369c5-4d6c-427a-90c7-2c49e47827ac-tmp\") pod \"opendatahub-operator-controller-manager-597db99f65-jjzbb\" (UID: \"559369c5-4d6c-427a-90c7-2c49e47827ac\") " pod="opendatahub/opendatahub-operator-controller-manager-597db99f65-jjzbb" Jul 07 22:09:00.459893 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:09:00.459835 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cdg7h\" (UniqueName: \"kubernetes.io/projected/559369c5-4d6c-427a-90c7-2c49e47827ac-kube-api-access-cdg7h\") pod \"opendatahub-operator-controller-manager-597db99f65-jjzbb\" (UID: \"559369c5-4d6c-427a-90c7-2c49e47827ac\") " pod="opendatahub/opendatahub-operator-controller-manager-597db99f65-jjzbb" Jul 07 22:09:00.459893 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:09:00.459870 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/559369c5-4d6c-427a-90c7-2c49e47827ac-apiservice-cert\") pod \"opendatahub-operator-controller-manager-597db99f65-jjzbb\" (UID: \"559369c5-4d6c-427a-90c7-2c49e47827ac\") " pod="opendatahub/opendatahub-operator-controller-manager-597db99f65-jjzbb" Jul 07 22:09:00.459986 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:09:00.459961 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/559369c5-4d6c-427a-90c7-2c49e47827ac-manifests\") pod \"opendatahub-operator-controller-manager-597db99f65-jjzbb\" (UID: \"559369c5-4d6c-427a-90c7-2c49e47827ac\") " pod="opendatahub/opendatahub-operator-controller-manager-597db99f65-jjzbb" Jul 07 22:09:00.560799 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:09:00.560709 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/559369c5-4d6c-427a-90c7-2c49e47827ac-manifests\") pod \"opendatahub-operator-controller-manager-597db99f65-jjzbb\" (UID: \"559369c5-4d6c-427a-90c7-2c49e47827ac\") " pod="opendatahub/opendatahub-operator-controller-manager-597db99f65-jjzbb" Jul 07 22:09:00.560799 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:09:00.560758 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/559369c5-4d6c-427a-90c7-2c49e47827ac-webhook-cert\") pod \"opendatahub-operator-controller-manager-597db99f65-jjzbb\" (UID: \"559369c5-4d6c-427a-90c7-2c49e47827ac\") " pod="opendatahub/opendatahub-operator-controller-manager-597db99f65-jjzbb" Jul 07 22:09:00.561029 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:09:00.560803 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/559369c5-4d6c-427a-90c7-2c49e47827ac-tmp\") pod \"opendatahub-operator-controller-manager-597db99f65-jjzbb\" (UID: \"559369c5-4d6c-427a-90c7-2c49e47827ac\") " pod="opendatahub/opendatahub-operator-controller-manager-597db99f65-jjzbb" Jul 07 22:09:00.561311 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:09:00.561072 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cdg7h\" (UniqueName: \"kubernetes.io/projected/559369c5-4d6c-427a-90c7-2c49e47827ac-kube-api-access-cdg7h\") pod \"opendatahub-operator-controller-manager-597db99f65-jjzbb\" (UID: \"559369c5-4d6c-427a-90c7-2c49e47827ac\") " pod="opendatahub/opendatahub-operator-controller-manager-597db99f65-jjzbb" Jul 07 22:09:00.561311 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:09:00.561154 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/559369c5-4d6c-427a-90c7-2c49e47827ac-apiservice-cert\") pod \"opendatahub-operator-controller-manager-597db99f65-jjzbb\" (UID: \"559369c5-4d6c-427a-90c7-2c49e47827ac\") " pod="opendatahub/opendatahub-operator-controller-manager-597db99f65-jjzbb" Jul 07 22:09:00.561311 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:09:00.561202 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/559369c5-4d6c-427a-90c7-2c49e47827ac-manifests\") pod \"opendatahub-operator-controller-manager-597db99f65-jjzbb\" (UID: \"559369c5-4d6c-427a-90c7-2c49e47827ac\") " pod="opendatahub/opendatahub-operator-controller-manager-597db99f65-jjzbb" Jul 07 22:09:00.563447 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:09:00.563412 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/559369c5-4d6c-427a-90c7-2c49e47827ac-tmp\") pod \"opendatahub-operator-controller-manager-597db99f65-jjzbb\" (UID: \"559369c5-4d6c-427a-90c7-2c49e47827ac\") " pod="opendatahub/opendatahub-operator-controller-manager-597db99f65-jjzbb" Jul 07 22:09:00.563761 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:09:00.563742 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/559369c5-4d6c-427a-90c7-2c49e47827ac-webhook-cert\") pod \"opendatahub-operator-controller-manager-597db99f65-jjzbb\" (UID: \"559369c5-4d6c-427a-90c7-2c49e47827ac\") " pod="opendatahub/opendatahub-operator-controller-manager-597db99f65-jjzbb" Jul 07 22:09:00.563916 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:09:00.563874 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/559369c5-4d6c-427a-90c7-2c49e47827ac-apiservice-cert\") pod \"opendatahub-operator-controller-manager-597db99f65-jjzbb\" (UID: \"559369c5-4d6c-427a-90c7-2c49e47827ac\") " pod="opendatahub/opendatahub-operator-controller-manager-597db99f65-jjzbb" Jul 07 22:09:00.569219 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:09:00.569198 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cdg7h\" (UniqueName: \"kubernetes.io/projected/559369c5-4d6c-427a-90c7-2c49e47827ac-kube-api-access-cdg7h\") pod \"opendatahub-operator-controller-manager-597db99f65-jjzbb\" (UID: \"559369c5-4d6c-427a-90c7-2c49e47827ac\") " pod="opendatahub/opendatahub-operator-controller-manager-597db99f65-jjzbb" Jul 07 22:09:00.584900 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:09:00.584813 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-597db99f65-jjzbb" Jul 07 22:09:01.082914 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:09:01.082837 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-597db99f65-jjzbb"] Jul 07 22:09:01.086932 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:09:01.086899 2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod559369c5_4d6c_427a_90c7_2c49e47827ac.slice/crio-85a1a69ad6ce474e95cad0c227e2efa88a36b01539b283356558319d24d984f9 WatchSource:0}: Error finding container 85a1a69ad6ce474e95cad0c227e2efa88a36b01539b283356558319d24d984f9: Status 404 returned error can't find the container with id 85a1a69ad6ce474e95cad0c227e2efa88a36b01539b283356558319d24d984f9 Jul 07 22:09:01.755264 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:09:01.755218 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-654d77bd8f-dnshd" event={"ID":"8e1d7f55-e188-4c00-84c2-024a2d313383","Type":"ContainerStarted","Data":"97e9689ff3d778f837294c981692e069cea6704aefbdc4706171cf97e797d5b5"} Jul 07 22:09:01.755716 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:09:01.755353 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-654d77bd8f-dnshd" Jul 07 22:09:01.757289 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:09:01.757254 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-597db99f65-jjzbb" event={"ID":"559369c5-4d6c-427a-90c7-2c49e47827ac","Type":"ContainerStarted","Data":"85a1a69ad6ce474e95cad0c227e2efa88a36b01539b283356558319d24d984f9"} Jul 07 22:09:01.772833 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:09:01.772778 2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-654d77bd8f-dnshd" podStartSLOduration=1.338113173 podStartE2EDuration="3.772760572s" totalImagesPullingTime="2.434647399s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 22:08:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:09:01.770494459 +0000 UTC m=+324.084717471" watchObservedRunningTime="2026-07-07 22:09:01.772760572 +0000 UTC m=+324.086983592" Jul 07 22:09:04.774538 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:09:04.774502 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-597db99f65-jjzbb" event={"ID":"559369c5-4d6c-427a-90c7-2c49e47827ac","Type":"ContainerStarted","Data":"817eb75dedd8dbb7b397fdc3e145f0f95397c9fd21c6335d8f91da0481aad827"} Jul 07 22:09:05.780479 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:09:05.780448 2545 generic.go:356] "Generic (PLEG): container finished" podID="559369c5-4d6c-427a-90c7-2c49e47827ac" containerID="817eb75dedd8dbb7b397fdc3e145f0f95397c9fd21c6335d8f91da0481aad827" exitCode=0 Jul 07 22:09:05.780963 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:09:05.780491 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-597db99f65-jjzbb" event={"ID":"559369c5-4d6c-427a-90c7-2c49e47827ac","Type":"ContainerDied","Data":"817eb75dedd8dbb7b397fdc3e145f0f95397c9fd21c6335d8f91da0481aad827"} Jul 07 22:09:06.787172 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:09:06.787136 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-597db99f65-jjzbb" event={"ID":"559369c5-4d6c-427a-90c7-2c49e47827ac","Type":"ContainerStarted","Data":"5e7510c50935fca97b50866ec94a7af5f49fd0c2d0e78bcf5fc18b898e946cf4"} Jul 07 22:09:06.787680 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:09:06.787261 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-597db99f65-jjzbb" Jul 07 22:09:06.815512 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:09:06.815465 2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-597db99f65-jjzbb" podStartSLOduration=2.650192195 podStartE2EDuration="6.815448896s" totalImagesPullingTime="3.102923157s" totalInitContainerRuntime="1.062333544s" isStatefulPod=false podCreationTimestamp="2026-07-07 22:09:00 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:09:06.814596324 +0000 UTC m=+329.128819344" watchObservedRunningTime="2026-07-07 22:09:06.815448896 +0000 UTC m=+329.129671914" Jul 07 22:09:12.766343 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:09:12.766312 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-654d77bd8f-dnshd" Jul 07 22:09:17.794220 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:09:17.794187 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-597db99f65-jjzbb" Jul 07 22:09:20.279346 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:09:20.279310 2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-6b4fccd749-28d8d"] Jul 07 22:09:20.409973 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:09:20.409938 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6b4fccd749-28d8d" Jul 07 22:09:20.412940 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:09:20.412859 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 07 22:09:20.413063 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:09:20.413032 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-vpc45\"" Jul 07 22:09:20.413152 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:09:20.413119 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 07 22:09:20.417374 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:09:20.417296 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6b4fccd749-28d8d"] Jul 07 22:09:20.524864 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:09:20.524836 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/980381c5-f1a1-41ea-ba0f-0a582f6c2577-tls-certs\") pod \"kube-auth-proxy-6b4fccd749-28d8d\" (UID: \"980381c5-f1a1-41ea-ba0f-0a582f6c2577\") " pod="openshift-ingress/kube-auth-proxy-6b4fccd749-28d8d" Jul 07 22:09:20.525067 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:09:20.524885 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wkwxt\" (UniqueName: \"kubernetes.io/projected/980381c5-f1a1-41ea-ba0f-0a582f6c2577-kube-api-access-wkwxt\") pod \"kube-auth-proxy-6b4fccd749-28d8d\" (UID: \"980381c5-f1a1-41ea-ba0f-0a582f6c2577\") " pod="openshift-ingress/kube-auth-proxy-6b4fccd749-28d8d" Jul 07 22:09:20.525067 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:09:20.524964 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/980381c5-f1a1-41ea-ba0f-0a582f6c2577-tmp\") pod \"kube-auth-proxy-6b4fccd749-28d8d\" (UID: \"980381c5-f1a1-41ea-ba0f-0a582f6c2577\") " pod="openshift-ingress/kube-auth-proxy-6b4fccd749-28d8d" Jul 07 22:09:20.626335 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:09:20.626253 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/980381c5-f1a1-41ea-ba0f-0a582f6c2577-tmp\") pod \"kube-auth-proxy-6b4fccd749-28d8d\" (UID: \"980381c5-f1a1-41ea-ba0f-0a582f6c2577\") " pod="openshift-ingress/kube-auth-proxy-6b4fccd749-28d8d" Jul 07 22:09:20.626335 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:09:20.626313 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/980381c5-f1a1-41ea-ba0f-0a582f6c2577-tls-certs\") pod \"kube-auth-proxy-6b4fccd749-28d8d\" (UID: \"980381c5-f1a1-41ea-ba0f-0a582f6c2577\") " pod="openshift-ingress/kube-auth-proxy-6b4fccd749-28d8d" Jul 07 22:09:20.626548 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:09:20.626368 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wkwxt\" (UniqueName: \"kubernetes.io/projected/980381c5-f1a1-41ea-ba0f-0a582f6c2577-kube-api-access-wkwxt\") pod \"kube-auth-proxy-6b4fccd749-28d8d\" (UID: \"980381c5-f1a1-41ea-ba0f-0a582f6c2577\") " pod="openshift-ingress/kube-auth-proxy-6b4fccd749-28d8d" Jul 07 22:09:20.628780 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:09:20.628757 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/980381c5-f1a1-41ea-ba0f-0a582f6c2577-tmp\") pod \"kube-auth-proxy-6b4fccd749-28d8d\" (UID: \"980381c5-f1a1-41ea-ba0f-0a582f6c2577\") " pod="openshift-ingress/kube-auth-proxy-6b4fccd749-28d8d" Jul 07 22:09:20.628974 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:09:20.628953 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/980381c5-f1a1-41ea-ba0f-0a582f6c2577-tls-certs\") pod \"kube-auth-proxy-6b4fccd749-28d8d\" (UID: \"980381c5-f1a1-41ea-ba0f-0a582f6c2577\") " pod="openshift-ingress/kube-auth-proxy-6b4fccd749-28d8d" Jul 07 22:09:20.644306 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:09:20.644280 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wkwxt\" (UniqueName: \"kubernetes.io/projected/980381c5-f1a1-41ea-ba0f-0a582f6c2577-kube-api-access-wkwxt\") pod \"kube-auth-proxy-6b4fccd749-28d8d\" (UID: \"980381c5-f1a1-41ea-ba0f-0a582f6c2577\") " pod="openshift-ingress/kube-auth-proxy-6b4fccd749-28d8d" Jul 07 22:09:20.719992 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:09:20.719959 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6b4fccd749-28d8d" Jul 07 22:09:20.874212 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:09:20.874186 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6b4fccd749-28d8d"] Jul 07 22:09:20.876485 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:09:20.876425 2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod980381c5_f1a1_41ea_ba0f_0a582f6c2577.slice/crio-540cec0c755404fd65978de1c5f9bc33375c58e6d8acd7effa987bed64901522 WatchSource:0}: Error finding container 540cec0c755404fd65978de1c5f9bc33375c58e6d8acd7effa987bed64901522: Status 404 returned error can't find the container with id 540cec0c755404fd65978de1c5f9bc33375c58e6d8acd7effa987bed64901522 Jul 07 22:09:21.858036 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:09:21.857985 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6b4fccd749-28d8d" event={"ID":"980381c5-f1a1-41ea-ba0f-0a582f6c2577","Type":"ContainerStarted","Data":"540cec0c755404fd65978de1c5f9bc33375c58e6d8acd7effa987bed64901522"} Jul 07 22:09:24.872715 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:09:24.872678 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6b4fccd749-28d8d" event={"ID":"980381c5-f1a1-41ea-ba0f-0a582f6c2577","Type":"ContainerStarted","Data":"b7a2a4fd777f0f831bfaf0f46b8bd6bca25649f9c32f8eba1a75de4e48bad35d"} Jul 07 22:09:24.888898 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:09:24.888849 2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-6b4fccd749-28d8d" podStartSLOduration=1.8344782849999999 podStartE2EDuration="4.888834334s" totalImagesPullingTime="3.054356049s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 22:09:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:09:24.887699069 +0000 UTC m=+347.201922088" watchObservedRunningTime="2026-07-07 22:09:24.888834334 +0000 UTC m=+347.203057354" Jul 07 22:11:04.571117 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:11:04.571018 2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-5z2vw"] Jul 07 22:11:04.573439 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:11:04.573419 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-5z2vw" Jul 07 22:11:04.577061 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:11:04.577044 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 07 22:11:04.577864 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:11:04.577848 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 07 22:11:04.577958 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:11:04.577850 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-5q9kl\"" Jul 07 22:11:04.577958 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:11:04.577851 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 07 22:11:04.586452 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:11:04.586429 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-5z2vw"] Jul 07 22:11:04.627337 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:11:04.627311 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zvsxs\" (UniqueName: \"kubernetes.io/projected/4687ae14-1e22-4218-9fec-004e72063d8b-kube-api-access-zvsxs\") pod \"dns-operator-controller-manager-864dd78c4c-5z2vw\" (UID: \"4687ae14-1e22-4218-9fec-004e72063d8b\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-5z2vw" Jul 07 22:11:04.728378 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:11:04.728344 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zvsxs\" (UniqueName: \"kubernetes.io/projected/4687ae14-1e22-4218-9fec-004e72063d8b-kube-api-access-zvsxs\") pod \"dns-operator-controller-manager-864dd78c4c-5z2vw\" (UID: \"4687ae14-1e22-4218-9fec-004e72063d8b\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-5z2vw" Jul 07 22:11:04.739645 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:11:04.739609 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zvsxs\" (UniqueName: \"kubernetes.io/projected/4687ae14-1e22-4218-9fec-004e72063d8b-kube-api-access-zvsxs\") pod \"dns-operator-controller-manager-864dd78c4c-5z2vw\" (UID: \"4687ae14-1e22-4218-9fec-004e72063d8b\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-5z2vw" Jul 07 22:11:04.883424 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:11:04.883347 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-5z2vw" Jul 07 22:11:05.011590 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:11:05.011340 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-5z2vw"] Jul 07 22:11:05.014028 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:11:05.014004 2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4687ae14_1e22_4218_9fec_004e72063d8b.slice/crio-15b64b4ac256f353234a82353b8656951f772def69a1efd14e6ed42ae612b6cb WatchSource:0}: Error finding container 15b64b4ac256f353234a82353b8656951f772def69a1efd14e6ed42ae612b6cb: Status 404 returned error can't find the container with id 15b64b4ac256f353234a82353b8656951f772def69a1efd14e6ed42ae612b6cb Jul 07 22:11:05.306924 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:11:05.306889 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-5z2vw" event={"ID":"4687ae14-1e22-4218-9fec-004e72063d8b","Type":"ContainerStarted","Data":"15b64b4ac256f353234a82353b8656951f772def69a1efd14e6ed42ae612b6cb"} Jul 07 22:11:07.319209 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:11:07.319125 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-5z2vw" event={"ID":"4687ae14-1e22-4218-9fec-004e72063d8b","Type":"ContainerStarted","Data":"e7dae3856addd4f04298ae8a62463524d23ae1003d7d057c26603c69a2a7afb4"} Jul 07 22:11:07.319554 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:11:07.319217 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-5z2vw" Jul 07 22:11:07.335548 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:11:07.335500 2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-5z2vw" podStartSLOduration=1.297215344 podStartE2EDuration="3.335483008s" totalImagesPullingTime="2.038267664s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 22:11:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:11:07.333959992 +0000 UTC m=+449.648183007" watchObservedRunningTime="2026-07-07 22:11:07.335483008 +0000 UTC m=+449.649706027" Jul 07 22:11:11.287867 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:11:11.287831 2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-m8nxm"] Jul 07 22:11:11.293229 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:11:11.291610 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-m8nxm" Jul 07 22:11:11.295003 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:11:11.294950 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jul 07 22:11:11.295359 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:11:11.295311 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jul 07 22:11:11.297047 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:11:11.295574 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-f97sq\"" Jul 07 22:11:11.300579 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:11:11.300551 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-m8nxm"] Jul 07 22:11:11.392254 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:11:11.392220 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x65vn\" (UniqueName: \"kubernetes.io/projected/6941be32-c4d0-4973-87ec-11caa5ee0fc9-kube-api-access-x65vn\") pod \"kuadrant-console-plugin-66b5774985-m8nxm\" (UID: \"6941be32-c4d0-4973-87ec-11caa5ee0fc9\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-m8nxm" Jul 07 22:11:11.392422 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:11:11.392269 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/6941be32-c4d0-4973-87ec-11caa5ee0fc9-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-m8nxm\" (UID: \"6941be32-c4d0-4973-87ec-11caa5ee0fc9\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-m8nxm" Jul 07 22:11:11.392422 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:11:11.392307 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/6941be32-c4d0-4973-87ec-11caa5ee0fc9-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-m8nxm\" (UID: \"6941be32-c4d0-4973-87ec-11caa5ee0fc9\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-m8nxm" Jul 07 22:11:11.493527 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:11:11.493489 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/6941be32-c4d0-4973-87ec-11caa5ee0fc9-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-m8nxm\" (UID: \"6941be32-c4d0-4973-87ec-11caa5ee0fc9\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-m8nxm" Jul 07 22:11:11.493690 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:11:11.493563 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x65vn\" (UniqueName: \"kubernetes.io/projected/6941be32-c4d0-4973-87ec-11caa5ee0fc9-kube-api-access-x65vn\") pod \"kuadrant-console-plugin-66b5774985-m8nxm\" (UID: \"6941be32-c4d0-4973-87ec-11caa5ee0fc9\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-m8nxm" Jul 07 22:11:11.493690 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:11:11.493589 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/6941be32-c4d0-4973-87ec-11caa5ee0fc9-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-m8nxm\" (UID: \"6941be32-c4d0-4973-87ec-11caa5ee0fc9\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-m8nxm" Jul 07 22:11:11.493690 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:11:11.493660 2545 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found Jul 07 22:11:11.493801 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:11:11.493760 2545 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6941be32-c4d0-4973-87ec-11caa5ee0fc9-plugin-serving-cert podName:6941be32-c4d0-4973-87ec-11caa5ee0fc9 nodeName:}" failed. No retries permitted until 2026-07-07 22:11:11.993740735 +0000 UTC m=+454.307963750 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/6941be32-c4d0-4973-87ec-11caa5ee0fc9-plugin-serving-cert") pod "kuadrant-console-plugin-66b5774985-m8nxm" (UID: "6941be32-c4d0-4973-87ec-11caa5ee0fc9") : secret "plugin-serving-cert" not found Jul 07 22:11:11.494323 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:11:11.494306 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/6941be32-c4d0-4973-87ec-11caa5ee0fc9-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-m8nxm\" (UID: \"6941be32-c4d0-4973-87ec-11caa5ee0fc9\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-m8nxm" Jul 07 22:11:11.504888 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:11:11.504870 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x65vn\" (UniqueName: \"kubernetes.io/projected/6941be32-c4d0-4973-87ec-11caa5ee0fc9-kube-api-access-x65vn\") pod \"kuadrant-console-plugin-66b5774985-m8nxm\" (UID: \"6941be32-c4d0-4973-87ec-11caa5ee0fc9\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-m8nxm" Jul 07 22:11:11.997518 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:11:11.997480 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/6941be32-c4d0-4973-87ec-11caa5ee0fc9-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-m8nxm\" (UID: \"6941be32-c4d0-4973-87ec-11caa5ee0fc9\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-m8nxm" Jul 07 22:11:11.999981 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:11:11.999955 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/6941be32-c4d0-4973-87ec-11caa5ee0fc9-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-m8nxm\" (UID: \"6941be32-c4d0-4973-87ec-11caa5ee0fc9\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-m8nxm" Jul 07 22:11:12.212144 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:11:12.212108 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-m8nxm" Jul 07 22:11:12.339529 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:11:12.339500 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-m8nxm"] Jul 07 22:11:12.341739 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:11:12.341712 2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6941be32_c4d0_4973_87ec_11caa5ee0fc9.slice/crio-4464b198b4cd89b17b697ca051b82a9b9cfd1ef4fd4834b0aaed1dc8d6fc6466 WatchSource:0}: Error finding container 4464b198b4cd89b17b697ca051b82a9b9cfd1ef4fd4834b0aaed1dc8d6fc6466: Status 404 returned error can't find the container with id 4464b198b4cd89b17b697ca051b82a9b9cfd1ef4fd4834b0aaed1dc8d6fc6466 Jul 07 22:11:13.346943 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:11:13.346907 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-m8nxm" event={"ID":"6941be32-c4d0-4973-87ec-11caa5ee0fc9","Type":"ContainerStarted","Data":"4464b198b4cd89b17b697ca051b82a9b9cfd1ef4fd4834b0aaed1dc8d6fc6466"} Jul 07 22:11:18.325447 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:11:18.325417 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-5z2vw" Jul 07 22:11:22.325680 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:11:22.325559 2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sqptd"] Jul 07 22:11:22.341595 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:11:22.341267 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sqptd"] Jul 07 22:11:22.341595 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:11:22.341375 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sqptd" Jul 07 22:11:22.344207 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:11:22.344154 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-5gdr9\"" Jul 07 22:11:22.495153 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:11:22.495116 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vkccc\" (UniqueName: \"kubernetes.io/projected/1e06e247-ff08-43e1-a884-79bb36006378-kube-api-access-vkccc\") pod \"limitador-operator-controller-manager-859d7b78bd-sqptd\" (UID: \"1e06e247-ff08-43e1-a884-79bb36006378\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sqptd" Jul 07 22:11:22.595739 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:11:22.595642 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vkccc\" (UniqueName: \"kubernetes.io/projected/1e06e247-ff08-43e1-a884-79bb36006378-kube-api-access-vkccc\") pod \"limitador-operator-controller-manager-859d7b78bd-sqptd\" (UID: \"1e06e247-ff08-43e1-a884-79bb36006378\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sqptd" Jul 07 22:11:22.606225 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:11:22.606188 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vkccc\" (UniqueName: \"kubernetes.io/projected/1e06e247-ff08-43e1-a884-79bb36006378-kube-api-access-vkccc\") pod \"limitador-operator-controller-manager-859d7b78bd-sqptd\" (UID: \"1e06e247-ff08-43e1-a884-79bb36006378\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sqptd" Jul 07 22:11:22.657057 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:11:22.657015 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sqptd" Jul 07 22:11:35.459623 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:11:35.459587 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sqptd"] Jul 07 22:11:35.462625 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:11:35.462590 2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e06e247_ff08_43e1_a884_79bb36006378.slice/crio-61b43770aa87ab0ab744d8f713aaf241b181abfb25281d228bc6f48b2d9780b3 WatchSource:0}: Error finding container 61b43770aa87ab0ab744d8f713aaf241b181abfb25281d228bc6f48b2d9780b3: Status 404 returned error can't find the container with id 61b43770aa87ab0ab744d8f713aaf241b181abfb25281d228bc6f48b2d9780b3 Jul 07 22:11:36.472306 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:11:36.472273 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sqptd" event={"ID":"1e06e247-ff08-43e1-a884-79bb36006378","Type":"ContainerStarted","Data":"61b43770aa87ab0ab744d8f713aaf241b181abfb25281d228bc6f48b2d9780b3"} Jul 07 22:11:36.473712 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:11:36.473687 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-m8nxm" event={"ID":"6941be32-c4d0-4973-87ec-11caa5ee0fc9","Type":"ContainerStarted","Data":"0430d0b153adfae5f0df689c81092bf22114b4f02e57b83f9d90266c5c6f9cdb"} Jul 07 22:11:36.491719 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:11:36.491671 2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-m8nxm" podStartSLOduration=2.1999636750000002 podStartE2EDuration="25.491655501s" totalImagesPullingTime="23.291691826s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 22:11:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:11:36.490541788 +0000 UTC m=+478.804764808" watchObservedRunningTime="2026-07-07 22:11:36.491655501 +0000 UTC m=+478.805878520" Jul 07 22:11:40.495280 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:11:40.495236 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sqptd" event={"ID":"1e06e247-ff08-43e1-a884-79bb36006378","Type":"ContainerStarted","Data":"8bc508d9042c3c31f913043ab8706fb33060fb80e249a747e339bbb657c82114"} Jul 07 22:11:40.495712 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:11:40.495363 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sqptd" Jul 07 22:11:40.512304 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:11:40.512254 2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sqptd" podStartSLOduration=14.596942924 podStartE2EDuration="18.512239554s" totalImagesPullingTime="3.91529663s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 22:11:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:11:40.510611808 +0000 UTC m=+482.824834830" watchObservedRunningTime="2026-07-07 22:11:40.512239554 +0000 UTC m=+482.826462572" Jul 07 22:11:51.502830 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:11:51.502792 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-sqptd" Jul 07 22:12:08.166484 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:12:08.166448 2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-f7dtr"] Jul 07 22:12:08.252399 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:12:08.252368 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-f7dtr"] Jul 07 22:12:08.252595 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:12:08.252494 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-f7dtr" Jul 07 22:12:08.254882 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:12:08.254856 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-bmfcm\"" Jul 07 22:12:08.407420 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:12:08.407378 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lq96w\" (UniqueName: \"kubernetes.io/projected/5946b615-2463-4674-9f9c-b20b36e3b6e6-kube-api-access-lq96w\") pod \"authorino-75d556bd77-f7dtr\" (UID: \"5946b615-2463-4674-9f9c-b20b36e3b6e6\") " pod="kuadrant-system/authorino-75d556bd77-f7dtr" Jul 07 22:12:08.507974 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:12:08.507933 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lq96w\" (UniqueName: \"kubernetes.io/projected/5946b615-2463-4674-9f9c-b20b36e3b6e6-kube-api-access-lq96w\") pod \"authorino-75d556bd77-f7dtr\" (UID: \"5946b615-2463-4674-9f9c-b20b36e3b6e6\") " pod="kuadrant-system/authorino-75d556bd77-f7dtr" Jul 07 22:12:08.515512 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:12:08.515489 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lq96w\" (UniqueName: \"kubernetes.io/projected/5946b615-2463-4674-9f9c-b20b36e3b6e6-kube-api-access-lq96w\") pod \"authorino-75d556bd77-f7dtr\" (UID: \"5946b615-2463-4674-9f9c-b20b36e3b6e6\") " pod="kuadrant-system/authorino-75d556bd77-f7dtr" Jul 07 22:12:08.568503 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:12:08.568472 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-f7dtr" Jul 07 22:12:08.697208 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:12:08.697181 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-f7dtr"] Jul 07 22:12:08.699661 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:12:08.699625 2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5946b615_2463_4674_9f9c_b20b36e3b6e6.slice/crio-54646a611b7fd7ec4917c99e09e9adf2e00a7431635213026464b333aa67eb0a WatchSource:0}: Error finding container 54646a611b7fd7ec4917c99e09e9adf2e00a7431635213026464b333aa67eb0a: Status 404 returned error can't find the container with id 54646a611b7fd7ec4917c99e09e9adf2e00a7431635213026464b333aa67eb0a Jul 07 22:12:09.635548 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:12:09.635512 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-f7dtr" event={"ID":"5946b615-2463-4674-9f9c-b20b36e3b6e6","Type":"ContainerStarted","Data":"54646a611b7fd7ec4917c99e09e9adf2e00a7431635213026464b333aa67eb0a"} Jul 07 22:12:12.652060 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:12:12.652023 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-f7dtr" event={"ID":"5946b615-2463-4674-9f9c-b20b36e3b6e6","Type":"ContainerStarted","Data":"4546e83fbb28ddb5207d4efa7ea40af0b20f514b019c3177151f6c2235533808"} Jul 07 22:12:12.667271 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:12:12.667219 2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-f7dtr" podStartSLOduration=1.495495269 podStartE2EDuration="4.667204466s" totalImagesPullingTime="3.171709197s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 22:12:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:12:12.665746349 +0000 UTC m=+514.979969369" watchObservedRunningTime="2026-07-07 22:12:12.667204466 +0000 UTC m=+514.981427485" Jul 07 22:12:37.921176 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:12:37.921142 2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5bf4d5c564-89zgc"] Jul 07 22:12:37.923603 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:12:37.923585 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5bf4d5c564-89zgc" Jul 07 22:12:37.925799 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:12:37.925767 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 07 22:12:37.931724 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:12:37.931701 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5bf4d5c564-89zgc"] Jul 07 22:12:37.962693 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:12:37.962660 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rbb5m\" (UniqueName: \"kubernetes.io/projected/6473e6a2-bfb7-48ba-8f1d-e1c302b1ce1f-kube-api-access-rbb5m\") pod \"authorino-5bf4d5c564-89zgc\" (UID: \"6473e6a2-bfb7-48ba-8f1d-e1c302b1ce1f\") " pod="kuadrant-system/authorino-5bf4d5c564-89zgc" Jul 07 22:12:37.962853 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:12:37.962835 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/6473e6a2-bfb7-48ba-8f1d-e1c302b1ce1f-tls-cert\") pod \"authorino-5bf4d5c564-89zgc\" (UID: \"6473e6a2-bfb7-48ba-8f1d-e1c302b1ce1f\") " pod="kuadrant-system/authorino-5bf4d5c564-89zgc" Jul 07 22:12:38.063946 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:12:38.063912 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rbb5m\" (UniqueName: \"kubernetes.io/projected/6473e6a2-bfb7-48ba-8f1d-e1c302b1ce1f-kube-api-access-rbb5m\") pod \"authorino-5bf4d5c564-89zgc\" (UID: \"6473e6a2-bfb7-48ba-8f1d-e1c302b1ce1f\") " pod="kuadrant-system/authorino-5bf4d5c564-89zgc" Jul 07 22:12:38.064116 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:12:38.064011 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/6473e6a2-bfb7-48ba-8f1d-e1c302b1ce1f-tls-cert\") pod \"authorino-5bf4d5c564-89zgc\" (UID: \"6473e6a2-bfb7-48ba-8f1d-e1c302b1ce1f\") " pod="kuadrant-system/authorino-5bf4d5c564-89zgc" Jul 07 22:12:38.066361 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:12:38.066337 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/6473e6a2-bfb7-48ba-8f1d-e1c302b1ce1f-tls-cert\") pod \"authorino-5bf4d5c564-89zgc\" (UID: \"6473e6a2-bfb7-48ba-8f1d-e1c302b1ce1f\") " pod="kuadrant-system/authorino-5bf4d5c564-89zgc" Jul 07 22:12:38.072568 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:12:38.072539 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rbb5m\" (UniqueName: \"kubernetes.io/projected/6473e6a2-bfb7-48ba-8f1d-e1c302b1ce1f-kube-api-access-rbb5m\") pod \"authorino-5bf4d5c564-89zgc\" (UID: \"6473e6a2-bfb7-48ba-8f1d-e1c302b1ce1f\") " pod="kuadrant-system/authorino-5bf4d5c564-89zgc" Jul 07 22:12:38.233236 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:12:38.233207 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5bf4d5c564-89zgc" Jul 07 22:12:38.366728 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:12:38.366705 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5bf4d5c564-89zgc"] Jul 07 22:12:38.369381 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:12:38.369354 2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6473e6a2_bfb7_48ba_8f1d_e1c302b1ce1f.slice/crio-8d22994e5bbfee9c7d6b6db652b1d0beda4824aabc261e3944b9c625e34332d4 WatchSource:0}: Error finding container 8d22994e5bbfee9c7d6b6db652b1d0beda4824aabc261e3944b9c625e34332d4: Status 404 returned error can't find the container with id 8d22994e5bbfee9c7d6b6db652b1d0beda4824aabc261e3944b9c625e34332d4 Jul 07 22:12:38.776894 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:12:38.776858 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5bf4d5c564-89zgc" event={"ID":"6473e6a2-bfb7-48ba-8f1d-e1c302b1ce1f","Type":"ContainerStarted","Data":"c16a9856154473424e394271ddd49236801f759680a2f1b2b87609414d726408"} Jul 07 22:12:38.777041 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:12:38.776904 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5bf4d5c564-89zgc" event={"ID":"6473e6a2-bfb7-48ba-8f1d-e1c302b1ce1f","Type":"ContainerStarted","Data":"8d22994e5bbfee9c7d6b6db652b1d0beda4824aabc261e3944b9c625e34332d4"} Jul 07 22:12:38.793268 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:12:38.793211 2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5bf4d5c564-89zgc" podStartSLOduration=1.482031442 podStartE2EDuration="1.793194521s" totalImagesPullingTime="311.163079ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 22:12:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:12:38.792446807 +0000 UTC m=+541.106669827" watchObservedRunningTime="2026-07-07 22:12:38.793194521 +0000 UTC m=+541.107417539" Jul 07 22:12:38.829063 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:12:38.829026 2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-f7dtr"] Jul 07 22:12:38.829298 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:12:38.829264 2545 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-f7dtr" podUID="5946b615-2463-4674-9f9c-b20b36e3b6e6" containerName="authorino" containerID="cri-o://4546e83fbb28ddb5207d4efa7ea40af0b20f514b019c3177151f6c2235533808" gracePeriod=30 Jul 07 22:12:39.060970 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:12:39.060945 2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-f7dtr" Jul 07 22:12:39.174987 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:12:39.174956 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lq96w\" (UniqueName: \"kubernetes.io/projected/5946b615-2463-4674-9f9c-b20b36e3b6e6-kube-api-access-lq96w\") pod \"5946b615-2463-4674-9f9c-b20b36e3b6e6\" (UID: \"5946b615-2463-4674-9f9c-b20b36e3b6e6\") " Jul 07 22:12:39.177070 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:12:39.177047 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5946b615-2463-4674-9f9c-b20b36e3b6e6-kube-api-access-lq96w" (OuterVolumeSpecName: "kube-api-access-lq96w") pod "5946b615-2463-4674-9f9c-b20b36e3b6e6" (UID: "5946b615-2463-4674-9f9c-b20b36e3b6e6"). InnerVolumeSpecName "kube-api-access-lq96w". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 22:12:39.275704 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:12:39.275669 2545 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lq96w\" (UniqueName: \"kubernetes.io/projected/5946b615-2463-4674-9f9c-b20b36e3b6e6-kube-api-access-lq96w\") on node \"ip-10-0-131-55.ec2.internal\" DevicePath \"\"" Jul 07 22:12:39.782911 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:12:39.782875 2545 generic.go:356] "Generic (PLEG): container finished" podID="5946b615-2463-4674-9f9c-b20b36e3b6e6" containerID="4546e83fbb28ddb5207d4efa7ea40af0b20f514b019c3177151f6c2235533808" exitCode=0 Jul 07 22:12:39.782911 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:12:39.782910 2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-f7dtr" Jul 07 22:12:39.783123 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:12:39.782956 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-f7dtr" event={"ID":"5946b615-2463-4674-9f9c-b20b36e3b6e6","Type":"ContainerDied","Data":"4546e83fbb28ddb5207d4efa7ea40af0b20f514b019c3177151f6c2235533808"} Jul 07 22:12:39.783123 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:12:39.782998 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-f7dtr" event={"ID":"5946b615-2463-4674-9f9c-b20b36e3b6e6","Type":"ContainerDied","Data":"54646a611b7fd7ec4917c99e09e9adf2e00a7431635213026464b333aa67eb0a"} Jul 07 22:12:39.783123 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:12:39.783017 2545 scope.go:117] "RemoveContainer" containerID="4546e83fbb28ddb5207d4efa7ea40af0b20f514b019c3177151f6c2235533808" Jul 07 22:12:39.791595 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:12:39.791578 2545 scope.go:117] "RemoveContainer" containerID="4546e83fbb28ddb5207d4efa7ea40af0b20f514b019c3177151f6c2235533808" Jul 07 22:12:39.791834 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:12:39.791814 2545 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4546e83fbb28ddb5207d4efa7ea40af0b20f514b019c3177151f6c2235533808\": container with ID starting with 4546e83fbb28ddb5207d4efa7ea40af0b20f514b019c3177151f6c2235533808 not found: ID does not exist" containerID="4546e83fbb28ddb5207d4efa7ea40af0b20f514b019c3177151f6c2235533808" Jul 07 22:12:39.791888 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:12:39.791842 2545 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4546e83fbb28ddb5207d4efa7ea40af0b20f514b019c3177151f6c2235533808"} err="failed to get container status \"4546e83fbb28ddb5207d4efa7ea40af0b20f514b019c3177151f6c2235533808\": rpc error: code = NotFound desc = could not find container \"4546e83fbb28ddb5207d4efa7ea40af0b20f514b019c3177151f6c2235533808\": container with ID starting with 4546e83fbb28ddb5207d4efa7ea40af0b20f514b019c3177151f6c2235533808 not found: ID does not exist" Jul 07 22:12:39.802492 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:12:39.802468 2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-f7dtr"] Jul 07 22:12:39.804565 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:12:39.804546 2545 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-f7dtr"] Jul 07 22:12:40.311691 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:12:40.311656 2545 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5946b615-2463-4674-9f9c-b20b36e3b6e6" path="/var/lib/kubelet/pods/5946b615-2463-4674-9f9c-b20b36e3b6e6/volumes" Jul 07 22:13:38.156118 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:13:38.156074 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-rhg5r_71f36bfa-4b8a-4572-b522-57c16392aa0b/console-operator/1.log" Jul 07 22:13:38.156953 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:13:38.156934 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-rhg5r_71f36bfa-4b8a-4572-b522-57c16392aa0b/console-operator/1.log" Jul 07 22:14:11.110492 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:14:11.110413 2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-wtq5s"] Jul 07 22:14:11.113053 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:14:11.110908 2545 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5946b615-2463-4674-9f9c-b20b36e3b6e6" containerName="authorino" Jul 07 22:14:11.113053 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:14:11.110926 2545 state_mem.go:107] "Deleted CPUSet assignment" podUID="5946b615-2463-4674-9f9c-b20b36e3b6e6" containerName="authorino" Jul 07 22:14:11.113053 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:14:11.111005 2545 memory_manager.go:368] "RemoveStaleState removing state" podUID="5946b615-2463-4674-9f9c-b20b36e3b6e6" containerName="authorino" Jul 07 22:14:11.114014 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:14:11.113997 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wtq5s" Jul 07 22:14:11.117412 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:14:11.117387 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-wd56t\"" Jul 07 22:14:11.117538 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:14:11.117413 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 07 22:14:11.117538 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:14:11.117429 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 07 22:14:11.117538 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:14:11.117474 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 07 22:14:11.125911 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:14:11.125887 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-wtq5s"] Jul 07 22:14:11.163532 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:14:11.163497 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/302c7937-e5b4-49b8-a96f-1600395c20e1-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wtq5s\" (UID: \"302c7937-e5b4-49b8-a96f-1600395c20e1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wtq5s" Jul 07 22:14:11.163696 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:14:11.163541 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/302c7937-e5b4-49b8-a96f-1600395c20e1-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wtq5s\" (UID: \"302c7937-e5b4-49b8-a96f-1600395c20e1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wtq5s" Jul 07 22:14:11.163696 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:14:11.163601 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/302c7937-e5b4-49b8-a96f-1600395c20e1-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wtq5s\" (UID: \"302c7937-e5b4-49b8-a96f-1600395c20e1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wtq5s" Jul 07 22:14:11.163696 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:14:11.163642 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rdcjm\" (UniqueName: \"kubernetes.io/projected/302c7937-e5b4-49b8-a96f-1600395c20e1-kube-api-access-rdcjm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wtq5s\" (UID: \"302c7937-e5b4-49b8-a96f-1600395c20e1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wtq5s" Jul 07 22:14:11.163696 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:14:11.163679 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/302c7937-e5b4-49b8-a96f-1600395c20e1-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wtq5s\" (UID: \"302c7937-e5b4-49b8-a96f-1600395c20e1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wtq5s" Jul 07 22:14:11.163836 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:14:11.163701 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/302c7937-e5b4-49b8-a96f-1600395c20e1-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wtq5s\" (UID: \"302c7937-e5b4-49b8-a96f-1600395c20e1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wtq5s" Jul 07 22:14:11.265231 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:14:11.265189 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/302c7937-e5b4-49b8-a96f-1600395c20e1-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wtq5s\" (UID: \"302c7937-e5b4-49b8-a96f-1600395c20e1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wtq5s" Jul 07 22:14:11.265405 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:14:11.265242 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/302c7937-e5b4-49b8-a96f-1600395c20e1-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wtq5s\" (UID: \"302c7937-e5b4-49b8-a96f-1600395c20e1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wtq5s" Jul 07 22:14:11.265405 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:14:11.265284 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/302c7937-e5b4-49b8-a96f-1600395c20e1-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wtq5s\" (UID: \"302c7937-e5b4-49b8-a96f-1600395c20e1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wtq5s" Jul 07 22:14:11.265405 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:14:11.265321 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/302c7937-e5b4-49b8-a96f-1600395c20e1-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wtq5s\" (UID: \"302c7937-e5b4-49b8-a96f-1600395c20e1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wtq5s" Jul 07 22:14:11.265570 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:14:11.265452 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/302c7937-e5b4-49b8-a96f-1600395c20e1-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wtq5s\" (UID: \"302c7937-e5b4-49b8-a96f-1600395c20e1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wtq5s" Jul 07 22:14:11.265570 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:14:11.265535 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rdcjm\" (UniqueName: \"kubernetes.io/projected/302c7937-e5b4-49b8-a96f-1600395c20e1-kube-api-access-rdcjm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wtq5s\" (UID: \"302c7937-e5b4-49b8-a96f-1600395c20e1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wtq5s" Jul 07 22:14:11.265660 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:14:11.265617 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/302c7937-e5b4-49b8-a96f-1600395c20e1-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wtq5s\" (UID: \"302c7937-e5b4-49b8-a96f-1600395c20e1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wtq5s" Jul 07 22:14:11.265660 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:14:11.265639 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/302c7937-e5b4-49b8-a96f-1600395c20e1-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wtq5s\" (UID: \"302c7937-e5b4-49b8-a96f-1600395c20e1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wtq5s" Jul 07 22:14:11.265843 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:14:11.265816 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/302c7937-e5b4-49b8-a96f-1600395c20e1-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wtq5s\" (UID: \"302c7937-e5b4-49b8-a96f-1600395c20e1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wtq5s" Jul 07 22:14:11.267550 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:14:11.267528 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/302c7937-e5b4-49b8-a96f-1600395c20e1-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wtq5s\" (UID: \"302c7937-e5b4-49b8-a96f-1600395c20e1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wtq5s" Jul 07 22:14:11.267705 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:14:11.267688 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/302c7937-e5b4-49b8-a96f-1600395c20e1-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wtq5s\" (UID: \"302c7937-e5b4-49b8-a96f-1600395c20e1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wtq5s" Jul 07 22:14:11.272767 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:14:11.272743 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rdcjm\" (UniqueName: \"kubernetes.io/projected/302c7937-e5b4-49b8-a96f-1600395c20e1-kube-api-access-rdcjm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-wtq5s\" (UID: \"302c7937-e5b4-49b8-a96f-1600395c20e1\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wtq5s" Jul 07 22:14:11.425914 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:14:11.425822 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wtq5s" Jul 07 22:14:11.555258 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:14:11.555237 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-wtq5s"] Jul 07 22:14:11.557772 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:14:11.557744 2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod302c7937_e5b4_49b8_a96f_1600395c20e1.slice/crio-30020b9cb559d82577900c6ca0b1452a6a501dc69153246a9b80524a018a0f57 WatchSource:0}: Error finding container 30020b9cb559d82577900c6ca0b1452a6a501dc69153246a9b80524a018a0f57: Status 404 returned error can't find the container with id 30020b9cb559d82577900c6ca0b1452a6a501dc69153246a9b80524a018a0f57 Jul 07 22:14:11.559424 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:14:11.559406 2545 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 22:14:12.217592 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:14:12.217544 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wtq5s" event={"ID":"302c7937-e5b4-49b8-a96f-1600395c20e1","Type":"ContainerStarted","Data":"30020b9cb559d82577900c6ca0b1452a6a501dc69153246a9b80524a018a0f57"} Jul 07 22:14:13.223249 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:14:13.223210 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wtq5s" event={"ID":"302c7937-e5b4-49b8-a96f-1600395c20e1","Type":"ContainerStarted","Data":"8f6e7a3d5089de9380606cefb2dccc0eda6ccd3932c949e688b407d542878c90"} Jul 07 22:14:13.240300 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:14:13.240252 2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wtq5s" podStartSLOduration=1.119201144 podStartE2EDuration="2.24023577s" totalImagesPullingTime="1.121034626s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 22:14:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:14:13.239338106 +0000 UTC m=+635.553561145" watchObservedRunningTime="2026-07-07 22:14:13.24023577 +0000 UTC m=+635.554458793" Jul 07 22:14:21.426624 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:14:21.426590 2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wtq5s" Jul 07 22:14:21.427043 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:14:21.426676 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wtq5s" Jul 07 22:14:21.439058 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:14:21.439032 2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wtq5s" Jul 07 22:14:22.276044 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:14:22.276011 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-wtq5s" Jul 07 22:14:52.912252 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:14:52.912220 2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7m5n5"] Jul 07 22:14:52.917311 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:14:52.917292 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7m5n5" Jul 07 22:14:52.919795 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:14:52.919769 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jul 07 22:14:52.925033 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:14:52.925011 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7m5n5"] Jul 07 22:14:52.953501 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:14:52.953476 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/55987ec6-aed9-47dd-8410-9aebcfcfa32d-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7m5n5\" (UID: \"55987ec6-aed9-47dd-8410-9aebcfcfa32d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7m5n5" Jul 07 22:14:52.953617 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:14:52.953524 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/55987ec6-aed9-47dd-8410-9aebcfcfa32d-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7m5n5\" (UID: \"55987ec6-aed9-47dd-8410-9aebcfcfa32d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7m5n5" Jul 07 22:14:52.953617 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:14:52.953549 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/55987ec6-aed9-47dd-8410-9aebcfcfa32d-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7m5n5\" (UID: \"55987ec6-aed9-47dd-8410-9aebcfcfa32d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7m5n5" Jul 07 22:14:52.953712 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:14:52.953613 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/55987ec6-aed9-47dd-8410-9aebcfcfa32d-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7m5n5\" (UID: \"55987ec6-aed9-47dd-8410-9aebcfcfa32d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7m5n5" Jul 07 22:14:52.953712 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:14:52.953634 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dwbmh\" (UniqueName: \"kubernetes.io/projected/55987ec6-aed9-47dd-8410-9aebcfcfa32d-kube-api-access-dwbmh\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7m5n5\" (UID: \"55987ec6-aed9-47dd-8410-9aebcfcfa32d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7m5n5" Jul 07 22:14:52.953712 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:14:52.953655 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/55987ec6-aed9-47dd-8410-9aebcfcfa32d-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7m5n5\" (UID: \"55987ec6-aed9-47dd-8410-9aebcfcfa32d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7m5n5" Jul 07 22:14:53.054225 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:14:53.054190 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/55987ec6-aed9-47dd-8410-9aebcfcfa32d-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7m5n5\" (UID: \"55987ec6-aed9-47dd-8410-9aebcfcfa32d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7m5n5" Jul 07 22:14:53.054396 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:14:53.054238 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/55987ec6-aed9-47dd-8410-9aebcfcfa32d-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7m5n5\" (UID: \"55987ec6-aed9-47dd-8410-9aebcfcfa32d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7m5n5" Jul 07 22:14:53.054396 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:14:53.054264 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/55987ec6-aed9-47dd-8410-9aebcfcfa32d-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7m5n5\" (UID: \"55987ec6-aed9-47dd-8410-9aebcfcfa32d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7m5n5" Jul 07 22:14:53.054396 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:14:53.054288 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/55987ec6-aed9-47dd-8410-9aebcfcfa32d-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7m5n5\" (UID: \"55987ec6-aed9-47dd-8410-9aebcfcfa32d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7m5n5" Jul 07 22:14:53.054396 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:14:53.054308 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dwbmh\" (UniqueName: \"kubernetes.io/projected/55987ec6-aed9-47dd-8410-9aebcfcfa32d-kube-api-access-dwbmh\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7m5n5\" (UID: \"55987ec6-aed9-47dd-8410-9aebcfcfa32d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7m5n5" Jul 07 22:14:53.054396 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:14:53.054327 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/55987ec6-aed9-47dd-8410-9aebcfcfa32d-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7m5n5\" (UID: \"55987ec6-aed9-47dd-8410-9aebcfcfa32d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7m5n5" Jul 07 22:14:53.054679 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:14:53.054655 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/55987ec6-aed9-47dd-8410-9aebcfcfa32d-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7m5n5\" (UID: \"55987ec6-aed9-47dd-8410-9aebcfcfa32d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7m5n5" Jul 07 22:14:53.054736 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:14:53.054666 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/55987ec6-aed9-47dd-8410-9aebcfcfa32d-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7m5n5\" (UID: \"55987ec6-aed9-47dd-8410-9aebcfcfa32d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7m5n5" Jul 07 22:14:53.054736 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:14:53.054716 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/55987ec6-aed9-47dd-8410-9aebcfcfa32d-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7m5n5\" (UID: \"55987ec6-aed9-47dd-8410-9aebcfcfa32d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7m5n5" Jul 07 22:14:53.056688 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:14:53.056659 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/55987ec6-aed9-47dd-8410-9aebcfcfa32d-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7m5n5\" (UID: \"55987ec6-aed9-47dd-8410-9aebcfcfa32d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7m5n5" Jul 07 22:14:53.056923 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:14:53.056907 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/55987ec6-aed9-47dd-8410-9aebcfcfa32d-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7m5n5\" (UID: \"55987ec6-aed9-47dd-8410-9aebcfcfa32d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7m5n5" Jul 07 22:14:53.061383 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:14:53.061362 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dwbmh\" (UniqueName: \"kubernetes.io/projected/55987ec6-aed9-47dd-8410-9aebcfcfa32d-kube-api-access-dwbmh\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-7m5n5\" (UID: \"55987ec6-aed9-47dd-8410-9aebcfcfa32d\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7m5n5" Jul 07 22:14:53.227372 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:14:53.227334 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7m5n5" Jul 07 22:14:53.356868 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:14:53.356843 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7m5n5"] Jul 07 22:14:53.359001 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:14:53.358973 2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod55987ec6_aed9_47dd_8410_9aebcfcfa32d.slice/crio-35594d4cfcc8eb2d0a33eb0a647fc0ca982d49b004dca325a267e20f9d942f7b WatchSource:0}: Error finding container 35594d4cfcc8eb2d0a33eb0a647fc0ca982d49b004dca325a267e20f9d942f7b: Status 404 returned error can't find the container with id 35594d4cfcc8eb2d0a33eb0a647fc0ca982d49b004dca325a267e20f9d942f7b Jul 07 22:14:53.416958 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:14:53.416928 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7m5n5" event={"ID":"55987ec6-aed9-47dd-8410-9aebcfcfa32d","Type":"ContainerStarted","Data":"35594d4cfcc8eb2d0a33eb0a647fc0ca982d49b004dca325a267e20f9d942f7b"} Jul 07 22:14:54.422759 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:14:54.422723 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7m5n5" event={"ID":"55987ec6-aed9-47dd-8410-9aebcfcfa32d","Type":"ContainerStarted","Data":"d364ccc8bb3d6e1256ff5d1ae96b10febbe9da656eccf6f69c0d1bb669d6bcca"} Jul 07 22:14:54.439720 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:14:54.439672 2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7m5n5" podStartSLOduration=2.253931512 podStartE2EDuration="2.439655296s" totalImagesPullingTime="185.723784ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 22:14:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:14:54.438162232 +0000 UTC m=+676.752385253" watchObservedRunningTime="2026-07-07 22:14:54.439655296 +0000 UTC m=+676.753878316" Jul 07 22:15:03.228058 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:15:03.228017 2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7m5n5" Jul 07 22:15:03.228447 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:15:03.228070 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7m5n5" Jul 07 22:15:03.240286 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:15:03.240261 2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7m5n5" Jul 07 22:15:03.474990 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:15:03.474959 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-7m5n5" Jul 07 22:15:43.108651 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:15:43.108619 2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6c9b657c9c-jqr7l"] Jul 07 22:15:43.111866 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:15:43.111845 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6c9b657c9c-jqr7l" Jul 07 22:15:43.119288 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:15:43.119257 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6c9b657c9c-jqr7l"] Jul 07 22:15:43.197872 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:15:43.197839 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/303bd640-3c9c-4a1f-8cd6-5d3fb366af14-tls-cert\") pod \"authorino-6c9b657c9c-jqr7l\" (UID: \"303bd640-3c9c-4a1f-8cd6-5d3fb366af14\") " pod="kuadrant-system/authorino-6c9b657c9c-jqr7l" Jul 07 22:15:43.198039 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:15:43.197892 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hgr42\" (UniqueName: \"kubernetes.io/projected/303bd640-3c9c-4a1f-8cd6-5d3fb366af14-kube-api-access-hgr42\") pod \"authorino-6c9b657c9c-jqr7l\" (UID: \"303bd640-3c9c-4a1f-8cd6-5d3fb366af14\") " pod="kuadrant-system/authorino-6c9b657c9c-jqr7l" Jul 07 22:15:43.298969 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:15:43.298928 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/303bd640-3c9c-4a1f-8cd6-5d3fb366af14-tls-cert\") pod \"authorino-6c9b657c9c-jqr7l\" (UID: \"303bd640-3c9c-4a1f-8cd6-5d3fb366af14\") " pod="kuadrant-system/authorino-6c9b657c9c-jqr7l" Jul 07 22:15:43.299180 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:15:43.298984 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hgr42\" (UniqueName: \"kubernetes.io/projected/303bd640-3c9c-4a1f-8cd6-5d3fb366af14-kube-api-access-hgr42\") pod \"authorino-6c9b657c9c-jqr7l\" (UID: \"303bd640-3c9c-4a1f-8cd6-5d3fb366af14\") " pod="kuadrant-system/authorino-6c9b657c9c-jqr7l" Jul 07 22:15:43.301423 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:15:43.301400 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/303bd640-3c9c-4a1f-8cd6-5d3fb366af14-tls-cert\") pod \"authorino-6c9b657c9c-jqr7l\" (UID: \"303bd640-3c9c-4a1f-8cd6-5d3fb366af14\") " pod="kuadrant-system/authorino-6c9b657c9c-jqr7l" Jul 07 22:15:43.306712 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:15:43.306693 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hgr42\" (UniqueName: \"kubernetes.io/projected/303bd640-3c9c-4a1f-8cd6-5d3fb366af14-kube-api-access-hgr42\") pod \"authorino-6c9b657c9c-jqr7l\" (UID: \"303bd640-3c9c-4a1f-8cd6-5d3fb366af14\") " pod="kuadrant-system/authorino-6c9b657c9c-jqr7l" Jul 07 22:15:43.421775 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:15:43.421677 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6c9b657c9c-jqr7l" Jul 07 22:15:43.560364 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:15:43.560339 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6c9b657c9c-jqr7l"] Jul 07 22:15:43.565169 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:15:43.563168 2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod303bd640_3c9c_4a1f_8cd6_5d3fb366af14.slice/crio-9b0ad72e667a06aad88b6a1bc55a18e05187ec1c7469f17c96617a73ebdaa480 WatchSource:0}: Error finding container 9b0ad72e667a06aad88b6a1bc55a18e05187ec1c7469f17c96617a73ebdaa480: Status 404 returned error can't find the container with id 9b0ad72e667a06aad88b6a1bc55a18e05187ec1c7469f17c96617a73ebdaa480 Jul 07 22:15:43.649569 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:15:43.649532 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6c9b657c9c-jqr7l" event={"ID":"303bd640-3c9c-4a1f-8cd6-5d3fb366af14","Type":"ContainerStarted","Data":"9b0ad72e667a06aad88b6a1bc55a18e05187ec1c7469f17c96617a73ebdaa480"} Jul 07 22:15:44.656720 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:15:44.656678 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6c9b657c9c-jqr7l" event={"ID":"303bd640-3c9c-4a1f-8cd6-5d3fb366af14","Type":"ContainerStarted","Data":"33a4cbce558334515ac33cc389b4d1eb9ee48e59d657090a086c342a81cf00a9"} Jul 07 22:15:44.713103 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:15:44.712509 2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-6c9b657c9c-jqr7l" podStartSLOduration=1.255540812 podStartE2EDuration="1.712485959s" totalImagesPullingTime="456.945147ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 22:15:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:15:44.671983782 +0000 UTC m=+726.986206802" watchObservedRunningTime="2026-07-07 22:15:44.712485959 +0000 UTC m=+727.026708982" Jul 07 22:15:44.720132 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:15:44.715452 2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5bf4d5c564-89zgc"] Jul 07 22:15:44.720132 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:15:44.715668 2545 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-5bf4d5c564-89zgc" podUID="6473e6a2-bfb7-48ba-8f1d-e1c302b1ce1f" containerName="authorino" containerID="cri-o://c16a9856154473424e394271ddd49236801f759680a2f1b2b87609414d726408" gracePeriod=30 Jul 07 22:15:44.993195 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:15:44.993169 2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5bf4d5c564-89zgc" Jul 07 22:15:45.019925 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:15:45.019894 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/6473e6a2-bfb7-48ba-8f1d-e1c302b1ce1f-tls-cert\") pod \"6473e6a2-bfb7-48ba-8f1d-e1c302b1ce1f\" (UID: \"6473e6a2-bfb7-48ba-8f1d-e1c302b1ce1f\") " Jul 07 22:15:45.020163 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:15:45.019946 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rbb5m\" (UniqueName: \"kubernetes.io/projected/6473e6a2-bfb7-48ba-8f1d-e1c302b1ce1f-kube-api-access-rbb5m\") pod \"6473e6a2-bfb7-48ba-8f1d-e1c302b1ce1f\" (UID: \"6473e6a2-bfb7-48ba-8f1d-e1c302b1ce1f\") " Jul 07 22:15:45.022145 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:15:45.022110 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6473e6a2-bfb7-48ba-8f1d-e1c302b1ce1f-kube-api-access-rbb5m" (OuterVolumeSpecName: "kube-api-access-rbb5m") pod "6473e6a2-bfb7-48ba-8f1d-e1c302b1ce1f" (UID: "6473e6a2-bfb7-48ba-8f1d-e1c302b1ce1f"). InnerVolumeSpecName "kube-api-access-rbb5m". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 22:15:45.030597 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:15:45.030567 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6473e6a2-bfb7-48ba-8f1d-e1c302b1ce1f-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "6473e6a2-bfb7-48ba-8f1d-e1c302b1ce1f" (UID: "6473e6a2-bfb7-48ba-8f1d-e1c302b1ce1f"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 22:15:45.121050 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:15:45.121016 2545 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/6473e6a2-bfb7-48ba-8f1d-e1c302b1ce1f-tls-cert\") on node \"ip-10-0-131-55.ec2.internal\" DevicePath \"\"" Jul 07 22:15:45.121050 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:15:45.121044 2545 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rbb5m\" (UniqueName: \"kubernetes.io/projected/6473e6a2-bfb7-48ba-8f1d-e1c302b1ce1f-kube-api-access-rbb5m\") on node \"ip-10-0-131-55.ec2.internal\" DevicePath \"\"" Jul 07 22:15:45.663380 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:15:45.663286 2545 generic.go:356] "Generic (PLEG): container finished" podID="6473e6a2-bfb7-48ba-8f1d-e1c302b1ce1f" containerID="c16a9856154473424e394271ddd49236801f759680a2f1b2b87609414d726408" exitCode=0 Jul 07 22:15:45.663380 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:15:45.663345 2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5bf4d5c564-89zgc" Jul 07 22:15:45.663863 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:15:45.663380 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5bf4d5c564-89zgc" event={"ID":"6473e6a2-bfb7-48ba-8f1d-e1c302b1ce1f","Type":"ContainerDied","Data":"c16a9856154473424e394271ddd49236801f759680a2f1b2b87609414d726408"} Jul 07 22:15:45.663863 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:15:45.663431 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5bf4d5c564-89zgc" event={"ID":"6473e6a2-bfb7-48ba-8f1d-e1c302b1ce1f","Type":"ContainerDied","Data":"8d22994e5bbfee9c7d6b6db652b1d0beda4824aabc261e3944b9c625e34332d4"} Jul 07 22:15:45.663863 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:15:45.663458 2545 scope.go:117] "RemoveContainer" containerID="c16a9856154473424e394271ddd49236801f759680a2f1b2b87609414d726408" Jul 07 22:15:45.672315 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:15:45.672296 2545 scope.go:117] "RemoveContainer" containerID="c16a9856154473424e394271ddd49236801f759680a2f1b2b87609414d726408" Jul 07 22:15:45.672556 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:15:45.672537 2545 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c16a9856154473424e394271ddd49236801f759680a2f1b2b87609414d726408\": container with ID starting with c16a9856154473424e394271ddd49236801f759680a2f1b2b87609414d726408 not found: ID does not exist" containerID="c16a9856154473424e394271ddd49236801f759680a2f1b2b87609414d726408" Jul 07 22:15:45.672611 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:15:45.672564 2545 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c16a9856154473424e394271ddd49236801f759680a2f1b2b87609414d726408"} err="failed to get container status \"c16a9856154473424e394271ddd49236801f759680a2f1b2b87609414d726408\": rpc error: code = NotFound desc = could not find container \"c16a9856154473424e394271ddd49236801f759680a2f1b2b87609414d726408\": container with ID starting with c16a9856154473424e394271ddd49236801f759680a2f1b2b87609414d726408 not found: ID does not exist" Jul 07 22:15:45.687180 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:15:45.687149 2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5bf4d5c564-89zgc"] Jul 07 22:15:45.689203 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:15:45.689184 2545 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5bf4d5c564-89zgc"] Jul 07 22:15:46.311792 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:15:46.311750 2545 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6473e6a2-bfb7-48ba-8f1d-e1c302b1ce1f" path="/var/lib/kubelet/pods/6473e6a2-bfb7-48ba-8f1d-e1c302b1ce1f/volumes" Jul 07 22:18:38.189983 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:18:38.189951 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-rhg5r_71f36bfa-4b8a-4572-b522-57c16392aa0b/console-operator/1.log" Jul 07 22:18:38.192038 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:18:38.192013 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-rhg5r_71f36bfa-4b8a-4572-b522-57c16392aa0b/console-operator/1.log" Jul 07 22:23:38.218869 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:23:38.218838 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-rhg5r_71f36bfa-4b8a-4572-b522-57c16392aa0b/console-operator/1.log" Jul 07 22:23:38.222574 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:23:38.222546 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-rhg5r_71f36bfa-4b8a-4572-b522-57c16392aa0b/console-operator/1.log" Jul 07 22:28:38.250942 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:28:38.250910 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-rhg5r_71f36bfa-4b8a-4572-b522-57c16392aa0b/console-operator/1.log" Jul 07 22:28:38.253399 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:28:38.252720 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-rhg5r_71f36bfa-4b8a-4572-b522-57c16392aa0b/console-operator/1.log" Jul 07 22:29:18.772753 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:29:18.772716 2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-86400e9a-openshift-default-75d7ddfd65-99b9r"] Jul 07 22:29:18.773522 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:29:18.773499 2545 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6473e6a2-bfb7-48ba-8f1d-e1c302b1ce1f" containerName="authorino" Jul 07 22:29:18.773522 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:29:18.773524 2545 state_mem.go:107] "Deleted CPUSet assignment" podUID="6473e6a2-bfb7-48ba-8f1d-e1c302b1ce1f" containerName="authorino" Jul 07 22:29:18.773684 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:29:18.773658 2545 memory_manager.go:368] "RemoveStaleState removing state" podUID="6473e6a2-bfb7-48ba-8f1d-e1c302b1ce1f" containerName="authorino" Jul 07 22:29:18.776691 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:29:18.776660 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-86400e9a-openshift-default-75d7ddfd65-99b9r" Jul 07 22:29:18.779278 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:29:18.779255 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-86400e9a-openshift-default-dockercfg-tsgxq\"" Jul 07 22:29:18.779390 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:29:18.779261 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 07 22:29:18.789859 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:29:18.789836 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-86400e9a-openshift-default-75d7ddfd65-99b9r"] Jul 07 22:29:18.877532 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:29:18.877498 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1257de19-2b28-4db0-9eca-f5f45eb904b1-istio-podinfo\") pod \"e2e-ait-86400e9a-openshift-default-75d7ddfd65-99b9r\" (UID: \"1257de19-2b28-4db0-9eca-f5f45eb904b1\") " pod="openshift-ingress/e2e-ait-86400e9a-openshift-default-75d7ddfd65-99b9r" Jul 07 22:29:18.877532 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:29:18.877543 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1257de19-2b28-4db0-9eca-f5f45eb904b1-workload-socket\") pod \"e2e-ait-86400e9a-openshift-default-75d7ddfd65-99b9r\" (UID: \"1257de19-2b28-4db0-9eca-f5f45eb904b1\") " pod="openshift-ingress/e2e-ait-86400e9a-openshift-default-75d7ddfd65-99b9r" Jul 07 22:29:18.877736 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:29:18.877568 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1257de19-2b28-4db0-9eca-f5f45eb904b1-istiod-ca-cert\") pod \"e2e-ait-86400e9a-openshift-default-75d7ddfd65-99b9r\" (UID: \"1257de19-2b28-4db0-9eca-f5f45eb904b1\") " pod="openshift-ingress/e2e-ait-86400e9a-openshift-default-75d7ddfd65-99b9r" Jul 07 22:29:18.877736 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:29:18.877667 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1257de19-2b28-4db0-9eca-f5f45eb904b1-istio-envoy\") pod \"e2e-ait-86400e9a-openshift-default-75d7ddfd65-99b9r\" (UID: \"1257de19-2b28-4db0-9eca-f5f45eb904b1\") " pod="openshift-ingress/e2e-ait-86400e9a-openshift-default-75d7ddfd65-99b9r" Jul 07 22:29:18.877736 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:29:18.877696 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1257de19-2b28-4db0-9eca-f5f45eb904b1-istio-data\") pod \"e2e-ait-86400e9a-openshift-default-75d7ddfd65-99b9r\" (UID: \"1257de19-2b28-4db0-9eca-f5f45eb904b1\") " pod="openshift-ingress/e2e-ait-86400e9a-openshift-default-75d7ddfd65-99b9r" Jul 07 22:29:18.877736 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:29:18.877727 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1257de19-2b28-4db0-9eca-f5f45eb904b1-workload-certs\") pod \"e2e-ait-86400e9a-openshift-default-75d7ddfd65-99b9r\" (UID: \"1257de19-2b28-4db0-9eca-f5f45eb904b1\") " pod="openshift-ingress/e2e-ait-86400e9a-openshift-default-75d7ddfd65-99b9r" Jul 07 22:29:18.877863 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:29:18.877774 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1257de19-2b28-4db0-9eca-f5f45eb904b1-credential-socket\") pod \"e2e-ait-86400e9a-openshift-default-75d7ddfd65-99b9r\" (UID: \"1257de19-2b28-4db0-9eca-f5f45eb904b1\") " pod="openshift-ingress/e2e-ait-86400e9a-openshift-default-75d7ddfd65-99b9r" Jul 07 22:29:18.877863 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:29:18.877834 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1257de19-2b28-4db0-9eca-f5f45eb904b1-istio-token\") pod \"e2e-ait-86400e9a-openshift-default-75d7ddfd65-99b9r\" (UID: \"1257de19-2b28-4db0-9eca-f5f45eb904b1\") " pod="openshift-ingress/e2e-ait-86400e9a-openshift-default-75d7ddfd65-99b9r" Jul 07 22:29:18.877926 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:29:18.877867 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cdnl6\" (UniqueName: \"kubernetes.io/projected/1257de19-2b28-4db0-9eca-f5f45eb904b1-kube-api-access-cdnl6\") pod \"e2e-ait-86400e9a-openshift-default-75d7ddfd65-99b9r\" (UID: \"1257de19-2b28-4db0-9eca-f5f45eb904b1\") " pod="openshift-ingress/e2e-ait-86400e9a-openshift-default-75d7ddfd65-99b9r" Jul 07 22:29:18.978579 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:29:18.978541 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1257de19-2b28-4db0-9eca-f5f45eb904b1-istio-token\") pod \"e2e-ait-86400e9a-openshift-default-75d7ddfd65-99b9r\" (UID: \"1257de19-2b28-4db0-9eca-f5f45eb904b1\") " pod="openshift-ingress/e2e-ait-86400e9a-openshift-default-75d7ddfd65-99b9r" Jul 07 22:29:18.978760 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:29:18.978589 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cdnl6\" (UniqueName: \"kubernetes.io/projected/1257de19-2b28-4db0-9eca-f5f45eb904b1-kube-api-access-cdnl6\") pod \"e2e-ait-86400e9a-openshift-default-75d7ddfd65-99b9r\" (UID: \"1257de19-2b28-4db0-9eca-f5f45eb904b1\") " pod="openshift-ingress/e2e-ait-86400e9a-openshift-default-75d7ddfd65-99b9r" Jul 07 22:29:18.978760 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:29:18.978622 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1257de19-2b28-4db0-9eca-f5f45eb904b1-istio-podinfo\") pod \"e2e-ait-86400e9a-openshift-default-75d7ddfd65-99b9r\" (UID: \"1257de19-2b28-4db0-9eca-f5f45eb904b1\") " pod="openshift-ingress/e2e-ait-86400e9a-openshift-default-75d7ddfd65-99b9r" Jul 07 22:29:18.978760 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:29:18.978654 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1257de19-2b28-4db0-9eca-f5f45eb904b1-workload-socket\") pod \"e2e-ait-86400e9a-openshift-default-75d7ddfd65-99b9r\" (UID: \"1257de19-2b28-4db0-9eca-f5f45eb904b1\") " pod="openshift-ingress/e2e-ait-86400e9a-openshift-default-75d7ddfd65-99b9r" Jul 07 22:29:18.978760 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:29:18.978674 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1257de19-2b28-4db0-9eca-f5f45eb904b1-istiod-ca-cert\") pod \"e2e-ait-86400e9a-openshift-default-75d7ddfd65-99b9r\" (UID: \"1257de19-2b28-4db0-9eca-f5f45eb904b1\") " pod="openshift-ingress/e2e-ait-86400e9a-openshift-default-75d7ddfd65-99b9r" Jul 07 22:29:18.978969 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:29:18.978806 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1257de19-2b28-4db0-9eca-f5f45eb904b1-istio-envoy\") pod \"e2e-ait-86400e9a-openshift-default-75d7ddfd65-99b9r\" (UID: \"1257de19-2b28-4db0-9eca-f5f45eb904b1\") " pod="openshift-ingress/e2e-ait-86400e9a-openshift-default-75d7ddfd65-99b9r" Jul 07 22:29:18.978969 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:29:18.978848 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1257de19-2b28-4db0-9eca-f5f45eb904b1-istio-data\") pod \"e2e-ait-86400e9a-openshift-default-75d7ddfd65-99b9r\" (UID: \"1257de19-2b28-4db0-9eca-f5f45eb904b1\") " pod="openshift-ingress/e2e-ait-86400e9a-openshift-default-75d7ddfd65-99b9r" Jul 07 22:29:18.978969 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:29:18.978884 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1257de19-2b28-4db0-9eca-f5f45eb904b1-workload-certs\") pod \"e2e-ait-86400e9a-openshift-default-75d7ddfd65-99b9r\" (UID: \"1257de19-2b28-4db0-9eca-f5f45eb904b1\") " pod="openshift-ingress/e2e-ait-86400e9a-openshift-default-75d7ddfd65-99b9r" Jul 07 22:29:18.978969 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:29:18.978936 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1257de19-2b28-4db0-9eca-f5f45eb904b1-credential-socket\") pod \"e2e-ait-86400e9a-openshift-default-75d7ddfd65-99b9r\" (UID: \"1257de19-2b28-4db0-9eca-f5f45eb904b1\") " pod="openshift-ingress/e2e-ait-86400e9a-openshift-default-75d7ddfd65-99b9r" Jul 07 22:29:18.979208 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:29:18.979182 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1257de19-2b28-4db0-9eca-f5f45eb904b1-workload-socket\") pod \"e2e-ait-86400e9a-openshift-default-75d7ddfd65-99b9r\" (UID: \"1257de19-2b28-4db0-9eca-f5f45eb904b1\") " pod="openshift-ingress/e2e-ait-86400e9a-openshift-default-75d7ddfd65-99b9r" Jul 07 22:29:18.979398 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:29:18.979288 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1257de19-2b28-4db0-9eca-f5f45eb904b1-istio-data\") pod \"e2e-ait-86400e9a-openshift-default-75d7ddfd65-99b9r\" (UID: \"1257de19-2b28-4db0-9eca-f5f45eb904b1\") " pod="openshift-ingress/e2e-ait-86400e9a-openshift-default-75d7ddfd65-99b9r" Jul 07 22:29:18.979458 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:29:18.979316 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1257de19-2b28-4db0-9eca-f5f45eb904b1-credential-socket\") pod \"e2e-ait-86400e9a-openshift-default-75d7ddfd65-99b9r\" (UID: \"1257de19-2b28-4db0-9eca-f5f45eb904b1\") " pod="openshift-ingress/e2e-ait-86400e9a-openshift-default-75d7ddfd65-99b9r" Jul 07 22:29:18.979515 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:29:18.979496 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1257de19-2b28-4db0-9eca-f5f45eb904b1-istiod-ca-cert\") pod \"e2e-ait-86400e9a-openshift-default-75d7ddfd65-99b9r\" (UID: \"1257de19-2b28-4db0-9eca-f5f45eb904b1\") " pod="openshift-ingress/e2e-ait-86400e9a-openshift-default-75d7ddfd65-99b9r" Jul 07 22:29:18.979563 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:29:18.979512 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1257de19-2b28-4db0-9eca-f5f45eb904b1-workload-certs\") pod \"e2e-ait-86400e9a-openshift-default-75d7ddfd65-99b9r\" (UID: \"1257de19-2b28-4db0-9eca-f5f45eb904b1\") " pod="openshift-ingress/e2e-ait-86400e9a-openshift-default-75d7ddfd65-99b9r" Jul 07 22:29:18.981061 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:29:18.981040 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1257de19-2b28-4db0-9eca-f5f45eb904b1-istio-envoy\") pod \"e2e-ait-86400e9a-openshift-default-75d7ddfd65-99b9r\" (UID: \"1257de19-2b28-4db0-9eca-f5f45eb904b1\") " pod="openshift-ingress/e2e-ait-86400e9a-openshift-default-75d7ddfd65-99b9r" Jul 07 22:29:18.981322 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:29:18.981303 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1257de19-2b28-4db0-9eca-f5f45eb904b1-istio-podinfo\") pod \"e2e-ait-86400e9a-openshift-default-75d7ddfd65-99b9r\" (UID: \"1257de19-2b28-4db0-9eca-f5f45eb904b1\") " pod="openshift-ingress/e2e-ait-86400e9a-openshift-default-75d7ddfd65-99b9r" Jul 07 22:29:18.985621 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:29:18.985597 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1257de19-2b28-4db0-9eca-f5f45eb904b1-istio-token\") pod \"e2e-ait-86400e9a-openshift-default-75d7ddfd65-99b9r\" (UID: \"1257de19-2b28-4db0-9eca-f5f45eb904b1\") " pod="openshift-ingress/e2e-ait-86400e9a-openshift-default-75d7ddfd65-99b9r" Jul 07 22:29:18.985889 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:29:18.985870 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cdnl6\" (UniqueName: \"kubernetes.io/projected/1257de19-2b28-4db0-9eca-f5f45eb904b1-kube-api-access-cdnl6\") pod \"e2e-ait-86400e9a-openshift-default-75d7ddfd65-99b9r\" (UID: \"1257de19-2b28-4db0-9eca-f5f45eb904b1\") " pod="openshift-ingress/e2e-ait-86400e9a-openshift-default-75d7ddfd65-99b9r" Jul 07 22:29:19.089971 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:29:19.089889 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-86400e9a-openshift-default-75d7ddfd65-99b9r" Jul 07 22:29:19.234329 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:29:19.234303 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-86400e9a-openshift-default-75d7ddfd65-99b9r"] Jul 07 22:29:19.236488 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:29:19.236457 2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1257de19_2b28_4db0_9eca_f5f45eb904b1.slice/crio-6fc75d0f5a4a4be63b2b5caeb62eaecbf78b11b9ba39b414dadf04cea0ea03e7 WatchSource:0}: Error finding container 6fc75d0f5a4a4be63b2b5caeb62eaecbf78b11b9ba39b414dadf04cea0ea03e7: Status 404 returned error can't find the container with id 6fc75d0f5a4a4be63b2b5caeb62eaecbf78b11b9ba39b414dadf04cea0ea03e7 Jul 07 22:29:19.238474 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:29:19.238454 2545 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 22:29:19.489923 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:29:19.489863 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-86400e9a-openshift-default-75d7ddfd65-99b9r" event={"ID":"1257de19-2b28-4db0-9eca-f5f45eb904b1","Type":"ContainerStarted","Data":"6fc75d0f5a4a4be63b2b5caeb62eaecbf78b11b9ba39b414dadf04cea0ea03e7"} Jul 07 22:29:21.745659 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:29:21.745617 2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 22:29:21.745929 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:29:21.745699 2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 22:29:21.745929 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:29:21.745728 2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 22:29:22.505441 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:29:22.505401 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-86400e9a-openshift-default-75d7ddfd65-99b9r" event={"ID":"1257de19-2b28-4db0-9eca-f5f45eb904b1","Type":"ContainerStarted","Data":"c3dbcb1daf00e4a492c7ccd951c13b5738127d55e9026ef9d9d4248af7812e9a"} Jul 07 22:29:22.524046 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:29:22.523984 2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-86400e9a-openshift-default-75d7ddfd65-99b9r" podStartSLOduration=2.0172378 podStartE2EDuration="4.523965648s" totalImagesPullingTime="2.506727848s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 22:29:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:29:22.523727495 +0000 UTC m=+1544.837950515" watchObservedRunningTime="2026-07-07 22:29:22.523965648 +0000 UTC m=+1544.838188666" Jul 07 22:29:23.090964 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:29:23.090921 2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-86400e9a-openshift-default-75d7ddfd65-99b9r" Jul 07 22:29:23.095751 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:29:23.095725 2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-86400e9a-openshift-default-75d7ddfd65-99b9r" Jul 07 22:29:23.510234 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:29:23.510198 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-86400e9a-openshift-default-75d7ddfd65-99b9r" Jul 07 22:29:23.511322 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:29:23.511303 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-86400e9a-openshift-default-75d7ddfd65-99b9r" Jul 07 22:29:31.058367 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:29:31.058333 2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-86400e9a-openshift-default-75d7ddfd65-99b9r"] Jul 07 22:29:31.058770 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:29:31.058503 2545 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-86400e9a-openshift-default-75d7ddfd65-99b9r" podUID="1257de19-2b28-4db0-9eca-f5f45eb904b1" containerName="istio-proxy" containerID="cri-o://c3dbcb1daf00e4a492c7ccd951c13b5738127d55e9026ef9d9d4248af7812e9a" gracePeriod=30 Jul 07 22:29:31.196912 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:29:31.196887 2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-86400e9a-openshift-default-75d7ddfd65-99b9r" Jul 07 22:29:31.208143 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:29:31.208116 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1257de19-2b28-4db0-9eca-f5f45eb904b1-istiod-ca-cert\") pod \"1257de19-2b28-4db0-9eca-f5f45eb904b1\" (UID: \"1257de19-2b28-4db0-9eca-f5f45eb904b1\") " Jul 07 22:29:31.208245 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:29:31.208151 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1257de19-2b28-4db0-9eca-f5f45eb904b1-istio-token\") pod \"1257de19-2b28-4db0-9eca-f5f45eb904b1\" (UID: \"1257de19-2b28-4db0-9eca-f5f45eb904b1\") " Jul 07 22:29:31.208245 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:29:31.208209 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1257de19-2b28-4db0-9eca-f5f45eb904b1-istio-podinfo\") pod \"1257de19-2b28-4db0-9eca-f5f45eb904b1\" (UID: \"1257de19-2b28-4db0-9eca-f5f45eb904b1\") " Jul 07 22:29:31.208245 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:29:31.208241 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cdnl6\" (UniqueName: \"kubernetes.io/projected/1257de19-2b28-4db0-9eca-f5f45eb904b1-kube-api-access-cdnl6\") pod \"1257de19-2b28-4db0-9eca-f5f45eb904b1\" (UID: \"1257de19-2b28-4db0-9eca-f5f45eb904b1\") " Jul 07 22:29:31.208402 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:29:31.208261 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1257de19-2b28-4db0-9eca-f5f45eb904b1-workload-socket\") pod \"1257de19-2b28-4db0-9eca-f5f45eb904b1\" (UID: \"1257de19-2b28-4db0-9eca-f5f45eb904b1\") " Jul 07 22:29:31.208402 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:29:31.208282 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1257de19-2b28-4db0-9eca-f5f45eb904b1-credential-socket\") pod \"1257de19-2b28-4db0-9eca-f5f45eb904b1\" (UID: \"1257de19-2b28-4db0-9eca-f5f45eb904b1\") " Jul 07 22:29:31.208402 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:29:31.208304 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1257de19-2b28-4db0-9eca-f5f45eb904b1-istio-envoy\") pod \"1257de19-2b28-4db0-9eca-f5f45eb904b1\" (UID: \"1257de19-2b28-4db0-9eca-f5f45eb904b1\") " Jul 07 22:29:31.208402 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:29:31.208367 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1257de19-2b28-4db0-9eca-f5f45eb904b1-workload-certs\") pod \"1257de19-2b28-4db0-9eca-f5f45eb904b1\" (UID: \"1257de19-2b28-4db0-9eca-f5f45eb904b1\") " Jul 07 22:29:31.208402 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:29:31.208391 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1257de19-2b28-4db0-9eca-f5f45eb904b1-istio-data\") pod \"1257de19-2b28-4db0-9eca-f5f45eb904b1\" (UID: \"1257de19-2b28-4db0-9eca-f5f45eb904b1\") " Jul 07 22:29:31.208661 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:29:31.208543 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1257de19-2b28-4db0-9eca-f5f45eb904b1-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1257de19-2b28-4db0-9eca-f5f45eb904b1" (UID: "1257de19-2b28-4db0-9eca-f5f45eb904b1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 22:29:31.208661 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:29:31.208466 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1257de19-2b28-4db0-9eca-f5f45eb904b1-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1257de19-2b28-4db0-9eca-f5f45eb904b1" (UID: "1257de19-2b28-4db0-9eca-f5f45eb904b1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 22:29:31.208761 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:29:31.208675 2545 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1257de19-2b28-4db0-9eca-f5f45eb904b1-istiod-ca-cert\") on node \"ip-10-0-131-55.ec2.internal\" DevicePath \"\"" Jul 07 22:29:31.208761 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:29:31.208695 2545 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1257de19-2b28-4db0-9eca-f5f45eb904b1-workload-socket\") on node \"ip-10-0-131-55.ec2.internal\" DevicePath \"\"" Jul 07 22:29:31.208761 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:29:31.208752 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1257de19-2b28-4db0-9eca-f5f45eb904b1-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1257de19-2b28-4db0-9eca-f5f45eb904b1" (UID: "1257de19-2b28-4db0-9eca-f5f45eb904b1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 22:29:31.208909 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:29:31.208892 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1257de19-2b28-4db0-9eca-f5f45eb904b1-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1257de19-2b28-4db0-9eca-f5f45eb904b1" (UID: "1257de19-2b28-4db0-9eca-f5f45eb904b1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 22:29:31.209478 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:29:31.209450 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1257de19-2b28-4db0-9eca-f5f45eb904b1-istio-data" (OuterVolumeSpecName: "istio-data") pod "1257de19-2b28-4db0-9eca-f5f45eb904b1" (UID: "1257de19-2b28-4db0-9eca-f5f45eb904b1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 22:29:31.210551 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:29:31.210529 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1257de19-2b28-4db0-9eca-f5f45eb904b1-istio-token" (OuterVolumeSpecName: "istio-token") pod "1257de19-2b28-4db0-9eca-f5f45eb904b1" (UID: "1257de19-2b28-4db0-9eca-f5f45eb904b1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 22:29:31.210646 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:29:31.210620 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1257de19-2b28-4db0-9eca-f5f45eb904b1-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1257de19-2b28-4db0-9eca-f5f45eb904b1" (UID: "1257de19-2b28-4db0-9eca-f5f45eb904b1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 22:29:31.210687 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:29:31.210643 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1257de19-2b28-4db0-9eca-f5f45eb904b1-kube-api-access-cdnl6" (OuterVolumeSpecName: "kube-api-access-cdnl6") pod "1257de19-2b28-4db0-9eca-f5f45eb904b1" (UID: "1257de19-2b28-4db0-9eca-f5f45eb904b1"). InnerVolumeSpecName "kube-api-access-cdnl6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 22:29:31.210723 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:29:31.210680 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1257de19-2b28-4db0-9eca-f5f45eb904b1-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1257de19-2b28-4db0-9eca-f5f45eb904b1" (UID: "1257de19-2b28-4db0-9eca-f5f45eb904b1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 22:29:31.309460 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:29:31.309426 2545 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1257de19-2b28-4db0-9eca-f5f45eb904b1-istio-podinfo\") on node \"ip-10-0-131-55.ec2.internal\" DevicePath \"\"" Jul 07 22:29:31.309460 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:29:31.309456 2545 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cdnl6\" (UniqueName: \"kubernetes.io/projected/1257de19-2b28-4db0-9eca-f5f45eb904b1-kube-api-access-cdnl6\") on node \"ip-10-0-131-55.ec2.internal\" DevicePath \"\"" Jul 07 22:29:31.309460 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:29:31.309468 2545 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1257de19-2b28-4db0-9eca-f5f45eb904b1-credential-socket\") on node \"ip-10-0-131-55.ec2.internal\" DevicePath \"\"" Jul 07 22:29:31.309684 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:29:31.309477 2545 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1257de19-2b28-4db0-9eca-f5f45eb904b1-istio-envoy\") on node \"ip-10-0-131-55.ec2.internal\" DevicePath \"\"" Jul 07 22:29:31.309684 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:29:31.309485 2545 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1257de19-2b28-4db0-9eca-f5f45eb904b1-workload-certs\") on node \"ip-10-0-131-55.ec2.internal\" DevicePath \"\"" Jul 07 22:29:31.309684 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:29:31.309493 2545 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1257de19-2b28-4db0-9eca-f5f45eb904b1-istio-data\") on node \"ip-10-0-131-55.ec2.internal\" DevicePath \"\"" Jul 07 22:29:31.309684 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:29:31.309500 2545 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1257de19-2b28-4db0-9eca-f5f45eb904b1-istio-token\") on node \"ip-10-0-131-55.ec2.internal\" DevicePath \"\"" Jul 07 22:29:31.550982 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:29:31.550892 2545 generic.go:356] "Generic (PLEG): container finished" podID="1257de19-2b28-4db0-9eca-f5f45eb904b1" containerID="c3dbcb1daf00e4a492c7ccd951c13b5738127d55e9026ef9d9d4248af7812e9a" exitCode=0 Jul 07 22:29:31.550982 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:29:31.550935 2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-86400e9a-openshift-default-75d7ddfd65-99b9r" Jul 07 22:29:31.551175 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:29:31.550977 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-86400e9a-openshift-default-75d7ddfd65-99b9r" event={"ID":"1257de19-2b28-4db0-9eca-f5f45eb904b1","Type":"ContainerDied","Data":"c3dbcb1daf00e4a492c7ccd951c13b5738127d55e9026ef9d9d4248af7812e9a"} Jul 07 22:29:31.551175 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:29:31.551029 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-86400e9a-openshift-default-75d7ddfd65-99b9r" event={"ID":"1257de19-2b28-4db0-9eca-f5f45eb904b1","Type":"ContainerDied","Data":"6fc75d0f5a4a4be63b2b5caeb62eaecbf78b11b9ba39b414dadf04cea0ea03e7"} Jul 07 22:29:31.551175 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:29:31.551052 2545 scope.go:117] "RemoveContainer" containerID="c3dbcb1daf00e4a492c7ccd951c13b5738127d55e9026ef9d9d4248af7812e9a" Jul 07 22:29:31.560943 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:29:31.560924 2545 scope.go:117] "RemoveContainer" containerID="c3dbcb1daf00e4a492c7ccd951c13b5738127d55e9026ef9d9d4248af7812e9a" Jul 07 22:29:31.561292 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:29:31.561266 2545 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c3dbcb1daf00e4a492c7ccd951c13b5738127d55e9026ef9d9d4248af7812e9a\": container with ID starting with c3dbcb1daf00e4a492c7ccd951c13b5738127d55e9026ef9d9d4248af7812e9a not found: ID does not exist" containerID="c3dbcb1daf00e4a492c7ccd951c13b5738127d55e9026ef9d9d4248af7812e9a" Jul 07 22:29:31.561360 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:29:31.561295 2545 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c3dbcb1daf00e4a492c7ccd951c13b5738127d55e9026ef9d9d4248af7812e9a"} err="failed to get container status \"c3dbcb1daf00e4a492c7ccd951c13b5738127d55e9026ef9d9d4248af7812e9a\": rpc error: code = NotFound desc = could not find container \"c3dbcb1daf00e4a492c7ccd951c13b5738127d55e9026ef9d9d4248af7812e9a\": container with ID starting with c3dbcb1daf00e4a492c7ccd951c13b5738127d55e9026ef9d9d4248af7812e9a not found: ID does not exist" Jul 07 22:29:31.571974 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:29:31.571950 2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-86400e9a-openshift-default-75d7ddfd65-99b9r"] Jul 07 22:29:31.582725 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:29:31.578334 2545 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-86400e9a-openshift-default-75d7ddfd65-99b9r"] Jul 07 22:29:32.311504 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:29:32.311469 2545 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1257de19-2b28-4db0-9eca-f5f45eb904b1" path="/var/lib/kubelet/pods/1257de19-2b28-4db0-9eca-f5f45eb904b1/volumes" Jul 07 22:31:22.668140 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:31:22.668101 2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-ac5ac682-openshift-default-8b4dc599-mtfnp"] Jul 07 22:31:22.668738 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:31:22.668668 2545 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1257de19-2b28-4db0-9eca-f5f45eb904b1" containerName="istio-proxy" Jul 07 22:31:22.668738 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:31:22.668689 2545 state_mem.go:107] "Deleted CPUSet assignment" podUID="1257de19-2b28-4db0-9eca-f5f45eb904b1" containerName="istio-proxy" Jul 07 22:31:22.668866 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:31:22.668786 2545 memory_manager.go:368] "RemoveStaleState removing state" podUID="1257de19-2b28-4db0-9eca-f5f45eb904b1" containerName="istio-proxy" Jul 07 22:31:22.671955 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:31:22.671932 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ac5ac682-openshift-default-8b4dc599-mtfnp" Jul 07 22:31:22.674371 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:31:22.674346 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 07 22:31:22.674501 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:31:22.674346 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-ac5ac682-openshift-default-dockercfg-v5xpf\"" Jul 07 22:31:22.682691 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:31:22.682668 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-ac5ac682-openshift-default-8b4dc599-mtfnp"] Jul 07 22:31:22.721557 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:31:22.721527 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aad75251-208d-4bed-b41b-82fe01aa46f8-istiod-ca-cert\") pod \"e2e-mt-ac5ac682-openshift-default-8b4dc599-mtfnp\" (UID: \"aad75251-208d-4bed-b41b-82fe01aa46f8\") " pod="openshift-ingress/e2e-mt-ac5ac682-openshift-default-8b4dc599-mtfnp" Jul 07 22:31:22.721687 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:31:22.721568 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aad75251-208d-4bed-b41b-82fe01aa46f8-workload-certs\") pod \"e2e-mt-ac5ac682-openshift-default-8b4dc599-mtfnp\" (UID: \"aad75251-208d-4bed-b41b-82fe01aa46f8\") " pod="openshift-ingress/e2e-mt-ac5ac682-openshift-default-8b4dc599-mtfnp" Jul 07 22:31:22.721687 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:31:22.721618 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aad75251-208d-4bed-b41b-82fe01aa46f8-istio-podinfo\") pod \"e2e-mt-ac5ac682-openshift-default-8b4dc599-mtfnp\" (UID: \"aad75251-208d-4bed-b41b-82fe01aa46f8\") " pod="openshift-ingress/e2e-mt-ac5ac682-openshift-default-8b4dc599-mtfnp" Jul 07 22:31:22.721687 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:31:22.721649 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nctqt\" (UniqueName: \"kubernetes.io/projected/aad75251-208d-4bed-b41b-82fe01aa46f8-kube-api-access-nctqt\") pod \"e2e-mt-ac5ac682-openshift-default-8b4dc599-mtfnp\" (UID: \"aad75251-208d-4bed-b41b-82fe01aa46f8\") " pod="openshift-ingress/e2e-mt-ac5ac682-openshift-default-8b4dc599-mtfnp" Jul 07 22:31:22.721687 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:31:22.721684 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aad75251-208d-4bed-b41b-82fe01aa46f8-istio-token\") pod \"e2e-mt-ac5ac682-openshift-default-8b4dc599-mtfnp\" (UID: \"aad75251-208d-4bed-b41b-82fe01aa46f8\") " pod="openshift-ingress/e2e-mt-ac5ac682-openshift-default-8b4dc599-mtfnp" Jul 07 22:31:22.721820 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:31:22.721721 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aad75251-208d-4bed-b41b-82fe01aa46f8-workload-socket\") pod \"e2e-mt-ac5ac682-openshift-default-8b4dc599-mtfnp\" (UID: \"aad75251-208d-4bed-b41b-82fe01aa46f8\") " pod="openshift-ingress/e2e-mt-ac5ac682-openshift-default-8b4dc599-mtfnp" Jul 07 22:31:22.721820 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:31:22.721800 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aad75251-208d-4bed-b41b-82fe01aa46f8-istio-envoy\") pod \"e2e-mt-ac5ac682-openshift-default-8b4dc599-mtfnp\" (UID: \"aad75251-208d-4bed-b41b-82fe01aa46f8\") " pod="openshift-ingress/e2e-mt-ac5ac682-openshift-default-8b4dc599-mtfnp" Jul 07 22:31:22.721883 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:31:22.721830 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aad75251-208d-4bed-b41b-82fe01aa46f8-istio-data\") pod \"e2e-mt-ac5ac682-openshift-default-8b4dc599-mtfnp\" (UID: \"aad75251-208d-4bed-b41b-82fe01aa46f8\") " pod="openshift-ingress/e2e-mt-ac5ac682-openshift-default-8b4dc599-mtfnp" Jul 07 22:31:22.721883 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:31:22.721853 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aad75251-208d-4bed-b41b-82fe01aa46f8-credential-socket\") pod \"e2e-mt-ac5ac682-openshift-default-8b4dc599-mtfnp\" (UID: \"aad75251-208d-4bed-b41b-82fe01aa46f8\") " pod="openshift-ingress/e2e-mt-ac5ac682-openshift-default-8b4dc599-mtfnp" Jul 07 22:31:22.822955 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:31:22.822924 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aad75251-208d-4bed-b41b-82fe01aa46f8-istio-envoy\") pod \"e2e-mt-ac5ac682-openshift-default-8b4dc599-mtfnp\" (UID: \"aad75251-208d-4bed-b41b-82fe01aa46f8\") " pod="openshift-ingress/e2e-mt-ac5ac682-openshift-default-8b4dc599-mtfnp" Jul 07 22:31:22.822955 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:31:22.822957 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aad75251-208d-4bed-b41b-82fe01aa46f8-istio-data\") pod \"e2e-mt-ac5ac682-openshift-default-8b4dc599-mtfnp\" (UID: \"aad75251-208d-4bed-b41b-82fe01aa46f8\") " pod="openshift-ingress/e2e-mt-ac5ac682-openshift-default-8b4dc599-mtfnp" Jul 07 22:31:22.823253 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:31:22.822977 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aad75251-208d-4bed-b41b-82fe01aa46f8-credential-socket\") pod \"e2e-mt-ac5ac682-openshift-default-8b4dc599-mtfnp\" (UID: \"aad75251-208d-4bed-b41b-82fe01aa46f8\") " pod="openshift-ingress/e2e-mt-ac5ac682-openshift-default-8b4dc599-mtfnp" Jul 07 22:31:22.823253 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:31:22.823012 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aad75251-208d-4bed-b41b-82fe01aa46f8-istiod-ca-cert\") pod \"e2e-mt-ac5ac682-openshift-default-8b4dc599-mtfnp\" (UID: \"aad75251-208d-4bed-b41b-82fe01aa46f8\") " pod="openshift-ingress/e2e-mt-ac5ac682-openshift-default-8b4dc599-mtfnp" Jul 07 22:31:22.823253 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:31:22.823034 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aad75251-208d-4bed-b41b-82fe01aa46f8-workload-certs\") pod \"e2e-mt-ac5ac682-openshift-default-8b4dc599-mtfnp\" (UID: \"aad75251-208d-4bed-b41b-82fe01aa46f8\") " pod="openshift-ingress/e2e-mt-ac5ac682-openshift-default-8b4dc599-mtfnp" Jul 07 22:31:22.823253 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:31:22.823073 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aad75251-208d-4bed-b41b-82fe01aa46f8-istio-podinfo\") pod \"e2e-mt-ac5ac682-openshift-default-8b4dc599-mtfnp\" (UID: \"aad75251-208d-4bed-b41b-82fe01aa46f8\") " pod="openshift-ingress/e2e-mt-ac5ac682-openshift-default-8b4dc599-mtfnp" Jul 07 22:31:22.823253 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:31:22.823130 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nctqt\" (UniqueName: \"kubernetes.io/projected/aad75251-208d-4bed-b41b-82fe01aa46f8-kube-api-access-nctqt\") pod \"e2e-mt-ac5ac682-openshift-default-8b4dc599-mtfnp\" (UID: \"aad75251-208d-4bed-b41b-82fe01aa46f8\") " pod="openshift-ingress/e2e-mt-ac5ac682-openshift-default-8b4dc599-mtfnp" Jul 07 22:31:22.823253 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:31:22.823179 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aad75251-208d-4bed-b41b-82fe01aa46f8-istio-token\") pod \"e2e-mt-ac5ac682-openshift-default-8b4dc599-mtfnp\" (UID: \"aad75251-208d-4bed-b41b-82fe01aa46f8\") " pod="openshift-ingress/e2e-mt-ac5ac682-openshift-default-8b4dc599-mtfnp" Jul 07 22:31:22.823564 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:31:22.823434 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aad75251-208d-4bed-b41b-82fe01aa46f8-credential-socket\") pod \"e2e-mt-ac5ac682-openshift-default-8b4dc599-mtfnp\" (UID: \"aad75251-208d-4bed-b41b-82fe01aa46f8\") " pod="openshift-ingress/e2e-mt-ac5ac682-openshift-default-8b4dc599-mtfnp" Jul 07 22:31:22.823564 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:31:22.823537 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aad75251-208d-4bed-b41b-82fe01aa46f8-workload-certs\") pod \"e2e-mt-ac5ac682-openshift-default-8b4dc599-mtfnp\" (UID: \"aad75251-208d-4bed-b41b-82fe01aa46f8\") " pod="openshift-ingress/e2e-mt-ac5ac682-openshift-default-8b4dc599-mtfnp" Jul 07 22:31:22.823666 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:31:22.823557 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aad75251-208d-4bed-b41b-82fe01aa46f8-workload-socket\") pod \"e2e-mt-ac5ac682-openshift-default-8b4dc599-mtfnp\" (UID: \"aad75251-208d-4bed-b41b-82fe01aa46f8\") " pod="openshift-ingress/e2e-mt-ac5ac682-openshift-default-8b4dc599-mtfnp" Jul 07 22:31:22.823777 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:31:22.823744 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aad75251-208d-4bed-b41b-82fe01aa46f8-istio-data\") pod \"e2e-mt-ac5ac682-openshift-default-8b4dc599-mtfnp\" (UID: \"aad75251-208d-4bed-b41b-82fe01aa46f8\") " pod="openshift-ingress/e2e-mt-ac5ac682-openshift-default-8b4dc599-mtfnp" Jul 07 22:31:22.823901 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:31:22.823783 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aad75251-208d-4bed-b41b-82fe01aa46f8-istiod-ca-cert\") pod \"e2e-mt-ac5ac682-openshift-default-8b4dc599-mtfnp\" (UID: \"aad75251-208d-4bed-b41b-82fe01aa46f8\") " pod="openshift-ingress/e2e-mt-ac5ac682-openshift-default-8b4dc599-mtfnp" Jul 07 22:31:22.823901 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:31:22.823793 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aad75251-208d-4bed-b41b-82fe01aa46f8-workload-socket\") pod \"e2e-mt-ac5ac682-openshift-default-8b4dc599-mtfnp\" (UID: \"aad75251-208d-4bed-b41b-82fe01aa46f8\") " pod="openshift-ingress/e2e-mt-ac5ac682-openshift-default-8b4dc599-mtfnp" Jul 07 22:31:22.825423 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:31:22.825405 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aad75251-208d-4bed-b41b-82fe01aa46f8-istio-envoy\") pod \"e2e-mt-ac5ac682-openshift-default-8b4dc599-mtfnp\" (UID: \"aad75251-208d-4bed-b41b-82fe01aa46f8\") " pod="openshift-ingress/e2e-mt-ac5ac682-openshift-default-8b4dc599-mtfnp" Jul 07 22:31:22.825686 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:31:22.825669 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aad75251-208d-4bed-b41b-82fe01aa46f8-istio-podinfo\") pod \"e2e-mt-ac5ac682-openshift-default-8b4dc599-mtfnp\" (UID: \"aad75251-208d-4bed-b41b-82fe01aa46f8\") " pod="openshift-ingress/e2e-mt-ac5ac682-openshift-default-8b4dc599-mtfnp" Jul 07 22:31:22.831838 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:31:22.831814 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aad75251-208d-4bed-b41b-82fe01aa46f8-istio-token\") pod \"e2e-mt-ac5ac682-openshift-default-8b4dc599-mtfnp\" (UID: \"aad75251-208d-4bed-b41b-82fe01aa46f8\") " pod="openshift-ingress/e2e-mt-ac5ac682-openshift-default-8b4dc599-mtfnp" Jul 07 22:31:22.832213 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:31:22.832191 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nctqt\" (UniqueName: \"kubernetes.io/projected/aad75251-208d-4bed-b41b-82fe01aa46f8-kube-api-access-nctqt\") pod \"e2e-mt-ac5ac682-openshift-default-8b4dc599-mtfnp\" (UID: \"aad75251-208d-4bed-b41b-82fe01aa46f8\") " pod="openshift-ingress/e2e-mt-ac5ac682-openshift-default-8b4dc599-mtfnp" Jul 07 22:31:22.983389 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:31:22.983353 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ac5ac682-openshift-default-8b4dc599-mtfnp" Jul 07 22:31:23.122627 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:31:23.122592 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-ac5ac682-openshift-default-8b4dc599-mtfnp"] Jul 07 22:31:23.125931 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:31:23.125900 2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaad75251_208d_4bed_b41b_82fe01aa46f8.slice/crio-46a11607e921db0c1e4cf7172e1255308651b5c9ca463dc30cce1d828923e174 WatchSource:0}: Error finding container 46a11607e921db0c1e4cf7172e1255308651b5c9ca463dc30cce1d828923e174: Status 404 returned error can't find the container with id 46a11607e921db0c1e4cf7172e1255308651b5c9ca463dc30cce1d828923e174 Jul 07 22:31:23.128054 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:31:23.128019 2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 22:31:23.128184 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:31:23.128112 2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 22:31:23.128184 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:31:23.128159 2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 22:31:24.080218 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:31:24.080183 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ac5ac682-openshift-default-8b4dc599-mtfnp" event={"ID":"aad75251-208d-4bed-b41b-82fe01aa46f8","Type":"ContainerStarted","Data":"46b35b7a4a27010fd702ae9692c11ef85f60942ef2616f930b095ea995bf78fc"} Jul 07 22:31:24.080618 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:31:24.080235 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ac5ac682-openshift-default-8b4dc599-mtfnp" event={"ID":"aad75251-208d-4bed-b41b-82fe01aa46f8","Type":"ContainerStarted","Data":"46a11607e921db0c1e4cf7172e1255308651b5c9ca463dc30cce1d828923e174"} Jul 07 22:31:24.098856 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:31:24.098802 2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-ac5ac682-openshift-default-8b4dc599-mtfnp" podStartSLOduration=2.098784119 podStartE2EDuration="2.098784119s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 22:31:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:31:24.098381708 +0000 UTC m=+1666.412604723" watchObservedRunningTime="2026-07-07 22:31:24.098784119 +0000 UTC m=+1666.413007138" Jul 07 22:31:24.984031 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:31:24.983997 2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-ac5ac682-openshift-default-8b4dc599-mtfnp" Jul 07 22:31:24.988668 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:31:24.988646 2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-ac5ac682-openshift-default-8b4dc599-mtfnp" Jul 07 22:31:25.085383 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:31:25.085341 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-ac5ac682-openshift-default-8b4dc599-mtfnp" Jul 07 22:31:25.086172 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:31:25.086153 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-ac5ac682-openshift-default-8b4dc599-mtfnp" Jul 07 22:31:37.146964 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:31:37.146863 2545 generic.go:356] "Generic (PLEG): container finished" podID="aad75251-208d-4bed-b41b-82fe01aa46f8" containerID="46b35b7a4a27010fd702ae9692c11ef85f60942ef2616f930b095ea995bf78fc" exitCode=0 Jul 07 22:31:37.146964 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:31:37.146935 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ac5ac682-openshift-default-8b4dc599-mtfnp" event={"ID":"aad75251-208d-4bed-b41b-82fe01aa46f8","Type":"ContainerDied","Data":"46b35b7a4a27010fd702ae9692c11ef85f60942ef2616f930b095ea995bf78fc"} Jul 07 22:31:37.147452 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:31:37.147357 2545 scope.go:117] "RemoveContainer" containerID="46b35b7a4a27010fd702ae9692c11ef85f60942ef2616f930b095ea995bf78fc" Jul 07 22:31:37.148360 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:31:37.148328 2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 22:31:37.148445 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:31:37.148398 2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 22:31:37.148485 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:31:37.148454 2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 22:31:38.157803 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:31:38.157751 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ac5ac682-openshift-default-8b4dc599-mtfnp" event={"ID":"aad75251-208d-4bed-b41b-82fe01aa46f8","Type":"ContainerStarted","Data":"f72d784ee0e09494e1497dcd05cd6160df4e8489b71cc741de10bc4f3deb635b"} Jul 07 22:31:38.984482 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:31:38.984439 2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-ac5ac682-openshift-default-8b4dc599-mtfnp" Jul 07 22:31:38.985505 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:31:38.985479 2545 patch_prober.go:28] interesting pod/e2e-mt-ac5ac682-openshift-default-8b4dc599-mtfnp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body= Jul 07 22:31:38.985566 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:31:38.985544 2545 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ac5ac682-openshift-default-8b4dc599-mtfnp" podUID="aad75251-208d-4bed-b41b-82fe01aa46f8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" Jul 07 22:31:39.983891 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:31:39.983850 2545 patch_prober.go:28] interesting pod/e2e-mt-ac5ac682-openshift-default-8b4dc599-mtfnp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body= Jul 07 22:31:39.984300 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:31:39.983916 2545 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ac5ac682-openshift-default-8b4dc599-mtfnp" podUID="aad75251-208d-4bed-b41b-82fe01aa46f8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" Jul 07 22:31:40.086308 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:31:40.086254 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-ac5ac682-openshift-default-8b4dc599-mtfnp" Jul 07 22:31:40.984661 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:31:40.984616 2545 patch_prober.go:28] interesting pod/e2e-mt-ac5ac682-openshift-default-8b4dc599-mtfnp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body= Jul 07 22:31:40.985145 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:31:40.984680 2545 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ac5ac682-openshift-default-8b4dc599-mtfnp" podUID="aad75251-208d-4bed-b41b-82fe01aa46f8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" Jul 07 22:31:41.984789 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:31:41.984752 2545 patch_prober.go:28] interesting pod/e2e-mt-ac5ac682-openshift-default-8b4dc599-mtfnp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body= Jul 07 22:31:41.985217 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:31:41.984811 2545 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ac5ac682-openshift-default-8b4dc599-mtfnp" podUID="aad75251-208d-4bed-b41b-82fe01aa46f8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" Jul 07 22:31:42.987792 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:31:42.987753 2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-ac5ac682-openshift-default-8b4dc599-mtfnp" Jul 07 22:31:42.988715 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:31:42.988693 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-ac5ac682-openshift-default-8b4dc599-mtfnp" Jul 07 22:31:48.986741 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:31:48.986700 2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-ac5ac682-openshift-default-8b4dc599-mtfnp"] Jul 07 22:31:48.987158 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:31:48.986932 2545 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-ac5ac682-openshift-default-8b4dc599-mtfnp" podUID="aad75251-208d-4bed-b41b-82fe01aa46f8" containerName="istio-proxy" containerID="cri-o://f72d784ee0e09494e1497dcd05cd6160df4e8489b71cc741de10bc4f3deb635b" gracePeriod=30 Jul 07 22:31:54.238392 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:31:54.238366 2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ac5ac682-openshift-default-8b4dc599-mtfnp" Jul 07 22:31:54.242265 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:31:54.242237 2545 generic.go:356] "Generic (PLEG): container finished" podID="aad75251-208d-4bed-b41b-82fe01aa46f8" containerID="f72d784ee0e09494e1497dcd05cd6160df4e8489b71cc741de10bc4f3deb635b" exitCode=0 Jul 07 22:31:54.242389 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:31:54.242324 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ac5ac682-openshift-default-8b4dc599-mtfnp" event={"ID":"aad75251-208d-4bed-b41b-82fe01aa46f8","Type":"ContainerDied","Data":"f72d784ee0e09494e1497dcd05cd6160df4e8489b71cc741de10bc4f3deb635b"} Jul 07 22:31:54.242389 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:31:54.242367 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ac5ac682-openshift-default-8b4dc599-mtfnp" event={"ID":"aad75251-208d-4bed-b41b-82fe01aa46f8","Type":"ContainerDied","Data":"46a11607e921db0c1e4cf7172e1255308651b5c9ca463dc30cce1d828923e174"} Jul 07 22:31:54.242389 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:31:54.242327 2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ac5ac682-openshift-default-8b4dc599-mtfnp" Jul 07 22:31:54.242492 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:31:54.242390 2545 scope.go:117] "RemoveContainer" containerID="f72d784ee0e09494e1497dcd05cd6160df4e8489b71cc741de10bc4f3deb635b" Jul 07 22:31:54.250582 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:31:54.250563 2545 scope.go:117] "RemoveContainer" containerID="46b35b7a4a27010fd702ae9692c11ef85f60942ef2616f930b095ea995bf78fc" Jul 07 22:31:54.258724 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:31:54.258699 2545 scope.go:117] "RemoveContainer" containerID="f72d784ee0e09494e1497dcd05cd6160df4e8489b71cc741de10bc4f3deb635b" Jul 07 22:31:54.258993 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:31:54.258975 2545 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f72d784ee0e09494e1497dcd05cd6160df4e8489b71cc741de10bc4f3deb635b\": container with ID starting with f72d784ee0e09494e1497dcd05cd6160df4e8489b71cc741de10bc4f3deb635b not found: ID does not exist" containerID="f72d784ee0e09494e1497dcd05cd6160df4e8489b71cc741de10bc4f3deb635b" Jul 07 22:31:54.259093 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:31:54.259005 2545 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f72d784ee0e09494e1497dcd05cd6160df4e8489b71cc741de10bc4f3deb635b"} err="failed to get container status \"f72d784ee0e09494e1497dcd05cd6160df4e8489b71cc741de10bc4f3deb635b\": rpc error: code = NotFound desc = could not find container \"f72d784ee0e09494e1497dcd05cd6160df4e8489b71cc741de10bc4f3deb635b\": container with ID starting with f72d784ee0e09494e1497dcd05cd6160df4e8489b71cc741de10bc4f3deb635b not found: ID does not exist" Jul 07 22:31:54.259093 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:31:54.259030 2545 scope.go:117] "RemoveContainer" containerID="46b35b7a4a27010fd702ae9692c11ef85f60942ef2616f930b095ea995bf78fc" Jul 07 22:31:54.259372 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:31:54.259343 2545 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"46b35b7a4a27010fd702ae9692c11ef85f60942ef2616f930b095ea995bf78fc\": container with ID starting with 46b35b7a4a27010fd702ae9692c11ef85f60942ef2616f930b095ea995bf78fc not found: ID does not exist" containerID="46b35b7a4a27010fd702ae9692c11ef85f60942ef2616f930b095ea995bf78fc" Jul 07 22:31:54.259508 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:31:54.259382 2545 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"46b35b7a4a27010fd702ae9692c11ef85f60942ef2616f930b095ea995bf78fc"} err="failed to get container status \"46b35b7a4a27010fd702ae9692c11ef85f60942ef2616f930b095ea995bf78fc\": rpc error: code = NotFound desc = could not find container \"46b35b7a4a27010fd702ae9692c11ef85f60942ef2616f930b095ea995bf78fc\": container with ID starting with 46b35b7a4a27010fd702ae9692c11ef85f60942ef2616f930b095ea995bf78fc not found: ID does not exist" Jul 07 22:31:54.319925 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:31:54.319896 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aad75251-208d-4bed-b41b-82fe01aa46f8-workload-certs\") pod \"aad75251-208d-4bed-b41b-82fe01aa46f8\" (UID: \"aad75251-208d-4bed-b41b-82fe01aa46f8\") " Jul 07 22:31:54.319925 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:31:54.319927 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aad75251-208d-4bed-b41b-82fe01aa46f8-istio-data\") pod \"aad75251-208d-4bed-b41b-82fe01aa46f8\" (UID: \"aad75251-208d-4bed-b41b-82fe01aa46f8\") " Jul 07 22:31:54.320270 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:31:54.319952 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aad75251-208d-4bed-b41b-82fe01aa46f8-workload-socket\") pod \"aad75251-208d-4bed-b41b-82fe01aa46f8\" (UID: \"aad75251-208d-4bed-b41b-82fe01aa46f8\") " Jul 07 22:31:54.320270 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:31:54.319998 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aad75251-208d-4bed-b41b-82fe01aa46f8-istio-envoy\") pod \"aad75251-208d-4bed-b41b-82fe01aa46f8\" (UID: \"aad75251-208d-4bed-b41b-82fe01aa46f8\") " Jul 07 22:31:54.320270 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:31:54.320027 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aad75251-208d-4bed-b41b-82fe01aa46f8-istio-token\") pod \"aad75251-208d-4bed-b41b-82fe01aa46f8\" (UID: \"aad75251-208d-4bed-b41b-82fe01aa46f8\") " Jul 07 22:31:54.320270 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:31:54.320051 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aad75251-208d-4bed-b41b-82fe01aa46f8-credential-socket\") pod \"aad75251-208d-4bed-b41b-82fe01aa46f8\" (UID: \"aad75251-208d-4bed-b41b-82fe01aa46f8\") " Jul 07 22:31:54.320270 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:31:54.320145 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nctqt\" (UniqueName: \"kubernetes.io/projected/aad75251-208d-4bed-b41b-82fe01aa46f8-kube-api-access-nctqt\") pod \"aad75251-208d-4bed-b41b-82fe01aa46f8\" (UID: \"aad75251-208d-4bed-b41b-82fe01aa46f8\") " Jul 07 22:31:54.320270 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:31:54.320196 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aad75251-208d-4bed-b41b-82fe01aa46f8-istiod-ca-cert\") pod \"aad75251-208d-4bed-b41b-82fe01aa46f8\" (UID: \"aad75251-208d-4bed-b41b-82fe01aa46f8\") " Jul 07 22:31:54.320591 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:31:54.320274 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aad75251-208d-4bed-b41b-82fe01aa46f8-istio-podinfo\") pod \"aad75251-208d-4bed-b41b-82fe01aa46f8\" (UID: \"aad75251-208d-4bed-b41b-82fe01aa46f8\") " Jul 07 22:31:54.320591 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:31:54.320276 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aad75251-208d-4bed-b41b-82fe01aa46f8-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "aad75251-208d-4bed-b41b-82fe01aa46f8" (UID: "aad75251-208d-4bed-b41b-82fe01aa46f8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 22:31:54.320591 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:31:54.320314 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aad75251-208d-4bed-b41b-82fe01aa46f8-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "aad75251-208d-4bed-b41b-82fe01aa46f8" (UID: "aad75251-208d-4bed-b41b-82fe01aa46f8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 22:31:54.320591 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:31:54.320536 2545 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aad75251-208d-4bed-b41b-82fe01aa46f8-workload-certs\") on node \"ip-10-0-131-55.ec2.internal\" DevicePath \"\"" Jul 07 22:31:54.320591 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:31:54.320549 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aad75251-208d-4bed-b41b-82fe01aa46f8-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "aad75251-208d-4bed-b41b-82fe01aa46f8" (UID: "aad75251-208d-4bed-b41b-82fe01aa46f8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 22:31:54.320591 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:31:54.320557 2545 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aad75251-208d-4bed-b41b-82fe01aa46f8-workload-socket\") on node \"ip-10-0-131-55.ec2.internal\" DevicePath \"\"" Jul 07 22:31:54.320591 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:31:54.320591 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aad75251-208d-4bed-b41b-82fe01aa46f8-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "aad75251-208d-4bed-b41b-82fe01aa46f8" (UID: "aad75251-208d-4bed-b41b-82fe01aa46f8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 22:31:54.320930 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:31:54.320904 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aad75251-208d-4bed-b41b-82fe01aa46f8-istio-data" (OuterVolumeSpecName: "istio-data") pod "aad75251-208d-4bed-b41b-82fe01aa46f8" (UID: "aad75251-208d-4bed-b41b-82fe01aa46f8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 22:31:54.322308 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:31:54.322266 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aad75251-208d-4bed-b41b-82fe01aa46f8-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "aad75251-208d-4bed-b41b-82fe01aa46f8" (UID: "aad75251-208d-4bed-b41b-82fe01aa46f8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 22:31:54.322421 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:31:54.322361 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aad75251-208d-4bed-b41b-82fe01aa46f8-kube-api-access-nctqt" (OuterVolumeSpecName: "kube-api-access-nctqt") pod "aad75251-208d-4bed-b41b-82fe01aa46f8" (UID: "aad75251-208d-4bed-b41b-82fe01aa46f8"). InnerVolumeSpecName "kube-api-access-nctqt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 22:31:54.322665 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:31:54.322636 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aad75251-208d-4bed-b41b-82fe01aa46f8-istio-token" (OuterVolumeSpecName: "istio-token") pod "aad75251-208d-4bed-b41b-82fe01aa46f8" (UID: "aad75251-208d-4bed-b41b-82fe01aa46f8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 22:31:54.322792 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:31:54.322700 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/aad75251-208d-4bed-b41b-82fe01aa46f8-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "aad75251-208d-4bed-b41b-82fe01aa46f8" (UID: "aad75251-208d-4bed-b41b-82fe01aa46f8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 22:31:54.422001 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:31:54.421969 2545 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aad75251-208d-4bed-b41b-82fe01aa46f8-istio-data\") on node \"ip-10-0-131-55.ec2.internal\" DevicePath \"\"" Jul 07 22:31:54.422001 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:31:54.421997 2545 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aad75251-208d-4bed-b41b-82fe01aa46f8-istio-envoy\") on node \"ip-10-0-131-55.ec2.internal\" DevicePath \"\"" Jul 07 22:31:54.422001 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:31:54.422005 2545 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aad75251-208d-4bed-b41b-82fe01aa46f8-istio-token\") on node \"ip-10-0-131-55.ec2.internal\" DevicePath \"\"" Jul 07 22:31:54.422259 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:31:54.422014 2545 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aad75251-208d-4bed-b41b-82fe01aa46f8-credential-socket\") on node \"ip-10-0-131-55.ec2.internal\" DevicePath \"\"" Jul 07 22:31:54.422259 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:31:54.422024 2545 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nctqt\" (UniqueName: \"kubernetes.io/projected/aad75251-208d-4bed-b41b-82fe01aa46f8-kube-api-access-nctqt\") on node \"ip-10-0-131-55.ec2.internal\" DevicePath \"\"" Jul 07 22:31:54.422259 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:31:54.422032 2545 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aad75251-208d-4bed-b41b-82fe01aa46f8-istiod-ca-cert\") on node \"ip-10-0-131-55.ec2.internal\" DevicePath \"\"" Jul 07 22:31:54.422259 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:31:54.422041 2545 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aad75251-208d-4bed-b41b-82fe01aa46f8-istio-podinfo\") on node \"ip-10-0-131-55.ec2.internal\" DevicePath \"\"" Jul 07 22:31:54.564478 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:31:54.564447 2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-ac5ac682-openshift-default-8b4dc599-mtfnp"] Jul 07 22:31:54.573313 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:31:54.569459 2545 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-ac5ac682-openshift-default-8b4dc599-mtfnp"] Jul 07 22:31:56.311272 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:31:56.311241 2545 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aad75251-208d-4bed-b41b-82fe01aa46f8" path="/var/lib/kubelet/pods/aad75251-208d-4bed-b41b-82fe01aa46f8/volumes" Jul 07 22:32:08.366833 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:32:08.366758 2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-28c37d17-openshift-default-78d7487b59-wmn9c"] Jul 07 22:32:08.367278 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:32:08.367183 2545 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aad75251-208d-4bed-b41b-82fe01aa46f8" containerName="istio-proxy" Jul 07 22:32:08.367278 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:32:08.367197 2545 state_mem.go:107] "Deleted CPUSet assignment" podUID="aad75251-208d-4bed-b41b-82fe01aa46f8" containerName="istio-proxy" Jul 07 22:32:08.367278 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:32:08.367209 2545 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aad75251-208d-4bed-b41b-82fe01aa46f8" containerName="istio-proxy" Jul 07 22:32:08.367278 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:32:08.367215 2545 state_mem.go:107] "Deleted CPUSet assignment" podUID="aad75251-208d-4bed-b41b-82fe01aa46f8" containerName="istio-proxy" Jul 07 22:32:08.367413 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:32:08.367300 2545 memory_manager.go:368] "RemoveStaleState removing state" podUID="aad75251-208d-4bed-b41b-82fe01aa46f8" containerName="istio-proxy" Jul 07 22:32:08.367413 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:32:08.367312 2545 memory_manager.go:368] "RemoveStaleState removing state" podUID="aad75251-208d-4bed-b41b-82fe01aa46f8" containerName="istio-proxy" Jul 07 22:32:08.371559 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:32:08.371539 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-28c37d17-openshift-default-78d7487b59-wmn9c" Jul 07 22:32:08.373950 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:32:08.373926 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 07 22:32:08.374073 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:32:08.373926 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-28c37d17-openshift-default-dockercfg-lcs7d\"" Jul 07 22:32:08.382824 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:32:08.382800 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-28c37d17-openshift-default-78d7487b59-wmn9c"] Jul 07 22:32:08.452782 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:32:08.452739 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1b2626d9-3efd-4940-aab7-8dac744fd28e-istio-envoy\") pod \"e2e-mt-28c37d17-openshift-default-78d7487b59-wmn9c\" (UID: \"1b2626d9-3efd-4940-aab7-8dac744fd28e\") " pod="openshift-ingress/e2e-mt-28c37d17-openshift-default-78d7487b59-wmn9c" Jul 07 22:32:08.452782 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:32:08.452781 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1b2626d9-3efd-4940-aab7-8dac744fd28e-workload-certs\") pod \"e2e-mt-28c37d17-openshift-default-78d7487b59-wmn9c\" (UID: \"1b2626d9-3efd-4940-aab7-8dac744fd28e\") " pod="openshift-ingress/e2e-mt-28c37d17-openshift-default-78d7487b59-wmn9c" Jul 07 22:32:08.453044 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:32:08.452816 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1b2626d9-3efd-4940-aab7-8dac744fd28e-istio-token\") pod \"e2e-mt-28c37d17-openshift-default-78d7487b59-wmn9c\" (UID: \"1b2626d9-3efd-4940-aab7-8dac744fd28e\") " pod="openshift-ingress/e2e-mt-28c37d17-openshift-default-78d7487b59-wmn9c" Jul 07 22:32:08.453044 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:32:08.452952 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1b2626d9-3efd-4940-aab7-8dac744fd28e-istiod-ca-cert\") pod \"e2e-mt-28c37d17-openshift-default-78d7487b59-wmn9c\" (UID: \"1b2626d9-3efd-4940-aab7-8dac744fd28e\") " pod="openshift-ingress/e2e-mt-28c37d17-openshift-default-78d7487b59-wmn9c" Jul 07 22:32:08.453044 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:32:08.452996 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1b2626d9-3efd-4940-aab7-8dac744fd28e-istio-data\") pod \"e2e-mt-28c37d17-openshift-default-78d7487b59-wmn9c\" (UID: \"1b2626d9-3efd-4940-aab7-8dac744fd28e\") " pod="openshift-ingress/e2e-mt-28c37d17-openshift-default-78d7487b59-wmn9c" Jul 07 22:32:08.453044 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:32:08.453027 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1b2626d9-3efd-4940-aab7-8dac744fd28e-istio-podinfo\") pod \"e2e-mt-28c37d17-openshift-default-78d7487b59-wmn9c\" (UID: \"1b2626d9-3efd-4940-aab7-8dac744fd28e\") " pod="openshift-ingress/e2e-mt-28c37d17-openshift-default-78d7487b59-wmn9c" Jul 07 22:32:08.453311 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:32:08.453102 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b2626d9-3efd-4940-aab7-8dac744fd28e-workload-socket\") pod \"e2e-mt-28c37d17-openshift-default-78d7487b59-wmn9c\" (UID: \"1b2626d9-3efd-4940-aab7-8dac744fd28e\") " pod="openshift-ingress/e2e-mt-28c37d17-openshift-default-78d7487b59-wmn9c" Jul 07 22:32:08.453311 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:32:08.453128 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b2626d9-3efd-4940-aab7-8dac744fd28e-credential-socket\") pod \"e2e-mt-28c37d17-openshift-default-78d7487b59-wmn9c\" (UID: \"1b2626d9-3efd-4940-aab7-8dac744fd28e\") " pod="openshift-ingress/e2e-mt-28c37d17-openshift-default-78d7487b59-wmn9c" Jul 07 22:32:08.453311 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:32:08.453158 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9dfsr\" (UniqueName: \"kubernetes.io/projected/1b2626d9-3efd-4940-aab7-8dac744fd28e-kube-api-access-9dfsr\") pod \"e2e-mt-28c37d17-openshift-default-78d7487b59-wmn9c\" (UID: \"1b2626d9-3efd-4940-aab7-8dac744fd28e\") " pod="openshift-ingress/e2e-mt-28c37d17-openshift-default-78d7487b59-wmn9c" Jul 07 22:32:08.554359 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:32:08.554316 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1b2626d9-3efd-4940-aab7-8dac744fd28e-istio-envoy\") pod \"e2e-mt-28c37d17-openshift-default-78d7487b59-wmn9c\" (UID: \"1b2626d9-3efd-4940-aab7-8dac744fd28e\") " pod="openshift-ingress/e2e-mt-28c37d17-openshift-default-78d7487b59-wmn9c" Jul 07 22:32:08.554359 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:32:08.554361 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1b2626d9-3efd-4940-aab7-8dac744fd28e-workload-certs\") pod \"e2e-mt-28c37d17-openshift-default-78d7487b59-wmn9c\" (UID: \"1b2626d9-3efd-4940-aab7-8dac744fd28e\") " pod="openshift-ingress/e2e-mt-28c37d17-openshift-default-78d7487b59-wmn9c" Jul 07 22:32:08.554610 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:32:08.554387 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1b2626d9-3efd-4940-aab7-8dac744fd28e-istio-token\") pod \"e2e-mt-28c37d17-openshift-default-78d7487b59-wmn9c\" (UID: \"1b2626d9-3efd-4940-aab7-8dac744fd28e\") " pod="openshift-ingress/e2e-mt-28c37d17-openshift-default-78d7487b59-wmn9c" Jul 07 22:32:08.554610 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:32:08.554447 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1b2626d9-3efd-4940-aab7-8dac744fd28e-istiod-ca-cert\") pod \"e2e-mt-28c37d17-openshift-default-78d7487b59-wmn9c\" (UID: \"1b2626d9-3efd-4940-aab7-8dac744fd28e\") " pod="openshift-ingress/e2e-mt-28c37d17-openshift-default-78d7487b59-wmn9c" Jul 07 22:32:08.554610 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:32:08.554467 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1b2626d9-3efd-4940-aab7-8dac744fd28e-istio-data\") pod \"e2e-mt-28c37d17-openshift-default-78d7487b59-wmn9c\" (UID: \"1b2626d9-3efd-4940-aab7-8dac744fd28e\") " pod="openshift-ingress/e2e-mt-28c37d17-openshift-default-78d7487b59-wmn9c" Jul 07 22:32:08.554610 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:32:08.554492 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1b2626d9-3efd-4940-aab7-8dac744fd28e-istio-podinfo\") pod \"e2e-mt-28c37d17-openshift-default-78d7487b59-wmn9c\" (UID: \"1b2626d9-3efd-4940-aab7-8dac744fd28e\") " pod="openshift-ingress/e2e-mt-28c37d17-openshift-default-78d7487b59-wmn9c" Jul 07 22:32:08.554610 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:32:08.554536 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b2626d9-3efd-4940-aab7-8dac744fd28e-workload-socket\") pod \"e2e-mt-28c37d17-openshift-default-78d7487b59-wmn9c\" (UID: \"1b2626d9-3efd-4940-aab7-8dac744fd28e\") " pod="openshift-ingress/e2e-mt-28c37d17-openshift-default-78d7487b59-wmn9c" Jul 07 22:32:08.554610 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:32:08.554552 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b2626d9-3efd-4940-aab7-8dac744fd28e-credential-socket\") pod \"e2e-mt-28c37d17-openshift-default-78d7487b59-wmn9c\" (UID: \"1b2626d9-3efd-4940-aab7-8dac744fd28e\") " pod="openshift-ingress/e2e-mt-28c37d17-openshift-default-78d7487b59-wmn9c" Jul 07 22:32:08.554610 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:32:08.554572 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9dfsr\" (UniqueName: \"kubernetes.io/projected/1b2626d9-3efd-4940-aab7-8dac744fd28e-kube-api-access-9dfsr\") pod \"e2e-mt-28c37d17-openshift-default-78d7487b59-wmn9c\" (UID: \"1b2626d9-3efd-4940-aab7-8dac744fd28e\") " pod="openshift-ingress/e2e-mt-28c37d17-openshift-default-78d7487b59-wmn9c" Jul 07 22:32:08.554961 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:32:08.554842 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1b2626d9-3efd-4940-aab7-8dac744fd28e-istio-data\") pod \"e2e-mt-28c37d17-openshift-default-78d7487b59-wmn9c\" (UID: \"1b2626d9-3efd-4940-aab7-8dac744fd28e\") " pod="openshift-ingress/e2e-mt-28c37d17-openshift-default-78d7487b59-wmn9c" Jul 07 22:32:08.554961 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:32:08.554913 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b2626d9-3efd-4940-aab7-8dac744fd28e-credential-socket\") pod \"e2e-mt-28c37d17-openshift-default-78d7487b59-wmn9c\" (UID: \"1b2626d9-3efd-4940-aab7-8dac744fd28e\") " pod="openshift-ingress/e2e-mt-28c37d17-openshift-default-78d7487b59-wmn9c" Jul 07 22:32:08.555076 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:32:08.554972 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1b2626d9-3efd-4940-aab7-8dac744fd28e-workload-certs\") pod \"e2e-mt-28c37d17-openshift-default-78d7487b59-wmn9c\" (UID: \"1b2626d9-3efd-4940-aab7-8dac744fd28e\") " pod="openshift-ingress/e2e-mt-28c37d17-openshift-default-78d7487b59-wmn9c" Jul 07 22:32:08.555076 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:32:08.554980 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b2626d9-3efd-4940-aab7-8dac744fd28e-workload-socket\") pod \"e2e-mt-28c37d17-openshift-default-78d7487b59-wmn9c\" (UID: \"1b2626d9-3efd-4940-aab7-8dac744fd28e\") " pod="openshift-ingress/e2e-mt-28c37d17-openshift-default-78d7487b59-wmn9c" Jul 07 22:32:08.555547 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:32:08.555521 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1b2626d9-3efd-4940-aab7-8dac744fd28e-istiod-ca-cert\") pod \"e2e-mt-28c37d17-openshift-default-78d7487b59-wmn9c\" (UID: \"1b2626d9-3efd-4940-aab7-8dac744fd28e\") " pod="openshift-ingress/e2e-mt-28c37d17-openshift-default-78d7487b59-wmn9c" Jul 07 22:32:08.556823 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:32:08.556796 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1b2626d9-3efd-4940-aab7-8dac744fd28e-istio-envoy\") pod \"e2e-mt-28c37d17-openshift-default-78d7487b59-wmn9c\" (UID: \"1b2626d9-3efd-4940-aab7-8dac744fd28e\") " pod="openshift-ingress/e2e-mt-28c37d17-openshift-default-78d7487b59-wmn9c" Jul 07 22:32:08.557030 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:32:08.557011 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1b2626d9-3efd-4940-aab7-8dac744fd28e-istio-podinfo\") pod \"e2e-mt-28c37d17-openshift-default-78d7487b59-wmn9c\" (UID: \"1b2626d9-3efd-4940-aab7-8dac744fd28e\") " pod="openshift-ingress/e2e-mt-28c37d17-openshift-default-78d7487b59-wmn9c" Jul 07 22:32:08.567398 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:32:08.567367 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1b2626d9-3efd-4940-aab7-8dac744fd28e-istio-token\") pod \"e2e-mt-28c37d17-openshift-default-78d7487b59-wmn9c\" (UID: \"1b2626d9-3efd-4940-aab7-8dac744fd28e\") " pod="openshift-ingress/e2e-mt-28c37d17-openshift-default-78d7487b59-wmn9c" Jul 07 22:32:08.567478 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:32:08.567421 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9dfsr\" (UniqueName: \"kubernetes.io/projected/1b2626d9-3efd-4940-aab7-8dac744fd28e-kube-api-access-9dfsr\") pod \"e2e-mt-28c37d17-openshift-default-78d7487b59-wmn9c\" (UID: \"1b2626d9-3efd-4940-aab7-8dac744fd28e\") " pod="openshift-ingress/e2e-mt-28c37d17-openshift-default-78d7487b59-wmn9c" Jul 07 22:32:08.683438 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:32:08.683343 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-28c37d17-openshift-default-78d7487b59-wmn9c" Jul 07 22:32:08.819792 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:32:08.819761 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-28c37d17-openshift-default-78d7487b59-wmn9c"] Jul 07 22:32:08.821486 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:32:08.821454 2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b2626d9_3efd_4940_aab7_8dac744fd28e.slice/crio-9458b592a0f1b2e300d56120c41067840366c75f5c2149ef915c4d2816b445e3 WatchSource:0}: Error finding container 9458b592a0f1b2e300d56120c41067840366c75f5c2149ef915c4d2816b445e3: Status 404 returned error can't find the container with id 9458b592a0f1b2e300d56120c41067840366c75f5c2149ef915c4d2816b445e3 Jul 07 22:32:08.824192 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:32:08.824161 2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 22:32:08.824260 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:32:08.824225 2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 22:32:08.824260 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:32:08.824252 2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 22:32:09.324189 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:32:09.324135 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-28c37d17-openshift-default-78d7487b59-wmn9c" event={"ID":"1b2626d9-3efd-4940-aab7-8dac744fd28e","Type":"ContainerStarted","Data":"927efd019b370be0e83a69d3b970e0fbf6259b164d1b17d5ba480e736bd5030b"} Jul 07 22:32:09.324394 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:32:09.324218 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-28c37d17-openshift-default-78d7487b59-wmn9c" event={"ID":"1b2626d9-3efd-4940-aab7-8dac744fd28e","Type":"ContainerStarted","Data":"9458b592a0f1b2e300d56120c41067840366c75f5c2149ef915c4d2816b445e3"} Jul 07 22:32:09.348378 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:32:09.348330 2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-28c37d17-openshift-default-78d7487b59-wmn9c" podStartSLOduration=1.348315302 podStartE2EDuration="1.348315302s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 22:32:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:32:09.346586672 +0000 UTC m=+1711.660809688" watchObservedRunningTime="2026-07-07 22:32:09.348315302 +0000 UTC m=+1711.662538385" Jul 07 22:32:09.684457 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:32:09.684363 2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-28c37d17-openshift-default-78d7487b59-wmn9c" Jul 07 22:32:09.689146 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:32:09.689122 2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-28c37d17-openshift-default-78d7487b59-wmn9c" Jul 07 22:32:10.329101 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:32:10.329059 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-28c37d17-openshift-default-78d7487b59-wmn9c" Jul 07 22:32:10.330152 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:32:10.330132 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-28c37d17-openshift-default-78d7487b59-wmn9c" Jul 07 22:32:20.609237 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:32:20.609197 2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-28c37d17/e2e-lifecycle-model-28c37d17-kserve-66d4f54799-986f5"] Jul 07 22:32:20.612814 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:32:20.612795 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-28c37d17/e2e-lifecycle-model-28c37d17-kserve-66d4f54799-986f5" Jul 07 22:32:20.616216 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:32:20.616198 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-28c37d17\"/\"openshift-service-ca.crt\"" Jul 07 22:32:20.616385 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:32:20.616368 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-28c37d17\"/\"default-dockercfg-gdzmk\"" Jul 07 22:32:20.616468 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:32:20.616400 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-28c37d17\"/\"e2e-lifecycle-model-28c37d17-kserve-self-signed-certs\"" Jul 07 22:32:20.616533 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:32:20.616461 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-28c37d17\"/\"kube-root-ca.crt\"" Jul 07 22:32:20.620825 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:32:20.620605 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-28c37d17/e2e-lifecycle-model-28c37d17-kserve-66d4f54799-986f5"] Jul 07 22:32:20.780780 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:32:20.780742 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9acff786-c441-47d1-ba3c-fa02c2f27dff-home\") pod \"e2e-lifecycle-model-28c37d17-kserve-66d4f54799-986f5\" (UID: \"9acff786-c441-47d1-ba3c-fa02c2f27dff\") " pod="ai-tenant-e2e-mt-28c37d17/e2e-lifecycle-model-28c37d17-kserve-66d4f54799-986f5" Jul 07 22:32:20.780956 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:32:20.780790 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9acff786-c441-47d1-ba3c-fa02c2f27dff-tls-certs\") pod \"e2e-lifecycle-model-28c37d17-kserve-66d4f54799-986f5\" (UID: \"9acff786-c441-47d1-ba3c-fa02c2f27dff\") " pod="ai-tenant-e2e-mt-28c37d17/e2e-lifecycle-model-28c37d17-kserve-66d4f54799-986f5" Jul 07 22:32:20.780956 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:32:20.780808 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9acff786-c441-47d1-ba3c-fa02c2f27dff-dshm\") pod \"e2e-lifecycle-model-28c37d17-kserve-66d4f54799-986f5\" (UID: \"9acff786-c441-47d1-ba3c-fa02c2f27dff\") " pod="ai-tenant-e2e-mt-28c37d17/e2e-lifecycle-model-28c37d17-kserve-66d4f54799-986f5" Jul 07 22:32:20.780956 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:32:20.780911 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9acff786-c441-47d1-ba3c-fa02c2f27dff-model-cache\") pod \"e2e-lifecycle-model-28c37d17-kserve-66d4f54799-986f5\" (UID: \"9acff786-c441-47d1-ba3c-fa02c2f27dff\") " pod="ai-tenant-e2e-mt-28c37d17/e2e-lifecycle-model-28c37d17-kserve-66d4f54799-986f5" Jul 07 22:32:20.781068 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:32:20.780956 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9acff786-c441-47d1-ba3c-fa02c2f27dff-tmp-dir\") pod \"e2e-lifecycle-model-28c37d17-kserve-66d4f54799-986f5\" (UID: \"9acff786-c441-47d1-ba3c-fa02c2f27dff\") " pod="ai-tenant-e2e-mt-28c37d17/e2e-lifecycle-model-28c37d17-kserve-66d4f54799-986f5" Jul 07 22:32:20.781068 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:32:20.781001 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hfgj4\" (UniqueName: \"kubernetes.io/projected/9acff786-c441-47d1-ba3c-fa02c2f27dff-kube-api-access-hfgj4\") pod \"e2e-lifecycle-model-28c37d17-kserve-66d4f54799-986f5\" (UID: \"9acff786-c441-47d1-ba3c-fa02c2f27dff\") " pod="ai-tenant-e2e-mt-28c37d17/e2e-lifecycle-model-28c37d17-kserve-66d4f54799-986f5" Jul 07 22:32:20.882248 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:32:20.882158 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9acff786-c441-47d1-ba3c-fa02c2f27dff-model-cache\") pod \"e2e-lifecycle-model-28c37d17-kserve-66d4f54799-986f5\" (UID: \"9acff786-c441-47d1-ba3c-fa02c2f27dff\") " pod="ai-tenant-e2e-mt-28c37d17/e2e-lifecycle-model-28c37d17-kserve-66d4f54799-986f5" Jul 07 22:32:20.882248 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:32:20.882210 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9acff786-c441-47d1-ba3c-fa02c2f27dff-tmp-dir\") pod \"e2e-lifecycle-model-28c37d17-kserve-66d4f54799-986f5\" (UID: \"9acff786-c441-47d1-ba3c-fa02c2f27dff\") " pod="ai-tenant-e2e-mt-28c37d17/e2e-lifecycle-model-28c37d17-kserve-66d4f54799-986f5" Jul 07 22:32:20.882464 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:32:20.882264 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hfgj4\" (UniqueName: \"kubernetes.io/projected/9acff786-c441-47d1-ba3c-fa02c2f27dff-kube-api-access-hfgj4\") pod \"e2e-lifecycle-model-28c37d17-kserve-66d4f54799-986f5\" (UID: \"9acff786-c441-47d1-ba3c-fa02c2f27dff\") " pod="ai-tenant-e2e-mt-28c37d17/e2e-lifecycle-model-28c37d17-kserve-66d4f54799-986f5" Jul 07 22:32:20.882464 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:32:20.882337 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9acff786-c441-47d1-ba3c-fa02c2f27dff-home\") pod \"e2e-lifecycle-model-28c37d17-kserve-66d4f54799-986f5\" (UID: \"9acff786-c441-47d1-ba3c-fa02c2f27dff\") " pod="ai-tenant-e2e-mt-28c37d17/e2e-lifecycle-model-28c37d17-kserve-66d4f54799-986f5" Jul 07 22:32:20.882464 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:32:20.882368 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9acff786-c441-47d1-ba3c-fa02c2f27dff-tls-certs\") pod \"e2e-lifecycle-model-28c37d17-kserve-66d4f54799-986f5\" (UID: \"9acff786-c441-47d1-ba3c-fa02c2f27dff\") " pod="ai-tenant-e2e-mt-28c37d17/e2e-lifecycle-model-28c37d17-kserve-66d4f54799-986f5" Jul 07 22:32:20.882464 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:32:20.882390 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9acff786-c441-47d1-ba3c-fa02c2f27dff-dshm\") pod \"e2e-lifecycle-model-28c37d17-kserve-66d4f54799-986f5\" (UID: \"9acff786-c441-47d1-ba3c-fa02c2f27dff\") " pod="ai-tenant-e2e-mt-28c37d17/e2e-lifecycle-model-28c37d17-kserve-66d4f54799-986f5" Jul 07 22:32:20.882676 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:32:20.882616 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9acff786-c441-47d1-ba3c-fa02c2f27dff-model-cache\") pod \"e2e-lifecycle-model-28c37d17-kserve-66d4f54799-986f5\" (UID: \"9acff786-c441-47d1-ba3c-fa02c2f27dff\") " pod="ai-tenant-e2e-mt-28c37d17/e2e-lifecycle-model-28c37d17-kserve-66d4f54799-986f5" Jul 07 22:32:20.882676 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:32:20.882636 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9acff786-c441-47d1-ba3c-fa02c2f27dff-tmp-dir\") pod \"e2e-lifecycle-model-28c37d17-kserve-66d4f54799-986f5\" (UID: \"9acff786-c441-47d1-ba3c-fa02c2f27dff\") " pod="ai-tenant-e2e-mt-28c37d17/e2e-lifecycle-model-28c37d17-kserve-66d4f54799-986f5" Jul 07 22:32:20.882770 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:32:20.882706 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9acff786-c441-47d1-ba3c-fa02c2f27dff-home\") pod \"e2e-lifecycle-model-28c37d17-kserve-66d4f54799-986f5\" (UID: \"9acff786-c441-47d1-ba3c-fa02c2f27dff\") " pod="ai-tenant-e2e-mt-28c37d17/e2e-lifecycle-model-28c37d17-kserve-66d4f54799-986f5" Jul 07 22:32:20.884578 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:32:20.884552 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9acff786-c441-47d1-ba3c-fa02c2f27dff-dshm\") pod \"e2e-lifecycle-model-28c37d17-kserve-66d4f54799-986f5\" (UID: \"9acff786-c441-47d1-ba3c-fa02c2f27dff\") " pod="ai-tenant-e2e-mt-28c37d17/e2e-lifecycle-model-28c37d17-kserve-66d4f54799-986f5" Jul 07 22:32:20.884954 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:32:20.884933 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9acff786-c441-47d1-ba3c-fa02c2f27dff-tls-certs\") pod \"e2e-lifecycle-model-28c37d17-kserve-66d4f54799-986f5\" (UID: \"9acff786-c441-47d1-ba3c-fa02c2f27dff\") " pod="ai-tenant-e2e-mt-28c37d17/e2e-lifecycle-model-28c37d17-kserve-66d4f54799-986f5" Jul 07 22:32:20.890139 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:32:20.890114 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hfgj4\" (UniqueName: \"kubernetes.io/projected/9acff786-c441-47d1-ba3c-fa02c2f27dff-kube-api-access-hfgj4\") pod \"e2e-lifecycle-model-28c37d17-kserve-66d4f54799-986f5\" (UID: \"9acff786-c441-47d1-ba3c-fa02c2f27dff\") " pod="ai-tenant-e2e-mt-28c37d17/e2e-lifecycle-model-28c37d17-kserve-66d4f54799-986f5" Jul 07 22:32:20.923846 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:32:20.923818 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-28c37d17/e2e-lifecycle-model-28c37d17-kserve-66d4f54799-986f5" Jul 07 22:32:21.082405 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:32:21.082381 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-28c37d17/e2e-lifecycle-model-28c37d17-kserve-66d4f54799-986f5"] Jul 07 22:32:21.084354 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:32:21.084322 2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9acff786_c441_47d1_ba3c_fa02c2f27dff.slice/crio-215d193dd3d2ac60c416af6957774854078062cd4c62e19871ce6bd15042c09c WatchSource:0}: Error finding container 215d193dd3d2ac60c416af6957774854078062cd4c62e19871ce6bd15042c09c: Status 404 returned error can't find the container with id 215d193dd3d2ac60c416af6957774854078062cd4c62e19871ce6bd15042c09c Jul 07 22:32:21.385310 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:32:21.385276 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-28c37d17/e2e-lifecycle-model-28c37d17-kserve-66d4f54799-986f5" event={"ID":"9acff786-c441-47d1-ba3c-fa02c2f27dff","Type":"ContainerStarted","Data":"9be11dc25fe4607252889480f9d85116bc1a9eb8881f757d306c325aaaf18c3a"} Jul 07 22:32:21.385506 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:32:21.385319 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-28c37d17/e2e-lifecycle-model-28c37d17-kserve-66d4f54799-986f5" event={"ID":"9acff786-c441-47d1-ba3c-fa02c2f27dff","Type":"ContainerStarted","Data":"215d193dd3d2ac60c416af6957774854078062cd4c62e19871ce6bd15042c09c"} Jul 07 22:32:21.402123 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:32:21.402013 2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-28c37d17/e2e-lifecycle-model-28c37d17-kserve-66d4f54799-986f5" podStartSLOduration=1.4020001 podStartE2EDuration="1.4020001s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 22:32:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:32:21.401240624 +0000 UTC m=+1723.715463643" watchObservedRunningTime="2026-07-07 22:32:21.4020001 +0000 UTC m=+1723.716223119" Jul 07 22:32:24.402126 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:32:24.402093 2545 generic.go:356] "Generic (PLEG): container finished" podID="1b2626d9-3efd-4940-aab7-8dac744fd28e" containerID="927efd019b370be0e83a69d3b970e0fbf6259b164d1b17d5ba480e736bd5030b" exitCode=0 Jul 07 22:32:24.402559 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:32:24.402109 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-28c37d17-openshift-default-78d7487b59-wmn9c" event={"ID":"1b2626d9-3efd-4940-aab7-8dac744fd28e","Type":"ContainerDied","Data":"927efd019b370be0e83a69d3b970e0fbf6259b164d1b17d5ba480e736bd5030b"} Jul 07 22:32:24.402559 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:32:24.402516 2545 scope.go:117] "RemoveContainer" containerID="927efd019b370be0e83a69d3b970e0fbf6259b164d1b17d5ba480e736bd5030b" Jul 07 22:32:24.403554 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:32:24.403524 2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 22:32:24.403655 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:32:24.403606 2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 22:32:24.403723 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:32:24.403654 2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 22:32:25.329602 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:32:25.329571 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-28c37d17-openshift-default-78d7487b59-wmn9c" Jul 07 22:32:25.409524 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:32:25.409489 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-28c37d17-openshift-default-78d7487b59-wmn9c" event={"ID":"1b2626d9-3efd-4940-aab7-8dac744fd28e","Type":"ContainerStarted","Data":"40c641c96ede65afacc7a39b000e7ac787b94e999e9e544af32ae56005d2c8f3"} Jul 07 22:32:25.684182 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:32:25.684068 2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-28c37d17-openshift-default-78d7487b59-wmn9c" Jul 07 22:32:25.685385 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:32:25.685361 2545 patch_prober.go:28] interesting pod/e2e-mt-28c37d17-openshift-default-78d7487b59-wmn9c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" start-of-body= Jul 07 22:32:25.685442 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:32:25.685414 2545 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-28c37d17-openshift-default-78d7487b59-wmn9c" podUID="1b2626d9-3efd-4940-aab7-8dac744fd28e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" Jul 07 22:32:26.684712 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:32:26.684670 2545 patch_prober.go:28] interesting pod/e2e-mt-28c37d17-openshift-default-78d7487b59-wmn9c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" start-of-body= Jul 07 22:32:26.685127 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:32:26.684731 2545 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-28c37d17-openshift-default-78d7487b59-wmn9c" podUID="1b2626d9-3efd-4940-aab7-8dac744fd28e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" Jul 07 22:32:27.683908 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:32:27.683874 2545 patch_prober.go:28] interesting pod/e2e-mt-28c37d17-openshift-default-78d7487b59-wmn9c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" start-of-body= Jul 07 22:32:27.684073 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:32:27.683928 2545 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-28c37d17-openshift-default-78d7487b59-wmn9c" podUID="1b2626d9-3efd-4940-aab7-8dac744fd28e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" Jul 07 22:32:28.684715 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:32:28.684683 2545 patch_prober.go:28] interesting pod/e2e-mt-28c37d17-openshift-default-78d7487b59-wmn9c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" start-of-body= Jul 07 22:32:28.685124 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:32:28.684735 2545 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-28c37d17-openshift-default-78d7487b59-wmn9c" podUID="1b2626d9-3efd-4940-aab7-8dac744fd28e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.38:15021/healthz/ready\": dial tcp 10.132.0.38:15021: connect: connection refused" Jul 07 22:32:29.687853 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:32:29.687823 2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-28c37d17-openshift-default-78d7487b59-wmn9c" Jul 07 22:32:29.688248 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:32:29.688110 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-28c37d17-openshift-default-78d7487b59-wmn9c" Jul 07 22:32:29.688907 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:32:29.688888 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-28c37d17-openshift-default-78d7487b59-wmn9c" Jul 07 22:32:30.924345 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:32:30.924301 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-28c37d17/e2e-lifecycle-model-28c37d17-kserve-66d4f54799-986f5" Jul 07 22:32:30.924744 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:32:30.924377 2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-28c37d17/e2e-lifecycle-model-28c37d17-kserve-66d4f54799-986f5" Jul 07 22:32:30.937126 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:32:30.937105 2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-28c37d17/e2e-lifecycle-model-28c37d17-kserve-66d4f54799-986f5" Jul 07 22:32:31.454395 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:32:31.454363 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-28c37d17/e2e-lifecycle-model-28c37d17-kserve-66d4f54799-986f5" Jul 07 22:33:04.202227 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:33:04.202188 2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-28c37d17/e2e-lifecycle-model-28c37d17-kserve-66d4f54799-986f5"] Jul 07 22:33:19.223133 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:33:19.223062 2545 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-mt-28c37d17/e2e-lifecycle-model-28c37d17-kserve-66d4f54799-986f5" podUID="9acff786-c441-47d1-ba3c-fa02c2f27dff" containerName="main" containerID="cri-o://9be11dc25fe4607252889480f9d85116bc1a9eb8881f757d306c325aaaf18c3a" gracePeriod=45 Jul 07 22:33:19.468628 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:33:19.468602 2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-28c37d17/e2e-lifecycle-model-28c37d17-kserve-66d4f54799-986f5" Jul 07 22:33:19.522712 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:33:19.522628 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9acff786-c441-47d1-ba3c-fa02c2f27dff-model-cache\") pod \"9acff786-c441-47d1-ba3c-fa02c2f27dff\" (UID: \"9acff786-c441-47d1-ba3c-fa02c2f27dff\") " Jul 07 22:33:19.522712 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:33:19.522682 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9acff786-c441-47d1-ba3c-fa02c2f27dff-tls-certs\") pod \"9acff786-c441-47d1-ba3c-fa02c2f27dff\" (UID: \"9acff786-c441-47d1-ba3c-fa02c2f27dff\") " Jul 07 22:33:19.522937 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:33:19.522830 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9acff786-c441-47d1-ba3c-fa02c2f27dff-dshm\") pod \"9acff786-c441-47d1-ba3c-fa02c2f27dff\" (UID: \"9acff786-c441-47d1-ba3c-fa02c2f27dff\") " Jul 07 22:33:19.522937 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:33:19.522878 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9acff786-c441-47d1-ba3c-fa02c2f27dff-model-cache" (OuterVolumeSpecName: "model-cache") pod "9acff786-c441-47d1-ba3c-fa02c2f27dff" (UID: "9acff786-c441-47d1-ba3c-fa02c2f27dff"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 22:33:19.522937 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:33:19.522900 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hfgj4\" (UniqueName: \"kubernetes.io/projected/9acff786-c441-47d1-ba3c-fa02c2f27dff-kube-api-access-hfgj4\") pod \"9acff786-c441-47d1-ba3c-fa02c2f27dff\" (UID: \"9acff786-c441-47d1-ba3c-fa02c2f27dff\") " Jul 07 22:33:19.522937 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:33:19.522936 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9acff786-c441-47d1-ba3c-fa02c2f27dff-tmp-dir\") pod \"9acff786-c441-47d1-ba3c-fa02c2f27dff\" (UID: \"9acff786-c441-47d1-ba3c-fa02c2f27dff\") " Jul 07 22:33:19.523192 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:33:19.522976 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9acff786-c441-47d1-ba3c-fa02c2f27dff-home\") pod \"9acff786-c441-47d1-ba3c-fa02c2f27dff\" (UID: \"9acff786-c441-47d1-ba3c-fa02c2f27dff\") " Jul 07 22:33:19.523280 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:33:19.523255 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9acff786-c441-47d1-ba3c-fa02c2f27dff-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "9acff786-c441-47d1-ba3c-fa02c2f27dff" (UID: "9acff786-c441-47d1-ba3c-fa02c2f27dff"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 22:33:19.523350 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:33:19.523287 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9acff786-c441-47d1-ba3c-fa02c2f27dff-home" (OuterVolumeSpecName: "home") pod "9acff786-c441-47d1-ba3c-fa02c2f27dff" (UID: "9acff786-c441-47d1-ba3c-fa02c2f27dff"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 22:33:19.523407 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:33:19.523351 2545 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9acff786-c441-47d1-ba3c-fa02c2f27dff-home\") on node \"ip-10-0-131-55.ec2.internal\" DevicePath \"\"" Jul 07 22:33:19.523407 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:33:19.523364 2545 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9acff786-c441-47d1-ba3c-fa02c2f27dff-model-cache\") on node \"ip-10-0-131-55.ec2.internal\" DevicePath \"\"" Jul 07 22:33:19.523407 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:33:19.523379 2545 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9acff786-c441-47d1-ba3c-fa02c2f27dff-tmp-dir\") on node \"ip-10-0-131-55.ec2.internal\" DevicePath \"\"" Jul 07 22:33:19.525041 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:33:19.525008 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9acff786-c441-47d1-ba3c-fa02c2f27dff-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "9acff786-c441-47d1-ba3c-fa02c2f27dff" (UID: "9acff786-c441-47d1-ba3c-fa02c2f27dff"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 22:33:19.525041 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:33:19.525015 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9acff786-c441-47d1-ba3c-fa02c2f27dff-dshm" (OuterVolumeSpecName: "dshm") pod "9acff786-c441-47d1-ba3c-fa02c2f27dff" (UID: "9acff786-c441-47d1-ba3c-fa02c2f27dff"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 22:33:19.525339 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:33:19.525318 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9acff786-c441-47d1-ba3c-fa02c2f27dff-kube-api-access-hfgj4" (OuterVolumeSpecName: "kube-api-access-hfgj4") pod "9acff786-c441-47d1-ba3c-fa02c2f27dff" (UID: "9acff786-c441-47d1-ba3c-fa02c2f27dff"). InnerVolumeSpecName "kube-api-access-hfgj4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 22:33:19.624484 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:33:19.624446 2545 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9acff786-c441-47d1-ba3c-fa02c2f27dff-tls-certs\") on node \"ip-10-0-131-55.ec2.internal\" DevicePath \"\"" Jul 07 22:33:19.624484 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:33:19.624480 2545 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9acff786-c441-47d1-ba3c-fa02c2f27dff-dshm\") on node \"ip-10-0-131-55.ec2.internal\" DevicePath \"\"" Jul 07 22:33:19.624484 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:33:19.624490 2545 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hfgj4\" (UniqueName: \"kubernetes.io/projected/9acff786-c441-47d1-ba3c-fa02c2f27dff-kube-api-access-hfgj4\") on node \"ip-10-0-131-55.ec2.internal\" DevicePath \"\"" Jul 07 22:33:19.677171 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:33:19.677132 2545 generic.go:356] "Generic (PLEG): container finished" podID="9acff786-c441-47d1-ba3c-fa02c2f27dff" containerID="9be11dc25fe4607252889480f9d85116bc1a9eb8881f757d306c325aaaf18c3a" exitCode=0 Jul 07 22:33:19.677336 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:33:19.677173 2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-28c37d17/e2e-lifecycle-model-28c37d17-kserve-66d4f54799-986f5" Jul 07 22:33:19.677336 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:33:19.677184 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-28c37d17/e2e-lifecycle-model-28c37d17-kserve-66d4f54799-986f5" event={"ID":"9acff786-c441-47d1-ba3c-fa02c2f27dff","Type":"ContainerDied","Data":"9be11dc25fe4607252889480f9d85116bc1a9eb8881f757d306c325aaaf18c3a"} Jul 07 22:33:19.677336 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:33:19.677222 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-28c37d17/e2e-lifecycle-model-28c37d17-kserve-66d4f54799-986f5" event={"ID":"9acff786-c441-47d1-ba3c-fa02c2f27dff","Type":"ContainerDied","Data":"215d193dd3d2ac60c416af6957774854078062cd4c62e19871ce6bd15042c09c"} Jul 07 22:33:19.677336 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:33:19.677242 2545 scope.go:117] "RemoveContainer" containerID="9be11dc25fe4607252889480f9d85116bc1a9eb8881f757d306c325aaaf18c3a" Jul 07 22:33:19.686483 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:33:19.686460 2545 scope.go:117] "RemoveContainer" containerID="9be11dc25fe4607252889480f9d85116bc1a9eb8881f757d306c325aaaf18c3a" Jul 07 22:33:19.686723 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:33:19.686701 2545 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9be11dc25fe4607252889480f9d85116bc1a9eb8881f757d306c325aaaf18c3a\": container with ID starting with 9be11dc25fe4607252889480f9d85116bc1a9eb8881f757d306c325aaaf18c3a not found: ID does not exist" containerID="9be11dc25fe4607252889480f9d85116bc1a9eb8881f757d306c325aaaf18c3a" Jul 07 22:33:19.686792 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:33:19.686738 2545 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9be11dc25fe4607252889480f9d85116bc1a9eb8881f757d306c325aaaf18c3a"} err="failed to get container status \"9be11dc25fe4607252889480f9d85116bc1a9eb8881f757d306c325aaaf18c3a\": rpc error: code = NotFound desc = could not find container \"9be11dc25fe4607252889480f9d85116bc1a9eb8881f757d306c325aaaf18c3a\": container with ID starting with 9be11dc25fe4607252889480f9d85116bc1a9eb8881f757d306c325aaaf18c3a not found: ID does not exist" Jul 07 22:33:19.698450 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:33:19.698425 2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-28c37d17/e2e-lifecycle-model-28c37d17-kserve-66d4f54799-986f5"] Jul 07 22:33:19.700126 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:33:19.700106 2545 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-28c37d17/e2e-lifecycle-model-28c37d17-kserve-66d4f54799-986f5"] Jul 07 22:33:20.312013 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:33:20.311979 2545 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9acff786-c441-47d1-ba3c-fa02c2f27dff" path="/var/lib/kubelet/pods/9acff786-c441-47d1-ba3c-fa02c2f27dff/volumes" Jul 07 22:33:38.283698 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:33:38.283668 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-rhg5r_71f36bfa-4b8a-4572-b522-57c16392aa0b/console-operator/1.log" Jul 07 22:33:38.286604 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:33:38.286569 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-rhg5r_71f36bfa-4b8a-4572-b522-57c16392aa0b/console-operator/1.log" Jul 07 22:33:52.632334 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:33:52.632292 2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-28c37d17-openshift-default-78d7487b59-wmn9c"] Jul 07 22:33:52.632735 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:33:52.632558 2545 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-28c37d17-openshift-default-78d7487b59-wmn9c" podUID="1b2626d9-3efd-4940-aab7-8dac744fd28e" containerName="istio-proxy" containerID="cri-o://40c641c96ede65afacc7a39b000e7ac787b94e999e9e544af32ae56005d2c8f3" gracePeriod=30 Jul 07 22:33:57.855717 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:33:57.855686 2545 generic.go:356] "Generic (PLEG): container finished" podID="1b2626d9-3efd-4940-aab7-8dac744fd28e" containerID="40c641c96ede65afacc7a39b000e7ac787b94e999e9e544af32ae56005d2c8f3" exitCode=0 Jul 07 22:33:57.856106 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:33:57.855755 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-28c37d17-openshift-default-78d7487b59-wmn9c" event={"ID":"1b2626d9-3efd-4940-aab7-8dac744fd28e","Type":"ContainerDied","Data":"40c641c96ede65afacc7a39b000e7ac787b94e999e9e544af32ae56005d2c8f3"} Jul 07 22:33:57.856106 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:33:57.855817 2545 scope.go:117] "RemoveContainer" containerID="927efd019b370be0e83a69d3b970e0fbf6259b164d1b17d5ba480e736bd5030b" Jul 07 22:33:57.895037 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:33:57.894972 2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-28c37d17-openshift-default-78d7487b59-wmn9c" Jul 07 22:33:57.962418 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:33:57.962379 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1b2626d9-3efd-4940-aab7-8dac744fd28e-istio-data\") pod \"1b2626d9-3efd-4940-aab7-8dac744fd28e\" (UID: \"1b2626d9-3efd-4940-aab7-8dac744fd28e\") " Jul 07 22:33:57.962608 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:33:57.962429 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1b2626d9-3efd-4940-aab7-8dac744fd28e-istio-envoy\") pod \"1b2626d9-3efd-4940-aab7-8dac744fd28e\" (UID: \"1b2626d9-3efd-4940-aab7-8dac744fd28e\") " Jul 07 22:33:57.962608 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:33:57.962530 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1b2626d9-3efd-4940-aab7-8dac744fd28e-istio-token\") pod \"1b2626d9-3efd-4940-aab7-8dac744fd28e\" (UID: \"1b2626d9-3efd-4940-aab7-8dac744fd28e\") " Jul 07 22:33:57.962608 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:33:57.962557 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1b2626d9-3efd-4940-aab7-8dac744fd28e-istiod-ca-cert\") pod \"1b2626d9-3efd-4940-aab7-8dac744fd28e\" (UID: \"1b2626d9-3efd-4940-aab7-8dac744fd28e\") " Jul 07 22:33:57.962608 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:33:57.962581 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b2626d9-3efd-4940-aab7-8dac744fd28e-workload-socket\") pod \"1b2626d9-3efd-4940-aab7-8dac744fd28e\" (UID: \"1b2626d9-3efd-4940-aab7-8dac744fd28e\") " Jul 07 22:33:57.962608 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:33:57.962605 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9dfsr\" (UniqueName: \"kubernetes.io/projected/1b2626d9-3efd-4940-aab7-8dac744fd28e-kube-api-access-9dfsr\") pod \"1b2626d9-3efd-4940-aab7-8dac744fd28e\" (UID: \"1b2626d9-3efd-4940-aab7-8dac744fd28e\") " Jul 07 22:33:57.962881 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:33:57.962638 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1b2626d9-3efd-4940-aab7-8dac744fd28e-workload-certs\") pod \"1b2626d9-3efd-4940-aab7-8dac744fd28e\" (UID: \"1b2626d9-3efd-4940-aab7-8dac744fd28e\") " Jul 07 22:33:57.962881 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:33:57.962658 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b2626d9-3efd-4940-aab7-8dac744fd28e-credential-socket\") pod \"1b2626d9-3efd-4940-aab7-8dac744fd28e\" (UID: \"1b2626d9-3efd-4940-aab7-8dac744fd28e\") " Jul 07 22:33:57.962881 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:33:57.962707 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1b2626d9-3efd-4940-aab7-8dac744fd28e-istio-podinfo\") pod \"1b2626d9-3efd-4940-aab7-8dac744fd28e\" (UID: \"1b2626d9-3efd-4940-aab7-8dac744fd28e\") " Jul 07 22:33:57.963044 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:33:57.962887 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b2626d9-3efd-4940-aab7-8dac744fd28e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1b2626d9-3efd-4940-aab7-8dac744fd28e" (UID: "1b2626d9-3efd-4940-aab7-8dac744fd28e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 22:33:57.963126 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:33:57.963098 2545 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b2626d9-3efd-4940-aab7-8dac744fd28e-workload-socket\") on node \"ip-10-0-131-55.ec2.internal\" DevicePath \"\"" Jul 07 22:33:57.963350 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:33:57.963242 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b2626d9-3efd-4940-aab7-8dac744fd28e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1b2626d9-3efd-4940-aab7-8dac744fd28e" (UID: "1b2626d9-3efd-4940-aab7-8dac744fd28e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 22:33:57.963350 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:33:57.963282 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b2626d9-3efd-4940-aab7-8dac744fd28e-istio-data" (OuterVolumeSpecName: "istio-data") pod "1b2626d9-3efd-4940-aab7-8dac744fd28e" (UID: "1b2626d9-3efd-4940-aab7-8dac744fd28e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 22:33:57.963498 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:33:57.963391 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1b2626d9-3efd-4940-aab7-8dac744fd28e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1b2626d9-3efd-4940-aab7-8dac744fd28e" (UID: "1b2626d9-3efd-4940-aab7-8dac744fd28e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 22:33:57.963498 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:33:57.963179 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b2626d9-3efd-4940-aab7-8dac744fd28e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1b2626d9-3efd-4940-aab7-8dac744fd28e" (UID: "1b2626d9-3efd-4940-aab7-8dac744fd28e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 22:33:57.965257 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:33:57.965221 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b2626d9-3efd-4940-aab7-8dac744fd28e-istio-token" (OuterVolumeSpecName: "istio-token") pod "1b2626d9-3efd-4940-aab7-8dac744fd28e" (UID: "1b2626d9-3efd-4940-aab7-8dac744fd28e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 22:33:57.965393 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:33:57.965348 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b2626d9-3efd-4940-aab7-8dac744fd28e-kube-api-access-9dfsr" (OuterVolumeSpecName: "kube-api-access-9dfsr") pod "1b2626d9-3efd-4940-aab7-8dac744fd28e" (UID: "1b2626d9-3efd-4940-aab7-8dac744fd28e"). InnerVolumeSpecName "kube-api-access-9dfsr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 22:33:57.965568 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:33:57.965545 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1b2626d9-3efd-4940-aab7-8dac744fd28e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1b2626d9-3efd-4940-aab7-8dac744fd28e" (UID: "1b2626d9-3efd-4940-aab7-8dac744fd28e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 22:33:57.965655 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:33:57.965606 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b2626d9-3efd-4940-aab7-8dac744fd28e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1b2626d9-3efd-4940-aab7-8dac744fd28e" (UID: "1b2626d9-3efd-4940-aab7-8dac744fd28e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 22:33:58.064200 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:33:58.064159 2545 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1b2626d9-3efd-4940-aab7-8dac744fd28e-istio-data\") on node \"ip-10-0-131-55.ec2.internal\" DevicePath \"\"" Jul 07 22:33:58.064200 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:33:58.064190 2545 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1b2626d9-3efd-4940-aab7-8dac744fd28e-istio-envoy\") on node \"ip-10-0-131-55.ec2.internal\" DevicePath \"\"" Jul 07 22:33:58.064200 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:33:58.064199 2545 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1b2626d9-3efd-4940-aab7-8dac744fd28e-istio-token\") on node \"ip-10-0-131-55.ec2.internal\" DevicePath \"\"" Jul 07 22:33:58.064200 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:33:58.064208 2545 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1b2626d9-3efd-4940-aab7-8dac744fd28e-istiod-ca-cert\") on node \"ip-10-0-131-55.ec2.internal\" DevicePath \"\"" Jul 07 22:33:58.064548 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:33:58.064217 2545 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9dfsr\" (UniqueName: \"kubernetes.io/projected/1b2626d9-3efd-4940-aab7-8dac744fd28e-kube-api-access-9dfsr\") on node \"ip-10-0-131-55.ec2.internal\" DevicePath \"\"" Jul 07 22:33:58.064548 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:33:58.064226 2545 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1b2626d9-3efd-4940-aab7-8dac744fd28e-workload-certs\") on node \"ip-10-0-131-55.ec2.internal\" DevicePath \"\"" Jul 07 22:33:58.064548 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:33:58.064235 2545 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b2626d9-3efd-4940-aab7-8dac744fd28e-credential-socket\") on node \"ip-10-0-131-55.ec2.internal\" DevicePath \"\"" Jul 07 22:33:58.064548 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:33:58.064243 2545 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1b2626d9-3efd-4940-aab7-8dac744fd28e-istio-podinfo\") on node \"ip-10-0-131-55.ec2.internal\" DevicePath \"\"" Jul 07 22:33:58.862590 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:33:58.862556 2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-28c37d17-openshift-default-78d7487b59-wmn9c" Jul 07 22:33:58.862590 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:33:58.862581 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-28c37d17-openshift-default-78d7487b59-wmn9c" event={"ID":"1b2626d9-3efd-4940-aab7-8dac744fd28e","Type":"ContainerDied","Data":"9458b592a0f1b2e300d56120c41067840366c75f5c2149ef915c4d2816b445e3"} Jul 07 22:33:58.863159 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:33:58.862664 2545 scope.go:117] "RemoveContainer" containerID="40c641c96ede65afacc7a39b000e7ac787b94e999e9e544af32ae56005d2c8f3" Jul 07 22:33:58.880350 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:33:58.880322 2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-28c37d17-openshift-default-78d7487b59-wmn9c"] Jul 07 22:33:58.887681 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:33:58.887657 2545 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-28c37d17-openshift-default-78d7487b59-wmn9c"] Jul 07 22:34:00.310801 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:34:00.310768 2545 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1b2626d9-3efd-4940-aab7-8dac744fd28e" path="/var/lib/kubelet/pods/1b2626d9-3efd-4940-aab7-8dac744fd28e/volumes" Jul 07 22:34:46.573933 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:34:46.573896 2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-f8bdec-openshift-default-6497577769-rkddz"] Jul 07 22:34:46.574346 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:34:46.574275 2545 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b2626d9-3efd-4940-aab7-8dac744fd28e" containerName="istio-proxy" Jul 07 22:34:46.574346 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:34:46.574287 2545 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b2626d9-3efd-4940-aab7-8dac744fd28e" containerName="istio-proxy" Jul 07 22:34:46.574346 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:34:46.574301 2545 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9acff786-c441-47d1-ba3c-fa02c2f27dff" containerName="main" Jul 07 22:34:46.574346 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:34:46.574306 2545 state_mem.go:107] "Deleted CPUSet assignment" podUID="9acff786-c441-47d1-ba3c-fa02c2f27dff" containerName="main" Jul 07 22:34:46.574478 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:34:46.574375 2545 memory_manager.go:368] "RemoveStaleState removing state" podUID="9acff786-c441-47d1-ba3c-fa02c2f27dff" containerName="main" Jul 07 22:34:46.574478 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:34:46.574383 2545 memory_manager.go:368] "RemoveStaleState removing state" podUID="1b2626d9-3efd-4940-aab7-8dac744fd28e" containerName="istio-proxy" Jul 07 22:34:46.574478 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:34:46.574392 2545 memory_manager.go:368] "RemoveStaleState removing state" podUID="1b2626d9-3efd-4940-aab7-8dac744fd28e" containerName="istio-proxy" Jul 07 22:34:46.577240 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:34:46.577225 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-f8bdec-openshift-default-6497577769-rkddz" Jul 07 22:34:46.579614 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:34:46.579591 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 07 22:34:46.579736 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:34:46.579591 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-f8bdec-openshift-default-dockercfg-2kzdj\"" Jul 07 22:34:46.588414 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:34:46.588388 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-f8bdec-openshift-default-6497577769-rkddz"] Jul 07 22:34:46.611217 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:34:46.611185 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c222317e-1422-4d04-8698-bdb14bc281e4-istio-token\") pod \"e2e-inf-a-f8bdec-openshift-default-6497577769-rkddz\" (UID: \"c222317e-1422-4d04-8698-bdb14bc281e4\") " pod="openshift-ingress/e2e-inf-a-f8bdec-openshift-default-6497577769-rkddz" Jul 07 22:34:46.611321 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:34:46.611224 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c222317e-1422-4d04-8698-bdb14bc281e4-credential-socket\") pod \"e2e-inf-a-f8bdec-openshift-default-6497577769-rkddz\" (UID: \"c222317e-1422-4d04-8698-bdb14bc281e4\") " pod="openshift-ingress/e2e-inf-a-f8bdec-openshift-default-6497577769-rkddz" Jul 07 22:34:46.611321 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:34:46.611248 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jrclc\" (UniqueName: \"kubernetes.io/projected/c222317e-1422-4d04-8698-bdb14bc281e4-kube-api-access-jrclc\") pod \"e2e-inf-a-f8bdec-openshift-default-6497577769-rkddz\" (UID: \"c222317e-1422-4d04-8698-bdb14bc281e4\") " pod="openshift-ingress/e2e-inf-a-f8bdec-openshift-default-6497577769-rkddz" Jul 07 22:34:46.611321 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:34:46.611294 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c222317e-1422-4d04-8698-bdb14bc281e4-istio-envoy\") pod \"e2e-inf-a-f8bdec-openshift-default-6497577769-rkddz\" (UID: \"c222317e-1422-4d04-8698-bdb14bc281e4\") " pod="openshift-ingress/e2e-inf-a-f8bdec-openshift-default-6497577769-rkddz" Jul 07 22:34:46.611321 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:34:46.611315 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c222317e-1422-4d04-8698-bdb14bc281e4-workload-certs\") pod \"e2e-inf-a-f8bdec-openshift-default-6497577769-rkddz\" (UID: \"c222317e-1422-4d04-8698-bdb14bc281e4\") " pod="openshift-ingress/e2e-inf-a-f8bdec-openshift-default-6497577769-rkddz" Jul 07 22:34:46.611454 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:34:46.611401 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c222317e-1422-4d04-8698-bdb14bc281e4-istiod-ca-cert\") pod \"e2e-inf-a-f8bdec-openshift-default-6497577769-rkddz\" (UID: \"c222317e-1422-4d04-8698-bdb14bc281e4\") " pod="openshift-ingress/e2e-inf-a-f8bdec-openshift-default-6497577769-rkddz" Jul 07 22:34:46.611454 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:34:46.611438 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c222317e-1422-4d04-8698-bdb14bc281e4-workload-socket\") pod \"e2e-inf-a-f8bdec-openshift-default-6497577769-rkddz\" (UID: \"c222317e-1422-4d04-8698-bdb14bc281e4\") " pod="openshift-ingress/e2e-inf-a-f8bdec-openshift-default-6497577769-rkddz" Jul 07 22:34:46.611521 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:34:46.611459 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c222317e-1422-4d04-8698-bdb14bc281e4-istio-podinfo\") pod \"e2e-inf-a-f8bdec-openshift-default-6497577769-rkddz\" (UID: \"c222317e-1422-4d04-8698-bdb14bc281e4\") " pod="openshift-ingress/e2e-inf-a-f8bdec-openshift-default-6497577769-rkddz" Jul 07 22:34:46.611555 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:34:46.611516 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c222317e-1422-4d04-8698-bdb14bc281e4-istio-data\") pod \"e2e-inf-a-f8bdec-openshift-default-6497577769-rkddz\" (UID: \"c222317e-1422-4d04-8698-bdb14bc281e4\") " pod="openshift-ingress/e2e-inf-a-f8bdec-openshift-default-6497577769-rkddz" Jul 07 22:34:46.712947 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:34:46.712910 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c222317e-1422-4d04-8698-bdb14bc281e4-istio-token\") pod \"e2e-inf-a-f8bdec-openshift-default-6497577769-rkddz\" (UID: \"c222317e-1422-4d04-8698-bdb14bc281e4\") " pod="openshift-ingress/e2e-inf-a-f8bdec-openshift-default-6497577769-rkddz" Jul 07 22:34:46.712947 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:34:46.712951 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c222317e-1422-4d04-8698-bdb14bc281e4-credential-socket\") pod \"e2e-inf-a-f8bdec-openshift-default-6497577769-rkddz\" (UID: \"c222317e-1422-4d04-8698-bdb14bc281e4\") " pod="openshift-ingress/e2e-inf-a-f8bdec-openshift-default-6497577769-rkddz" Jul 07 22:34:46.713216 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:34:46.712972 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jrclc\" (UniqueName: \"kubernetes.io/projected/c222317e-1422-4d04-8698-bdb14bc281e4-kube-api-access-jrclc\") pod \"e2e-inf-a-f8bdec-openshift-default-6497577769-rkddz\" (UID: \"c222317e-1422-4d04-8698-bdb14bc281e4\") " pod="openshift-ingress/e2e-inf-a-f8bdec-openshift-default-6497577769-rkddz" Jul 07 22:34:46.713216 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:34:46.712997 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c222317e-1422-4d04-8698-bdb14bc281e4-istio-envoy\") pod \"e2e-inf-a-f8bdec-openshift-default-6497577769-rkddz\" (UID: \"c222317e-1422-4d04-8698-bdb14bc281e4\") " pod="openshift-ingress/e2e-inf-a-f8bdec-openshift-default-6497577769-rkddz" Jul 07 22:34:46.713216 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:34:46.713026 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c222317e-1422-4d04-8698-bdb14bc281e4-workload-certs\") pod \"e2e-inf-a-f8bdec-openshift-default-6497577769-rkddz\" (UID: \"c222317e-1422-4d04-8698-bdb14bc281e4\") " pod="openshift-ingress/e2e-inf-a-f8bdec-openshift-default-6497577769-rkddz" Jul 07 22:34:46.713216 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:34:46.713120 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c222317e-1422-4d04-8698-bdb14bc281e4-istiod-ca-cert\") pod \"e2e-inf-a-f8bdec-openshift-default-6497577769-rkddz\" (UID: \"c222317e-1422-4d04-8698-bdb14bc281e4\") " pod="openshift-ingress/e2e-inf-a-f8bdec-openshift-default-6497577769-rkddz" Jul 07 22:34:46.713216 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:34:46.713157 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c222317e-1422-4d04-8698-bdb14bc281e4-workload-socket\") pod \"e2e-inf-a-f8bdec-openshift-default-6497577769-rkddz\" (UID: \"c222317e-1422-4d04-8698-bdb14bc281e4\") " pod="openshift-ingress/e2e-inf-a-f8bdec-openshift-default-6497577769-rkddz" Jul 07 22:34:46.713216 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:34:46.713182 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c222317e-1422-4d04-8698-bdb14bc281e4-istio-podinfo\") pod \"e2e-inf-a-f8bdec-openshift-default-6497577769-rkddz\" (UID: \"c222317e-1422-4d04-8698-bdb14bc281e4\") " pod="openshift-ingress/e2e-inf-a-f8bdec-openshift-default-6497577769-rkddz" Jul 07 22:34:46.713570 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:34:46.713218 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c222317e-1422-4d04-8698-bdb14bc281e4-istio-data\") pod \"e2e-inf-a-f8bdec-openshift-default-6497577769-rkddz\" (UID: \"c222317e-1422-4d04-8698-bdb14bc281e4\") " pod="openshift-ingress/e2e-inf-a-f8bdec-openshift-default-6497577769-rkddz" Jul 07 22:34:46.713570 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:34:46.713531 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c222317e-1422-4d04-8698-bdb14bc281e4-workload-certs\") pod \"e2e-inf-a-f8bdec-openshift-default-6497577769-rkddz\" (UID: \"c222317e-1422-4d04-8698-bdb14bc281e4\") " pod="openshift-ingress/e2e-inf-a-f8bdec-openshift-default-6497577769-rkddz" Jul 07 22:34:46.713570 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:34:46.713559 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c222317e-1422-4d04-8698-bdb14bc281e4-credential-socket\") pod \"e2e-inf-a-f8bdec-openshift-default-6497577769-rkddz\" (UID: \"c222317e-1422-4d04-8698-bdb14bc281e4\") " pod="openshift-ingress/e2e-inf-a-f8bdec-openshift-default-6497577769-rkddz" Jul 07 22:34:46.713780 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:34:46.713678 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c222317e-1422-4d04-8698-bdb14bc281e4-workload-socket\") pod \"e2e-inf-a-f8bdec-openshift-default-6497577769-rkddz\" (UID: \"c222317e-1422-4d04-8698-bdb14bc281e4\") " pod="openshift-ingress/e2e-inf-a-f8bdec-openshift-default-6497577769-rkddz" Jul 07 22:34:46.713780 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:34:46.713743 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c222317e-1422-4d04-8698-bdb14bc281e4-istio-data\") pod \"e2e-inf-a-f8bdec-openshift-default-6497577769-rkddz\" (UID: \"c222317e-1422-4d04-8698-bdb14bc281e4\") " pod="openshift-ingress/e2e-inf-a-f8bdec-openshift-default-6497577769-rkddz" Jul 07 22:34:46.713905 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:34:46.713886 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c222317e-1422-4d04-8698-bdb14bc281e4-istiod-ca-cert\") pod \"e2e-inf-a-f8bdec-openshift-default-6497577769-rkddz\" (UID: \"c222317e-1422-4d04-8698-bdb14bc281e4\") " pod="openshift-ingress/e2e-inf-a-f8bdec-openshift-default-6497577769-rkddz" Jul 07 22:34:46.715309 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:34:46.715293 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c222317e-1422-4d04-8698-bdb14bc281e4-istio-envoy\") pod \"e2e-inf-a-f8bdec-openshift-default-6497577769-rkddz\" (UID: \"c222317e-1422-4d04-8698-bdb14bc281e4\") " pod="openshift-ingress/e2e-inf-a-f8bdec-openshift-default-6497577769-rkddz" Jul 07 22:34:46.715694 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:34:46.715673 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c222317e-1422-4d04-8698-bdb14bc281e4-istio-podinfo\") pod \"e2e-inf-a-f8bdec-openshift-default-6497577769-rkddz\" (UID: \"c222317e-1422-4d04-8698-bdb14bc281e4\") " pod="openshift-ingress/e2e-inf-a-f8bdec-openshift-default-6497577769-rkddz" Jul 07 22:34:46.722628 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:34:46.722603 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c222317e-1422-4d04-8698-bdb14bc281e4-istio-token\") pod \"e2e-inf-a-f8bdec-openshift-default-6497577769-rkddz\" (UID: \"c222317e-1422-4d04-8698-bdb14bc281e4\") " pod="openshift-ingress/e2e-inf-a-f8bdec-openshift-default-6497577769-rkddz" Jul 07 22:34:46.722718 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:34:46.722656 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jrclc\" (UniqueName: \"kubernetes.io/projected/c222317e-1422-4d04-8698-bdb14bc281e4-kube-api-access-jrclc\") pod \"e2e-inf-a-f8bdec-openshift-default-6497577769-rkddz\" (UID: \"c222317e-1422-4d04-8698-bdb14bc281e4\") " pod="openshift-ingress/e2e-inf-a-f8bdec-openshift-default-6497577769-rkddz" Jul 07 22:34:46.888878 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:34:46.888789 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-f8bdec-openshift-default-6497577769-rkddz" Jul 07 22:34:47.024734 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:34:47.024711 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-f8bdec-openshift-default-6497577769-rkddz"] Jul 07 22:34:47.026574 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:34:47.026547 2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc222317e_1422_4d04_8698_bdb14bc281e4.slice/crio-4aa3938d28c52e4265ff5578ab7a30bbc897b1edc378bf913da01019965c101a WatchSource:0}: Error finding container 4aa3938d28c52e4265ff5578ab7a30bbc897b1edc378bf913da01019965c101a: Status 404 returned error can't find the container with id 4aa3938d28c52e4265ff5578ab7a30bbc897b1edc378bf913da01019965c101a Jul 07 22:34:47.028704 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:34:47.028672 2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 22:34:47.028791 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:34:47.028736 2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 22:34:47.028791 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:34:47.028768 2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 22:34:47.092956 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:34:47.092921 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-f8bdec-openshift-default-6497577769-rkddz" event={"ID":"c222317e-1422-4d04-8698-bdb14bc281e4","Type":"ContainerStarted","Data":"1f4af79731e53c3b709900333845827487484e455aa2f2152644c8c724a03447"} Jul 07 22:34:47.093094 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:34:47.092985 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-f8bdec-openshift-default-6497577769-rkddz" event={"ID":"c222317e-1422-4d04-8698-bdb14bc281e4","Type":"ContainerStarted","Data":"4aa3938d28c52e4265ff5578ab7a30bbc897b1edc378bf913da01019965c101a"} Jul 07 22:34:47.112946 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:34:47.112889 2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-f8bdec-openshift-default-6497577769-rkddz" podStartSLOduration=1.11286627 podStartE2EDuration="1.11286627s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 22:34:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:34:47.109813413 +0000 UTC m=+1869.424036433" watchObservedRunningTime="2026-07-07 22:34:47.11286627 +0000 UTC m=+1869.427089291" Jul 07 22:34:47.889218 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:34:47.889167 2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-f8bdec-openshift-default-6497577769-rkddz" Jul 07 22:34:48.894438 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:34:48.894405 2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-f8bdec-openshift-default-6497577769-rkddz" Jul 07 22:34:49.102959 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:34:49.102933 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-f8bdec-openshift-default-6497577769-rkddz" Jul 07 22:34:49.103829 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:34:49.103810 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-f8bdec-openshift-default-6497577769-rkddz" Jul 07 22:35:01.166458 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:35:01.166365 2545 generic.go:356] "Generic (PLEG): container finished" podID="c222317e-1422-4d04-8698-bdb14bc281e4" containerID="1f4af79731e53c3b709900333845827487484e455aa2f2152644c8c724a03447" exitCode=0 Jul 07 22:35:01.166458 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:35:01.166441 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-f8bdec-openshift-default-6497577769-rkddz" event={"ID":"c222317e-1422-4d04-8698-bdb14bc281e4","Type":"ContainerDied","Data":"1f4af79731e53c3b709900333845827487484e455aa2f2152644c8c724a03447"} Jul 07 22:35:01.166880 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:35:01.166855 2545 scope.go:117] "RemoveContainer" containerID="1f4af79731e53c3b709900333845827487484e455aa2f2152644c8c724a03447" Jul 07 22:35:01.167783 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:35:01.167757 2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 22:35:01.167853 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:35:01.167838 2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 22:35:01.167889 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:35:01.167876 2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 22:35:02.173145 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:35:02.173105 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-f8bdec-openshift-default-6497577769-rkddz" event={"ID":"c222317e-1422-4d04-8698-bdb14bc281e4","Type":"ContainerStarted","Data":"7881cd5263a7195ddba380429e3890ca5d8d5912f53362dcdf28d8c1c4af82b3"} Jul 07 22:35:02.889763 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:35:02.889721 2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-f8bdec-openshift-default-6497577769-rkddz" Jul 07 22:35:02.890997 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:35:02.890971 2545 patch_prober.go:28] interesting pod/e2e-inf-a-f8bdec-openshift-default-6497577769-rkddz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body= Jul 07 22:35:02.891061 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:35:02.891037 2545 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-f8bdec-openshift-default-6497577769-rkddz" podUID="c222317e-1422-4d04-8698-bdb14bc281e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" Jul 07 22:35:03.889883 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:35:03.889848 2545 patch_prober.go:28] interesting pod/e2e-inf-a-f8bdec-openshift-default-6497577769-rkddz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body= Jul 07 22:35:03.890297 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:35:03.889905 2545 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-f8bdec-openshift-default-6497577769-rkddz" podUID="c222317e-1422-4d04-8698-bdb14bc281e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" Jul 07 22:35:04.103805 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:35:04.103774 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-f8bdec-openshift-default-6497577769-rkddz" Jul 07 22:35:04.890155 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:35:04.890117 2545 patch_prober.go:28] interesting pod/e2e-inf-a-f8bdec-openshift-default-6497577769-rkddz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body= Jul 07 22:35:04.890542 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:35:04.890176 2545 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-f8bdec-openshift-default-6497577769-rkddz" podUID="c222317e-1422-4d04-8698-bdb14bc281e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" Jul 07 22:35:05.889679 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:35:05.889642 2545 patch_prober.go:28] interesting pod/e2e-inf-a-f8bdec-openshift-default-6497577769-rkddz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" start-of-body= Jul 07 22:35:05.889855 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:35:05.889710 2545 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-f8bdec-openshift-default-6497577769-rkddz" podUID="c222317e-1422-4d04-8698-bdb14bc281e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.40:15021/healthz/ready\": dial tcp 10.132.0.40:15021: connect: connection refused" Jul 07 22:35:06.893617 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:35:06.893585 2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-f8bdec-openshift-default-6497577769-rkddz" Jul 07 22:35:06.894596 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:35:06.894578 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-f8bdec-openshift-default-6497577769-rkddz" Jul 07 22:35:23.204430 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:35:23.204392 2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-f8bdec/test-model-f8bdec-kserve-7c4bc4859c-tqmkn"] Jul 07 22:35:23.205005 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:35:23.204977 2545 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b2626d9-3efd-4940-aab7-8dac744fd28e" containerName="istio-proxy" Jul 07 22:35:23.205005 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:35:23.204995 2545 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b2626d9-3efd-4940-aab7-8dac744fd28e" containerName="istio-proxy" Jul 07 22:35:23.208627 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:35:23.208606 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-f8bdec/test-model-f8bdec-kserve-7c4bc4859c-tqmkn" Jul 07 22:35:23.212494 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:35:23.212469 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-f8bdec\"/\"kube-root-ca.crt\"" Jul 07 22:35:23.212632 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:35:23.212547 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-f8bdec\"/\"openshift-service-ca.crt\"" Jul 07 22:35:23.212632 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:35:23.212476 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-f8bdec\"/\"test-model-f8bdec-kserve-self-signed-certs\"" Jul 07 22:35:23.212632 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:35:23.212475 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-f8bdec\"/\"default-dockercfg-pvx8m\"" Jul 07 22:35:23.214967 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:35:23.214944 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-f8bdec/test-model-f8bdec-kserve-7c4bc4859c-tqmkn"] Jul 07 22:35:23.357682 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:35:23.357640 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/de0bbb54-52f1-474b-aa83-6ceb153a9526-dshm\") pod \"test-model-f8bdec-kserve-7c4bc4859c-tqmkn\" (UID: \"de0bbb54-52f1-474b-aa83-6ceb153a9526\") " pod="ai-tenant-e2e-inf-a-f8bdec/test-model-f8bdec-kserve-7c4bc4859c-tqmkn" Jul 07 22:35:23.357860 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:35:23.357686 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/de0bbb54-52f1-474b-aa83-6ceb153a9526-tls-certs\") pod \"test-model-f8bdec-kserve-7c4bc4859c-tqmkn\" (UID: \"de0bbb54-52f1-474b-aa83-6ceb153a9526\") " pod="ai-tenant-e2e-inf-a-f8bdec/test-model-f8bdec-kserve-7c4bc4859c-tqmkn" Jul 07 22:35:23.357860 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:35:23.357762 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/de0bbb54-52f1-474b-aa83-6ceb153a9526-tmp-dir\") pod \"test-model-f8bdec-kserve-7c4bc4859c-tqmkn\" (UID: \"de0bbb54-52f1-474b-aa83-6ceb153a9526\") " pod="ai-tenant-e2e-inf-a-f8bdec/test-model-f8bdec-kserve-7c4bc4859c-tqmkn" Jul 07 22:35:23.357938 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:35:23.357851 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/de0bbb54-52f1-474b-aa83-6ceb153a9526-home\") pod \"test-model-f8bdec-kserve-7c4bc4859c-tqmkn\" (UID: \"de0bbb54-52f1-474b-aa83-6ceb153a9526\") " pod="ai-tenant-e2e-inf-a-f8bdec/test-model-f8bdec-kserve-7c4bc4859c-tqmkn" Jul 07 22:35:23.357977 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:35:23.357939 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l2j67\" (UniqueName: \"kubernetes.io/projected/de0bbb54-52f1-474b-aa83-6ceb153a9526-kube-api-access-l2j67\") pod \"test-model-f8bdec-kserve-7c4bc4859c-tqmkn\" (UID: \"de0bbb54-52f1-474b-aa83-6ceb153a9526\") " pod="ai-tenant-e2e-inf-a-f8bdec/test-model-f8bdec-kserve-7c4bc4859c-tqmkn" Jul 07 22:35:23.357977 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:35:23.357974 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/de0bbb54-52f1-474b-aa83-6ceb153a9526-model-cache\") pod \"test-model-f8bdec-kserve-7c4bc4859c-tqmkn\" (UID: \"de0bbb54-52f1-474b-aa83-6ceb153a9526\") " pod="ai-tenant-e2e-inf-a-f8bdec/test-model-f8bdec-kserve-7c4bc4859c-tqmkn" Jul 07 22:35:23.458861 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:35:23.458777 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/de0bbb54-52f1-474b-aa83-6ceb153a9526-tmp-dir\") pod \"test-model-f8bdec-kserve-7c4bc4859c-tqmkn\" (UID: \"de0bbb54-52f1-474b-aa83-6ceb153a9526\") " pod="ai-tenant-e2e-inf-a-f8bdec/test-model-f8bdec-kserve-7c4bc4859c-tqmkn" Jul 07 22:35:23.458861 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:35:23.458815 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/de0bbb54-52f1-474b-aa83-6ceb153a9526-home\") pod \"test-model-f8bdec-kserve-7c4bc4859c-tqmkn\" (UID: \"de0bbb54-52f1-474b-aa83-6ceb153a9526\") " pod="ai-tenant-e2e-inf-a-f8bdec/test-model-f8bdec-kserve-7c4bc4859c-tqmkn" Jul 07 22:35:23.459094 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:35:23.458873 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l2j67\" (UniqueName: \"kubernetes.io/projected/de0bbb54-52f1-474b-aa83-6ceb153a9526-kube-api-access-l2j67\") pod \"test-model-f8bdec-kserve-7c4bc4859c-tqmkn\" (UID: \"de0bbb54-52f1-474b-aa83-6ceb153a9526\") " pod="ai-tenant-e2e-inf-a-f8bdec/test-model-f8bdec-kserve-7c4bc4859c-tqmkn" Jul 07 22:35:23.459094 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:35:23.458901 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/de0bbb54-52f1-474b-aa83-6ceb153a9526-model-cache\") pod \"test-model-f8bdec-kserve-7c4bc4859c-tqmkn\" (UID: \"de0bbb54-52f1-474b-aa83-6ceb153a9526\") " pod="ai-tenant-e2e-inf-a-f8bdec/test-model-f8bdec-kserve-7c4bc4859c-tqmkn" Jul 07 22:35:23.459094 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:35:23.458929 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/de0bbb54-52f1-474b-aa83-6ceb153a9526-dshm\") pod \"test-model-f8bdec-kserve-7c4bc4859c-tqmkn\" (UID: \"de0bbb54-52f1-474b-aa83-6ceb153a9526\") " pod="ai-tenant-e2e-inf-a-f8bdec/test-model-f8bdec-kserve-7c4bc4859c-tqmkn" Jul 07 22:35:23.459094 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:35:23.458956 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/de0bbb54-52f1-474b-aa83-6ceb153a9526-tls-certs\") pod \"test-model-f8bdec-kserve-7c4bc4859c-tqmkn\" (UID: \"de0bbb54-52f1-474b-aa83-6ceb153a9526\") " pod="ai-tenant-e2e-inf-a-f8bdec/test-model-f8bdec-kserve-7c4bc4859c-tqmkn" Jul 07 22:35:23.459299 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:35:23.459176 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/de0bbb54-52f1-474b-aa83-6ceb153a9526-tmp-dir\") pod \"test-model-f8bdec-kserve-7c4bc4859c-tqmkn\" (UID: \"de0bbb54-52f1-474b-aa83-6ceb153a9526\") " pod="ai-tenant-e2e-inf-a-f8bdec/test-model-f8bdec-kserve-7c4bc4859c-tqmkn" Jul 07 22:35:23.459299 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:35:23.459216 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/de0bbb54-52f1-474b-aa83-6ceb153a9526-home\") pod \"test-model-f8bdec-kserve-7c4bc4859c-tqmkn\" (UID: \"de0bbb54-52f1-474b-aa83-6ceb153a9526\") " pod="ai-tenant-e2e-inf-a-f8bdec/test-model-f8bdec-kserve-7c4bc4859c-tqmkn" Jul 07 22:35:23.459373 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:35:23.459292 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/de0bbb54-52f1-474b-aa83-6ceb153a9526-model-cache\") pod \"test-model-f8bdec-kserve-7c4bc4859c-tqmkn\" (UID: \"de0bbb54-52f1-474b-aa83-6ceb153a9526\") " pod="ai-tenant-e2e-inf-a-f8bdec/test-model-f8bdec-kserve-7c4bc4859c-tqmkn" Jul 07 22:35:23.461180 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:35:23.461160 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/de0bbb54-52f1-474b-aa83-6ceb153a9526-dshm\") pod \"test-model-f8bdec-kserve-7c4bc4859c-tqmkn\" (UID: \"de0bbb54-52f1-474b-aa83-6ceb153a9526\") " pod="ai-tenant-e2e-inf-a-f8bdec/test-model-f8bdec-kserve-7c4bc4859c-tqmkn" Jul 07 22:35:23.461494 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:35:23.461479 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/de0bbb54-52f1-474b-aa83-6ceb153a9526-tls-certs\") pod \"test-model-f8bdec-kserve-7c4bc4859c-tqmkn\" (UID: \"de0bbb54-52f1-474b-aa83-6ceb153a9526\") " pod="ai-tenant-e2e-inf-a-f8bdec/test-model-f8bdec-kserve-7c4bc4859c-tqmkn" Jul 07 22:35:23.466618 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:35:23.466593 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l2j67\" (UniqueName: \"kubernetes.io/projected/de0bbb54-52f1-474b-aa83-6ceb153a9526-kube-api-access-l2j67\") pod \"test-model-f8bdec-kserve-7c4bc4859c-tqmkn\" (UID: \"de0bbb54-52f1-474b-aa83-6ceb153a9526\") " pod="ai-tenant-e2e-inf-a-f8bdec/test-model-f8bdec-kserve-7c4bc4859c-tqmkn" Jul 07 22:35:23.521658 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:35:23.521629 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-f8bdec/test-model-f8bdec-kserve-7c4bc4859c-tqmkn" Jul 07 22:35:23.652417 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:35:23.652385 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-f8bdec/test-model-f8bdec-kserve-7c4bc4859c-tqmkn"] Jul 07 22:35:23.654592 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:35:23.654561 2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podde0bbb54_52f1_474b_aa83_6ceb153a9526.slice/crio-5851449c23c08dc065ec11e1674b29e7e8edb5f46917c5ce97be886078e8065a WatchSource:0}: Error finding container 5851449c23c08dc065ec11e1674b29e7e8edb5f46917c5ce97be886078e8065a: Status 404 returned error can't find the container with id 5851449c23c08dc065ec11e1674b29e7e8edb5f46917c5ce97be886078e8065a Jul 07 22:35:24.278329 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:35:24.278284 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-f8bdec/test-model-f8bdec-kserve-7c4bc4859c-tqmkn" event={"ID":"de0bbb54-52f1-474b-aa83-6ceb153a9526","Type":"ContainerStarted","Data":"34209a25a09d9fbdf128c13f7576a48db1719798c2fa5dade1f981f6650e9bab"} Jul 07 22:35:24.278724 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:35:24.278342 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-f8bdec/test-model-f8bdec-kserve-7c4bc4859c-tqmkn" event={"ID":"de0bbb54-52f1-474b-aa83-6ceb153a9526","Type":"ContainerStarted","Data":"5851449c23c08dc065ec11e1674b29e7e8edb5f46917c5ce97be886078e8065a"} Jul 07 22:35:24.294072 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:35:24.294020 2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-f8bdec/test-model-f8bdec-kserve-7c4bc4859c-tqmkn" podStartSLOduration=1.294005657 podStartE2EDuration="1.294005657s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 22:35:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:35:24.292874699 +0000 UTC m=+1906.607097733" watchObservedRunningTime="2026-07-07 22:35:24.294005657 +0000 UTC m=+1906.608228676" Jul 07 22:35:33.522264 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:35:33.522224 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-f8bdec/test-model-f8bdec-kserve-7c4bc4859c-tqmkn" Jul 07 22:35:33.522686 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:35:33.522281 2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-f8bdec/test-model-f8bdec-kserve-7c4bc4859c-tqmkn" Jul 07 22:35:33.534552 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:35:33.534525 2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-f8bdec/test-model-f8bdec-kserve-7c4bc4859c-tqmkn" Jul 07 22:35:34.340591 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:35:34.340558 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-f8bdec/test-model-f8bdec-kserve-7c4bc4859c-tqmkn" Jul 07 22:36:19.988459 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:36:19.988418 2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-f8bdec/test-model-f8bdec-kserve-7c4bc4859c-tqmkn"] Jul 07 22:36:35.009237 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:36:35.009179 2545 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-f8bdec/test-model-f8bdec-kserve-7c4bc4859c-tqmkn" podUID="de0bbb54-52f1-474b-aa83-6ceb153a9526" containerName="main" containerID="cri-o://34209a25a09d9fbdf128c13f7576a48db1719798c2fa5dade1f981f6650e9bab" gracePeriod=45 Jul 07 22:36:35.258963 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:36:35.258939 2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-f8bdec/test-model-f8bdec-kserve-7c4bc4859c-tqmkn" Jul 07 22:36:35.306706 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:36:35.306621 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/de0bbb54-52f1-474b-aa83-6ceb153a9526-home\") pod \"de0bbb54-52f1-474b-aa83-6ceb153a9526\" (UID: \"de0bbb54-52f1-474b-aa83-6ceb153a9526\") " Jul 07 22:36:35.306851 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:36:35.306713 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/de0bbb54-52f1-474b-aa83-6ceb153a9526-model-cache\") pod \"de0bbb54-52f1-474b-aa83-6ceb153a9526\" (UID: \"de0bbb54-52f1-474b-aa83-6ceb153a9526\") " Jul 07 22:36:35.306851 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:36:35.306749 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/de0bbb54-52f1-474b-aa83-6ceb153a9526-tls-certs\") pod \"de0bbb54-52f1-474b-aa83-6ceb153a9526\" (UID: \"de0bbb54-52f1-474b-aa83-6ceb153a9526\") " Jul 07 22:36:35.306851 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:36:35.306770 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/de0bbb54-52f1-474b-aa83-6ceb153a9526-tmp-dir\") pod \"de0bbb54-52f1-474b-aa83-6ceb153a9526\" (UID: \"de0bbb54-52f1-474b-aa83-6ceb153a9526\") " Jul 07 22:36:35.306851 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:36:35.306808 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/de0bbb54-52f1-474b-aa83-6ceb153a9526-dshm\") pod \"de0bbb54-52f1-474b-aa83-6ceb153a9526\" (UID: \"de0bbb54-52f1-474b-aa83-6ceb153a9526\") " Jul 07 22:36:35.306851 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:36:35.306842 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l2j67\" (UniqueName: \"kubernetes.io/projected/de0bbb54-52f1-474b-aa83-6ceb153a9526-kube-api-access-l2j67\") pod \"de0bbb54-52f1-474b-aa83-6ceb153a9526\" (UID: \"de0bbb54-52f1-474b-aa83-6ceb153a9526\") " Jul 07 22:36:35.307114 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:36:35.306900 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de0bbb54-52f1-474b-aa83-6ceb153a9526-home" (OuterVolumeSpecName: "home") pod "de0bbb54-52f1-474b-aa83-6ceb153a9526" (UID: "de0bbb54-52f1-474b-aa83-6ceb153a9526"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 22:36:35.307114 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:36:35.307032 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de0bbb54-52f1-474b-aa83-6ceb153a9526-model-cache" (OuterVolumeSpecName: "model-cache") pod "de0bbb54-52f1-474b-aa83-6ceb153a9526" (UID: "de0bbb54-52f1-474b-aa83-6ceb153a9526"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 22:36:35.307114 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:36:35.307044 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de0bbb54-52f1-474b-aa83-6ceb153a9526-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "de0bbb54-52f1-474b-aa83-6ceb153a9526" (UID: "de0bbb54-52f1-474b-aa83-6ceb153a9526"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 22:36:35.307264 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:36:35.307130 2545 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/de0bbb54-52f1-474b-aa83-6ceb153a9526-home\") on node \"ip-10-0-131-55.ec2.internal\" DevicePath \"\"" Jul 07 22:36:35.307264 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:36:35.307151 2545 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/de0bbb54-52f1-474b-aa83-6ceb153a9526-model-cache\") on node \"ip-10-0-131-55.ec2.internal\" DevicePath \"\"" Jul 07 22:36:35.307264 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:36:35.307165 2545 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/de0bbb54-52f1-474b-aa83-6ceb153a9526-tmp-dir\") on node \"ip-10-0-131-55.ec2.internal\" DevicePath \"\"" Jul 07 22:36:35.309071 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:36:35.309041 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/de0bbb54-52f1-474b-aa83-6ceb153a9526-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "de0bbb54-52f1-474b-aa83-6ceb153a9526" (UID: "de0bbb54-52f1-474b-aa83-6ceb153a9526"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 22:36:35.309071 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:36:35.309043 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de0bbb54-52f1-474b-aa83-6ceb153a9526-dshm" (OuterVolumeSpecName: "dshm") pod "de0bbb54-52f1-474b-aa83-6ceb153a9526" (UID: "de0bbb54-52f1-474b-aa83-6ceb153a9526"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 22:36:35.309243 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:36:35.309111 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/de0bbb54-52f1-474b-aa83-6ceb153a9526-kube-api-access-l2j67" (OuterVolumeSpecName: "kube-api-access-l2j67") pod "de0bbb54-52f1-474b-aa83-6ceb153a9526" (UID: "de0bbb54-52f1-474b-aa83-6ceb153a9526"). InnerVolumeSpecName "kube-api-access-l2j67". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 22:36:35.408493 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:36:35.408460 2545 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/de0bbb54-52f1-474b-aa83-6ceb153a9526-tls-certs\") on node \"ip-10-0-131-55.ec2.internal\" DevicePath \"\"" Jul 07 22:36:35.408493 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:36:35.408489 2545 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/de0bbb54-52f1-474b-aa83-6ceb153a9526-dshm\") on node \"ip-10-0-131-55.ec2.internal\" DevicePath \"\"" Jul 07 22:36:35.408493 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:36:35.408498 2545 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l2j67\" (UniqueName: \"kubernetes.io/projected/de0bbb54-52f1-474b-aa83-6ceb153a9526-kube-api-access-l2j67\") on node \"ip-10-0-131-55.ec2.internal\" DevicePath \"\"" Jul 07 22:36:35.618171 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:36:35.618067 2545 generic.go:356] "Generic (PLEG): container finished" podID="de0bbb54-52f1-474b-aa83-6ceb153a9526" containerID="34209a25a09d9fbdf128c13f7576a48db1719798c2fa5dade1f981f6650e9bab" exitCode=0 Jul 07 22:36:35.618171 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:36:35.618129 2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-f8bdec/test-model-f8bdec-kserve-7c4bc4859c-tqmkn" Jul 07 22:36:35.618171 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:36:35.618150 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-f8bdec/test-model-f8bdec-kserve-7c4bc4859c-tqmkn" event={"ID":"de0bbb54-52f1-474b-aa83-6ceb153a9526","Type":"ContainerDied","Data":"34209a25a09d9fbdf128c13f7576a48db1719798c2fa5dade1f981f6650e9bab"} Jul 07 22:36:35.618445 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:36:35.618194 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-f8bdec/test-model-f8bdec-kserve-7c4bc4859c-tqmkn" event={"ID":"de0bbb54-52f1-474b-aa83-6ceb153a9526","Type":"ContainerDied","Data":"5851449c23c08dc065ec11e1674b29e7e8edb5f46917c5ce97be886078e8065a"} Jul 07 22:36:35.618445 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:36:35.618211 2545 scope.go:117] "RemoveContainer" containerID="34209a25a09d9fbdf128c13f7576a48db1719798c2fa5dade1f981f6650e9bab" Jul 07 22:36:35.627521 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:36:35.627501 2545 scope.go:117] "RemoveContainer" containerID="34209a25a09d9fbdf128c13f7576a48db1719798c2fa5dade1f981f6650e9bab" Jul 07 22:36:35.627778 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:36:35.627757 2545 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"34209a25a09d9fbdf128c13f7576a48db1719798c2fa5dade1f981f6650e9bab\": container with ID starting with 34209a25a09d9fbdf128c13f7576a48db1719798c2fa5dade1f981f6650e9bab not found: ID does not exist" containerID="34209a25a09d9fbdf128c13f7576a48db1719798c2fa5dade1f981f6650e9bab" Jul 07 22:36:35.627842 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:36:35.627789 2545 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"34209a25a09d9fbdf128c13f7576a48db1719798c2fa5dade1f981f6650e9bab"} err="failed to get container status \"34209a25a09d9fbdf128c13f7576a48db1719798c2fa5dade1f981f6650e9bab\": rpc error: code = NotFound desc = could not find container \"34209a25a09d9fbdf128c13f7576a48db1719798c2fa5dade1f981f6650e9bab\": container with ID starting with 34209a25a09d9fbdf128c13f7576a48db1719798c2fa5dade1f981f6650e9bab not found: ID does not exist" Jul 07 22:36:35.640957 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:36:35.640929 2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-f8bdec/test-model-f8bdec-kserve-7c4bc4859c-tqmkn"] Jul 07 22:36:35.644037 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:36:35.644011 2545 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-f8bdec/test-model-f8bdec-kserve-7c4bc4859c-tqmkn"] Jul 07 22:36:36.311482 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:36:36.311449 2545 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="de0bbb54-52f1-474b-aa83-6ceb153a9526" path="/var/lib/kubelet/pods/de0bbb54-52f1-474b-aa83-6ceb153a9526/volumes" Jul 07 22:37:03.474482 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:03.474448 2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-f8bdec-openshift-default-6497577769-rkddz"] Jul 07 22:37:03.475013 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:03.474658 2545 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-f8bdec-openshift-default-6497577769-rkddz" podUID="c222317e-1422-4d04-8698-bdb14bc281e4" containerName="istio-proxy" containerID="cri-o://7881cd5263a7195ddba380429e3890ca5d8d5912f53362dcdf28d8c1c4af82b3" gracePeriod=30 Jul 07 22:37:06.896387 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:06.896346 2545 patch_prober.go:28] interesting pod/e2e-inf-a-f8bdec-openshift-default-6497577769-rkddz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 07 22:37:06.896759 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:06.896408 2545 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-f8bdec-openshift-default-6497577769-rkddz" podUID="c222317e-1422-4d04-8698-bdb14bc281e4" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 07 22:37:08.724497 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:08.724468 2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-f8bdec-openshift-default-6497577769-rkddz" Jul 07 22:37:08.777944 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:08.777854 2545 generic.go:356] "Generic (PLEG): container finished" podID="c222317e-1422-4d04-8698-bdb14bc281e4" containerID="7881cd5263a7195ddba380429e3890ca5d8d5912f53362dcdf28d8c1c4af82b3" exitCode=0 Jul 07 22:37:08.777944 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:08.777900 2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-f8bdec-openshift-default-6497577769-rkddz" Jul 07 22:37:08.777944 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:08.777918 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-f8bdec-openshift-default-6497577769-rkddz" event={"ID":"c222317e-1422-4d04-8698-bdb14bc281e4","Type":"ContainerDied","Data":"7881cd5263a7195ddba380429e3890ca5d8d5912f53362dcdf28d8c1c4af82b3"} Jul 07 22:37:08.778209 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:08.777964 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-f8bdec-openshift-default-6497577769-rkddz" event={"ID":"c222317e-1422-4d04-8698-bdb14bc281e4","Type":"ContainerDied","Data":"4aa3938d28c52e4265ff5578ab7a30bbc897b1edc378bf913da01019965c101a"} Jul 07 22:37:08.778209 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:08.777987 2545 scope.go:117] "RemoveContainer" containerID="7881cd5263a7195ddba380429e3890ca5d8d5912f53362dcdf28d8c1c4af82b3" Jul 07 22:37:08.786345 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:08.786320 2545 scope.go:117] "RemoveContainer" containerID="1f4af79731e53c3b709900333845827487484e455aa2f2152644c8c724a03447" Jul 07 22:37:08.793738 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:08.793719 2545 scope.go:117] "RemoveContainer" containerID="7881cd5263a7195ddba380429e3890ca5d8d5912f53362dcdf28d8c1c4af82b3" Jul 07 22:37:08.793983 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:37:08.793964 2545 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7881cd5263a7195ddba380429e3890ca5d8d5912f53362dcdf28d8c1c4af82b3\": container with ID starting with 7881cd5263a7195ddba380429e3890ca5d8d5912f53362dcdf28d8c1c4af82b3 not found: ID does not exist" containerID="7881cd5263a7195ddba380429e3890ca5d8d5912f53362dcdf28d8c1c4af82b3" Jul 07 22:37:08.794061 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:08.793998 2545 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7881cd5263a7195ddba380429e3890ca5d8d5912f53362dcdf28d8c1c4af82b3"} err="failed to get container status \"7881cd5263a7195ddba380429e3890ca5d8d5912f53362dcdf28d8c1c4af82b3\": rpc error: code = NotFound desc = could not find container \"7881cd5263a7195ddba380429e3890ca5d8d5912f53362dcdf28d8c1c4af82b3\": container with ID starting with 7881cd5263a7195ddba380429e3890ca5d8d5912f53362dcdf28d8c1c4af82b3 not found: ID does not exist" Jul 07 22:37:08.794061 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:08.794022 2545 scope.go:117] "RemoveContainer" containerID="1f4af79731e53c3b709900333845827487484e455aa2f2152644c8c724a03447" Jul 07 22:37:08.794296 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:37:08.794277 2545 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1f4af79731e53c3b709900333845827487484e455aa2f2152644c8c724a03447\": container with ID starting with 1f4af79731e53c3b709900333845827487484e455aa2f2152644c8c724a03447 not found: ID does not exist" containerID="1f4af79731e53c3b709900333845827487484e455aa2f2152644c8c724a03447" Jul 07 22:37:08.794341 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:08.794305 2545 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f4af79731e53c3b709900333845827487484e455aa2f2152644c8c724a03447"} err="failed to get container status \"1f4af79731e53c3b709900333845827487484e455aa2f2152644c8c724a03447\": rpc error: code = NotFound desc = could not find container \"1f4af79731e53c3b709900333845827487484e455aa2f2152644c8c724a03447\": container with ID starting with 1f4af79731e53c3b709900333845827487484e455aa2f2152644c8c724a03447 not found: ID does not exist" Jul 07 22:37:08.820419 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:08.820394 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jrclc\" (UniqueName: \"kubernetes.io/projected/c222317e-1422-4d04-8698-bdb14bc281e4-kube-api-access-jrclc\") pod \"c222317e-1422-4d04-8698-bdb14bc281e4\" (UID: \"c222317e-1422-4d04-8698-bdb14bc281e4\") " Jul 07 22:37:08.820548 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:08.820431 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c222317e-1422-4d04-8698-bdb14bc281e4-istio-token\") pod \"c222317e-1422-4d04-8698-bdb14bc281e4\" (UID: \"c222317e-1422-4d04-8698-bdb14bc281e4\") " Jul 07 22:37:08.820548 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:08.820454 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c222317e-1422-4d04-8698-bdb14bc281e4-credential-socket\") pod \"c222317e-1422-4d04-8698-bdb14bc281e4\" (UID: \"c222317e-1422-4d04-8698-bdb14bc281e4\") " Jul 07 22:37:08.820548 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:08.820476 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c222317e-1422-4d04-8698-bdb14bc281e4-istiod-ca-cert\") pod \"c222317e-1422-4d04-8698-bdb14bc281e4\" (UID: \"c222317e-1422-4d04-8698-bdb14bc281e4\") " Jul 07 22:37:08.820548 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:08.820504 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c222317e-1422-4d04-8698-bdb14bc281e4-istio-data\") pod \"c222317e-1422-4d04-8698-bdb14bc281e4\" (UID: \"c222317e-1422-4d04-8698-bdb14bc281e4\") " Jul 07 22:37:08.820750 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:08.820570 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c222317e-1422-4d04-8698-bdb14bc281e4-workload-socket\") pod \"c222317e-1422-4d04-8698-bdb14bc281e4\" (UID: \"c222317e-1422-4d04-8698-bdb14bc281e4\") " Jul 07 22:37:08.820750 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:08.820653 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c222317e-1422-4d04-8698-bdb14bc281e4-istio-podinfo\") pod \"c222317e-1422-4d04-8698-bdb14bc281e4\" (UID: \"c222317e-1422-4d04-8698-bdb14bc281e4\") " Jul 07 22:37:08.820860 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:08.820758 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c222317e-1422-4d04-8698-bdb14bc281e4-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c222317e-1422-4d04-8698-bdb14bc281e4" (UID: "c222317e-1422-4d04-8698-bdb14bc281e4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 22:37:08.820924 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:08.820842 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c222317e-1422-4d04-8698-bdb14bc281e4-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c222317e-1422-4d04-8698-bdb14bc281e4" (UID: "c222317e-1422-4d04-8698-bdb14bc281e4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 22:37:08.820985 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:08.820931 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c222317e-1422-4d04-8698-bdb14bc281e4-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c222317e-1422-4d04-8698-bdb14bc281e4" (UID: "c222317e-1422-4d04-8698-bdb14bc281e4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 22:37:08.821065 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:08.821040 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c222317e-1422-4d04-8698-bdb14bc281e4-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c222317e-1422-4d04-8698-bdb14bc281e4" (UID: "c222317e-1422-4d04-8698-bdb14bc281e4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 22:37:08.821168 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:08.820690 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c222317e-1422-4d04-8698-bdb14bc281e4-workload-certs\") pod \"c222317e-1422-4d04-8698-bdb14bc281e4\" (UID: \"c222317e-1422-4d04-8698-bdb14bc281e4\") " Jul 07 22:37:08.821222 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:08.821164 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c222317e-1422-4d04-8698-bdb14bc281e4-istio-envoy\") pod \"c222317e-1422-4d04-8698-bdb14bc281e4\" (UID: \"c222317e-1422-4d04-8698-bdb14bc281e4\") " Jul 07 22:37:08.821449 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:08.821418 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c222317e-1422-4d04-8698-bdb14bc281e4-istio-data" (OuterVolumeSpecName: "istio-data") pod "c222317e-1422-4d04-8698-bdb14bc281e4" (UID: "c222317e-1422-4d04-8698-bdb14bc281e4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 22:37:08.821535 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:08.821505 2545 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c222317e-1422-4d04-8698-bdb14bc281e4-workload-socket\") on node \"ip-10-0-131-55.ec2.internal\" DevicePath \"\"" Jul 07 22:37:08.821535 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:08.821526 2545 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c222317e-1422-4d04-8698-bdb14bc281e4-workload-certs\") on node \"ip-10-0-131-55.ec2.internal\" DevicePath \"\"" Jul 07 22:37:08.821636 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:08.821541 2545 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c222317e-1422-4d04-8698-bdb14bc281e4-credential-socket\") on node \"ip-10-0-131-55.ec2.internal\" DevicePath \"\"" Jul 07 22:37:08.821636 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:08.821556 2545 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c222317e-1422-4d04-8698-bdb14bc281e4-istiod-ca-cert\") on node \"ip-10-0-131-55.ec2.internal\" DevicePath \"\"" Jul 07 22:37:08.821636 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:08.821572 2545 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c222317e-1422-4d04-8698-bdb14bc281e4-istio-data\") on node \"ip-10-0-131-55.ec2.internal\" DevicePath \"\"" Jul 07 22:37:08.822818 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:08.822782 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c222317e-1422-4d04-8698-bdb14bc281e4-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c222317e-1422-4d04-8698-bdb14bc281e4" (UID: "c222317e-1422-4d04-8698-bdb14bc281e4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 22:37:08.822818 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:08.822798 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c222317e-1422-4d04-8698-bdb14bc281e4-istio-token" (OuterVolumeSpecName: "istio-token") pod "c222317e-1422-4d04-8698-bdb14bc281e4" (UID: "c222317e-1422-4d04-8698-bdb14bc281e4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 22:37:08.822985 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:08.822857 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c222317e-1422-4d04-8698-bdb14bc281e4-kube-api-access-jrclc" (OuterVolumeSpecName: "kube-api-access-jrclc") pod "c222317e-1422-4d04-8698-bdb14bc281e4" (UID: "c222317e-1422-4d04-8698-bdb14bc281e4"). InnerVolumeSpecName "kube-api-access-jrclc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 22:37:08.823152 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:08.823131 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c222317e-1422-4d04-8698-bdb14bc281e4-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c222317e-1422-4d04-8698-bdb14bc281e4" (UID: "c222317e-1422-4d04-8698-bdb14bc281e4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 22:37:08.922276 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:08.922241 2545 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c222317e-1422-4d04-8698-bdb14bc281e4-istio-envoy\") on node \"ip-10-0-131-55.ec2.internal\" DevicePath \"\"" Jul 07 22:37:08.922276 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:08.922270 2545 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jrclc\" (UniqueName: \"kubernetes.io/projected/c222317e-1422-4d04-8698-bdb14bc281e4-kube-api-access-jrclc\") on node \"ip-10-0-131-55.ec2.internal\" DevicePath \"\"" Jul 07 22:37:08.922276 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:08.922281 2545 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c222317e-1422-4d04-8698-bdb14bc281e4-istio-token\") on node \"ip-10-0-131-55.ec2.internal\" DevicePath \"\"" Jul 07 22:37:08.922494 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:08.922289 2545 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c222317e-1422-4d04-8698-bdb14bc281e4-istio-podinfo\") on node \"ip-10-0-131-55.ec2.internal\" DevicePath \"\"" Jul 07 22:37:09.100069 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:09.100038 2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-f8bdec-openshift-default-6497577769-rkddz"] Jul 07 22:37:09.103565 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:09.103541 2545 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-f8bdec-openshift-default-6497577769-rkddz"] Jul 07 22:37:10.311308 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:10.311270 2545 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c222317e-1422-4d04-8698-bdb14bc281e4" path="/var/lib/kubelet/pods/c222317e-1422-4d04-8698-bdb14bc281e4/volumes" Jul 07 22:37:11.120773 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:11.120741 2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-75ee45-openshift-default-f4ffbb777-nqqb2"] Jul 07 22:37:11.121128 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:11.121115 2545 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c222317e-1422-4d04-8698-bdb14bc281e4" containerName="istio-proxy" Jul 07 22:37:11.121128 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:11.121129 2545 state_mem.go:107] "Deleted CPUSet assignment" podUID="c222317e-1422-4d04-8698-bdb14bc281e4" containerName="istio-proxy" Jul 07 22:37:11.121257 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:11.121138 2545 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de0bbb54-52f1-474b-aa83-6ceb153a9526" containerName="main" Jul 07 22:37:11.121257 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:11.121144 2545 state_mem.go:107] "Deleted CPUSet assignment" podUID="de0bbb54-52f1-474b-aa83-6ceb153a9526" containerName="main" Jul 07 22:37:11.121257 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:11.121168 2545 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c222317e-1422-4d04-8698-bdb14bc281e4" containerName="istio-proxy" Jul 07 22:37:11.121257 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:11.121177 2545 state_mem.go:107] "Deleted CPUSet assignment" podUID="c222317e-1422-4d04-8698-bdb14bc281e4" containerName="istio-proxy" Jul 07 22:37:11.121452 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:11.121314 2545 memory_manager.go:368] "RemoveStaleState removing state" podUID="de0bbb54-52f1-474b-aa83-6ceb153a9526" containerName="main" Jul 07 22:37:11.121452 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:11.121332 2545 memory_manager.go:368] "RemoveStaleState removing state" podUID="c222317e-1422-4d04-8698-bdb14bc281e4" containerName="istio-proxy" Jul 07 22:37:11.121452 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:11.121342 2545 memory_manager.go:368] "RemoveStaleState removing state" podUID="c222317e-1422-4d04-8698-bdb14bc281e4" containerName="istio-proxy" Jul 07 22:37:11.125945 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:11.125925 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-75ee45-openshift-default-f4ffbb777-nqqb2" Jul 07 22:37:11.128409 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:11.128386 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-75ee45-openshift-default-dockercfg-brr5h\"" Jul 07 22:37:11.128506 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:11.128392 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 07 22:37:11.136164 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:11.136139 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-75ee45-openshift-default-f4ffbb777-nqqb2"] Jul 07 22:37:11.247589 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:11.247555 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/22d3bb9a-ce44-42e5-89fb-fbe379d92bd8-istio-envoy\") pod \"e2e-api-a-75ee45-openshift-default-f4ffbb777-nqqb2\" (UID: \"22d3bb9a-ce44-42e5-89fb-fbe379d92bd8\") " pod="openshift-ingress/e2e-api-a-75ee45-openshift-default-f4ffbb777-nqqb2" Jul 07 22:37:11.247775 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:11.247597 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/22d3bb9a-ce44-42e5-89fb-fbe379d92bd8-credential-socket\") pod \"e2e-api-a-75ee45-openshift-default-f4ffbb777-nqqb2\" (UID: \"22d3bb9a-ce44-42e5-89fb-fbe379d92bd8\") " pod="openshift-ingress/e2e-api-a-75ee45-openshift-default-f4ffbb777-nqqb2" Jul 07 22:37:11.247775 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:11.247620 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7tvp7\" (UniqueName: \"kubernetes.io/projected/22d3bb9a-ce44-42e5-89fb-fbe379d92bd8-kube-api-access-7tvp7\") pod \"e2e-api-a-75ee45-openshift-default-f4ffbb777-nqqb2\" (UID: \"22d3bb9a-ce44-42e5-89fb-fbe379d92bd8\") " pod="openshift-ingress/e2e-api-a-75ee45-openshift-default-f4ffbb777-nqqb2" Jul 07 22:37:11.247775 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:11.247642 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/22d3bb9a-ce44-42e5-89fb-fbe379d92bd8-istiod-ca-cert\") pod \"e2e-api-a-75ee45-openshift-default-f4ffbb777-nqqb2\" (UID: \"22d3bb9a-ce44-42e5-89fb-fbe379d92bd8\") " pod="openshift-ingress/e2e-api-a-75ee45-openshift-default-f4ffbb777-nqqb2" Jul 07 22:37:11.247775 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:11.247657 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/22d3bb9a-ce44-42e5-89fb-fbe379d92bd8-istio-token\") pod \"e2e-api-a-75ee45-openshift-default-f4ffbb777-nqqb2\" (UID: \"22d3bb9a-ce44-42e5-89fb-fbe379d92bd8\") " pod="openshift-ingress/e2e-api-a-75ee45-openshift-default-f4ffbb777-nqqb2" Jul 07 22:37:11.247775 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:11.247697 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/22d3bb9a-ce44-42e5-89fb-fbe379d92bd8-istio-podinfo\") pod \"e2e-api-a-75ee45-openshift-default-f4ffbb777-nqqb2\" (UID: \"22d3bb9a-ce44-42e5-89fb-fbe379d92bd8\") " pod="openshift-ingress/e2e-api-a-75ee45-openshift-default-f4ffbb777-nqqb2" Jul 07 22:37:11.247775 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:11.247731 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/22d3bb9a-ce44-42e5-89fb-fbe379d92bd8-istio-data\") pod \"e2e-api-a-75ee45-openshift-default-f4ffbb777-nqqb2\" (UID: \"22d3bb9a-ce44-42e5-89fb-fbe379d92bd8\") " pod="openshift-ingress/e2e-api-a-75ee45-openshift-default-f4ffbb777-nqqb2" Jul 07 22:37:11.247775 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:11.247754 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/22d3bb9a-ce44-42e5-89fb-fbe379d92bd8-workload-certs\") pod \"e2e-api-a-75ee45-openshift-default-f4ffbb777-nqqb2\" (UID: \"22d3bb9a-ce44-42e5-89fb-fbe379d92bd8\") " pod="openshift-ingress/e2e-api-a-75ee45-openshift-default-f4ffbb777-nqqb2" Jul 07 22:37:11.248029 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:11.247780 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/22d3bb9a-ce44-42e5-89fb-fbe379d92bd8-workload-socket\") pod \"e2e-api-a-75ee45-openshift-default-f4ffbb777-nqqb2\" (UID: \"22d3bb9a-ce44-42e5-89fb-fbe379d92bd8\") " pod="openshift-ingress/e2e-api-a-75ee45-openshift-default-f4ffbb777-nqqb2" Jul 07 22:37:11.349291 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:11.349244 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/22d3bb9a-ce44-42e5-89fb-fbe379d92bd8-istio-podinfo\") pod \"e2e-api-a-75ee45-openshift-default-f4ffbb777-nqqb2\" (UID: \"22d3bb9a-ce44-42e5-89fb-fbe379d92bd8\") " pod="openshift-ingress/e2e-api-a-75ee45-openshift-default-f4ffbb777-nqqb2" Jul 07 22:37:11.349707 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:11.349303 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/22d3bb9a-ce44-42e5-89fb-fbe379d92bd8-istio-data\") pod \"e2e-api-a-75ee45-openshift-default-f4ffbb777-nqqb2\" (UID: \"22d3bb9a-ce44-42e5-89fb-fbe379d92bd8\") " pod="openshift-ingress/e2e-api-a-75ee45-openshift-default-f4ffbb777-nqqb2" Jul 07 22:37:11.349707 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:11.349340 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/22d3bb9a-ce44-42e5-89fb-fbe379d92bd8-workload-certs\") pod \"e2e-api-a-75ee45-openshift-default-f4ffbb777-nqqb2\" (UID: \"22d3bb9a-ce44-42e5-89fb-fbe379d92bd8\") " pod="openshift-ingress/e2e-api-a-75ee45-openshift-default-f4ffbb777-nqqb2" Jul 07 22:37:11.349707 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:11.349369 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/22d3bb9a-ce44-42e5-89fb-fbe379d92bd8-workload-socket\") pod \"e2e-api-a-75ee45-openshift-default-f4ffbb777-nqqb2\" (UID: \"22d3bb9a-ce44-42e5-89fb-fbe379d92bd8\") " pod="openshift-ingress/e2e-api-a-75ee45-openshift-default-f4ffbb777-nqqb2" Jul 07 22:37:11.349707 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:11.349430 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/22d3bb9a-ce44-42e5-89fb-fbe379d92bd8-istio-envoy\") pod \"e2e-api-a-75ee45-openshift-default-f4ffbb777-nqqb2\" (UID: \"22d3bb9a-ce44-42e5-89fb-fbe379d92bd8\") " pod="openshift-ingress/e2e-api-a-75ee45-openshift-default-f4ffbb777-nqqb2" Jul 07 22:37:11.349707 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:11.349465 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/22d3bb9a-ce44-42e5-89fb-fbe379d92bd8-credential-socket\") pod \"e2e-api-a-75ee45-openshift-default-f4ffbb777-nqqb2\" (UID: \"22d3bb9a-ce44-42e5-89fb-fbe379d92bd8\") " pod="openshift-ingress/e2e-api-a-75ee45-openshift-default-f4ffbb777-nqqb2" Jul 07 22:37:11.349707 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:11.349494 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7tvp7\" (UniqueName: \"kubernetes.io/projected/22d3bb9a-ce44-42e5-89fb-fbe379d92bd8-kube-api-access-7tvp7\") pod \"e2e-api-a-75ee45-openshift-default-f4ffbb777-nqqb2\" (UID: \"22d3bb9a-ce44-42e5-89fb-fbe379d92bd8\") " pod="openshift-ingress/e2e-api-a-75ee45-openshift-default-f4ffbb777-nqqb2" Jul 07 22:37:11.349707 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:11.349521 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/22d3bb9a-ce44-42e5-89fb-fbe379d92bd8-istiod-ca-cert\") pod \"e2e-api-a-75ee45-openshift-default-f4ffbb777-nqqb2\" (UID: \"22d3bb9a-ce44-42e5-89fb-fbe379d92bd8\") " pod="openshift-ingress/e2e-api-a-75ee45-openshift-default-f4ffbb777-nqqb2" Jul 07 22:37:11.349707 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:11.349547 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/22d3bb9a-ce44-42e5-89fb-fbe379d92bd8-istio-token\") pod \"e2e-api-a-75ee45-openshift-default-f4ffbb777-nqqb2\" (UID: \"22d3bb9a-ce44-42e5-89fb-fbe379d92bd8\") " pod="openshift-ingress/e2e-api-a-75ee45-openshift-default-f4ffbb777-nqqb2" Jul 07 22:37:11.350007 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:11.349784 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/22d3bb9a-ce44-42e5-89fb-fbe379d92bd8-istio-data\") pod \"e2e-api-a-75ee45-openshift-default-f4ffbb777-nqqb2\" (UID: \"22d3bb9a-ce44-42e5-89fb-fbe379d92bd8\") " pod="openshift-ingress/e2e-api-a-75ee45-openshift-default-f4ffbb777-nqqb2" Jul 07 22:37:11.350130 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:11.350108 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/22d3bb9a-ce44-42e5-89fb-fbe379d92bd8-workload-socket\") pod \"e2e-api-a-75ee45-openshift-default-f4ffbb777-nqqb2\" (UID: \"22d3bb9a-ce44-42e5-89fb-fbe379d92bd8\") " pod="openshift-ingress/e2e-api-a-75ee45-openshift-default-f4ffbb777-nqqb2" Jul 07 22:37:11.350215 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:11.350163 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/22d3bb9a-ce44-42e5-89fb-fbe379d92bd8-workload-certs\") pod \"e2e-api-a-75ee45-openshift-default-f4ffbb777-nqqb2\" (UID: \"22d3bb9a-ce44-42e5-89fb-fbe379d92bd8\") " pod="openshift-ingress/e2e-api-a-75ee45-openshift-default-f4ffbb777-nqqb2" Jul 07 22:37:11.350266 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:11.350221 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/22d3bb9a-ce44-42e5-89fb-fbe379d92bd8-credential-socket\") pod \"e2e-api-a-75ee45-openshift-default-f4ffbb777-nqqb2\" (UID: \"22d3bb9a-ce44-42e5-89fb-fbe379d92bd8\") " pod="openshift-ingress/e2e-api-a-75ee45-openshift-default-f4ffbb777-nqqb2" Jul 07 22:37:11.350404 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:11.350386 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/22d3bb9a-ce44-42e5-89fb-fbe379d92bd8-istiod-ca-cert\") pod \"e2e-api-a-75ee45-openshift-default-f4ffbb777-nqqb2\" (UID: \"22d3bb9a-ce44-42e5-89fb-fbe379d92bd8\") " pod="openshift-ingress/e2e-api-a-75ee45-openshift-default-f4ffbb777-nqqb2" Jul 07 22:37:11.351848 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:11.351820 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/22d3bb9a-ce44-42e5-89fb-fbe379d92bd8-istio-podinfo\") pod \"e2e-api-a-75ee45-openshift-default-f4ffbb777-nqqb2\" (UID: \"22d3bb9a-ce44-42e5-89fb-fbe379d92bd8\") " pod="openshift-ingress/e2e-api-a-75ee45-openshift-default-f4ffbb777-nqqb2" Jul 07 22:37:11.351954 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:11.351932 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/22d3bb9a-ce44-42e5-89fb-fbe379d92bd8-istio-envoy\") pod \"e2e-api-a-75ee45-openshift-default-f4ffbb777-nqqb2\" (UID: \"22d3bb9a-ce44-42e5-89fb-fbe379d92bd8\") " pod="openshift-ingress/e2e-api-a-75ee45-openshift-default-f4ffbb777-nqqb2" Jul 07 22:37:11.364754 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:11.364728 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/22d3bb9a-ce44-42e5-89fb-fbe379d92bd8-istio-token\") pod \"e2e-api-a-75ee45-openshift-default-f4ffbb777-nqqb2\" (UID: \"22d3bb9a-ce44-42e5-89fb-fbe379d92bd8\") " pod="openshift-ingress/e2e-api-a-75ee45-openshift-default-f4ffbb777-nqqb2" Jul 07 22:37:11.364851 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:11.364790 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7tvp7\" (UniqueName: \"kubernetes.io/projected/22d3bb9a-ce44-42e5-89fb-fbe379d92bd8-kube-api-access-7tvp7\") pod \"e2e-api-a-75ee45-openshift-default-f4ffbb777-nqqb2\" (UID: \"22d3bb9a-ce44-42e5-89fb-fbe379d92bd8\") " pod="openshift-ingress/e2e-api-a-75ee45-openshift-default-f4ffbb777-nqqb2" Jul 07 22:37:11.437940 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:11.437851 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-75ee45-openshift-default-f4ffbb777-nqqb2" Jul 07 22:37:11.572567 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:11.572538 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-75ee45-openshift-default-f4ffbb777-nqqb2"] Jul 07 22:37:11.574243 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:37:11.574216 2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22d3bb9a_ce44_42e5_89fb_fbe379d92bd8.slice/crio-f066553fbd6d35345361d02370a93b1174fe2fe936359571c5789f4421fce4bf WatchSource:0}: Error finding container f066553fbd6d35345361d02370a93b1174fe2fe936359571c5789f4421fce4bf: Status 404 returned error can't find the container with id f066553fbd6d35345361d02370a93b1174fe2fe936359571c5789f4421fce4bf Jul 07 22:37:11.580250 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:11.580203 2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 22:37:11.580359 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:11.580286 2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 22:37:11.580359 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:11.580326 2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 22:37:11.795494 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:11.795418 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-75ee45-openshift-default-f4ffbb777-nqqb2" event={"ID":"22d3bb9a-ce44-42e5-89fb-fbe379d92bd8","Type":"ContainerStarted","Data":"efe6cedeed4ba7f251b41a3e5e79334b56277dcc48742c8196de1a2d05aa161c"} Jul 07 22:37:11.795694 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:11.795519 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-75ee45-openshift-default-f4ffbb777-nqqb2" event={"ID":"22d3bb9a-ce44-42e5-89fb-fbe379d92bd8","Type":"ContainerStarted","Data":"f066553fbd6d35345361d02370a93b1174fe2fe936359571c5789f4421fce4bf"} Jul 07 22:37:11.823436 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:11.823376 2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-75ee45-openshift-default-f4ffbb777-nqqb2" podStartSLOduration=0.823362392 podStartE2EDuration="823.362392ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 22:37:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:37:11.822224721 +0000 UTC m=+2014.136447742" watchObservedRunningTime="2026-07-07 22:37:11.823362392 +0000 UTC m=+2014.137585409" Jul 07 22:37:12.438444 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:12.438408 2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-75ee45-openshift-default-f4ffbb777-nqqb2" Jul 07 22:37:12.443631 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:12.443606 2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-75ee45-openshift-default-f4ffbb777-nqqb2" Jul 07 22:37:12.800009 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:12.799978 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-75ee45-openshift-default-f4ffbb777-nqqb2" Jul 07 22:37:12.801076 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:12.801052 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-75ee45-openshift-default-f4ffbb777-nqqb2" Jul 07 22:37:28.801013 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:28.800969 2545 patch_prober.go:28] interesting pod/e2e-api-a-75ee45-openshift-default-f4ffbb777-nqqb2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 07 22:37:28.801534 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:28.801047 2545 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-75ee45-openshift-default-f4ffbb777-nqqb2" podUID="22d3bb9a-ce44-42e5-89fb-fbe379d92bd8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 07 22:37:30.899642 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:30.899557 2545 generic.go:356] "Generic (PLEG): container finished" podID="22d3bb9a-ce44-42e5-89fb-fbe379d92bd8" containerID="efe6cedeed4ba7f251b41a3e5e79334b56277dcc48742c8196de1a2d05aa161c" exitCode=0 Jul 07 22:37:30.899642 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:30.899631 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-75ee45-openshift-default-f4ffbb777-nqqb2" event={"ID":"22d3bb9a-ce44-42e5-89fb-fbe379d92bd8","Type":"ContainerDied","Data":"efe6cedeed4ba7f251b41a3e5e79334b56277dcc48742c8196de1a2d05aa161c"} Jul 07 22:37:30.900056 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:30.900035 2545 scope.go:117] "RemoveContainer" containerID="efe6cedeed4ba7f251b41a3e5e79334b56277dcc48742c8196de1a2d05aa161c" Jul 07 22:37:30.901062 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:30.901035 2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 22:37:30.901160 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:30.901118 2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 22:37:30.901160 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:30.901150 2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 22:37:31.906040 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:31.906005 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-75ee45-openshift-default-f4ffbb777-nqqb2" event={"ID":"22d3bb9a-ce44-42e5-89fb-fbe379d92bd8","Type":"ContainerStarted","Data":"1a76b5e99b6938d65153f6192e7c20127fcfb775211b38641ebfb24157f04a27"} Jul 07 22:37:32.438264 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:32.438227 2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-75ee45-openshift-default-f4ffbb777-nqqb2" Jul 07 22:37:32.438427 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:32.438402 2545 patch_prober.go:28] interesting pod/e2e-api-a-75ee45-openshift-default-f4ffbb777-nqqb2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body= Jul 07 22:37:32.438485 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:32.438465 2545 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-75ee45-openshift-default-f4ffbb777-nqqb2" podUID="22d3bb9a-ce44-42e5-89fb-fbe379d92bd8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" Jul 07 22:37:33.438840 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:33.438808 2545 patch_prober.go:28] interesting pod/e2e-api-a-75ee45-openshift-default-f4ffbb777-nqqb2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body= Jul 07 22:37:33.439318 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:33.438862 2545 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-75ee45-openshift-default-f4ffbb777-nqqb2" podUID="22d3bb9a-ce44-42e5-89fb-fbe379d92bd8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" Jul 07 22:37:34.438903 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:34.438869 2545 patch_prober.go:28] interesting pod/e2e-api-a-75ee45-openshift-default-f4ffbb777-nqqb2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body= Jul 07 22:37:34.439519 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:34.438924 2545 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-75ee45-openshift-default-f4ffbb777-nqqb2" podUID="22d3bb9a-ce44-42e5-89fb-fbe379d92bd8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" Jul 07 22:37:35.438214 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:35.438180 2545 patch_prober.go:28] interesting pod/e2e-api-a-75ee45-openshift-default-f4ffbb777-nqqb2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body= Jul 07 22:37:35.438386 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:35.438236 2545 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-75ee45-openshift-default-f4ffbb777-nqqb2" podUID="22d3bb9a-ce44-42e5-89fb-fbe379d92bd8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" Jul 07 22:37:36.442295 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:36.442263 2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-75ee45-openshift-default-f4ffbb777-nqqb2" Jul 07 22:37:36.442688 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:36.442532 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-75ee45-openshift-default-f4ffbb777-nqqb2" Jul 07 22:37:36.443291 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:36.443268 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-75ee45-openshift-default-f4ffbb777-nqqb2" Jul 07 22:37:41.108595 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:41.108559 2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-75ee45-openshift-default-f4ffbb777-nqqb2"] Jul 07 22:37:41.109033 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:41.108785 2545 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-75ee45-openshift-default-f4ffbb777-nqqb2" podUID="22d3bb9a-ce44-42e5-89fb-fbe379d92bd8" containerName="istio-proxy" containerID="cri-o://1a76b5e99b6938d65153f6192e7c20127fcfb775211b38641ebfb24157f04a27" gracePeriod=30 Jul 07 22:37:46.360362 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:46.360337 2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-75ee45-openshift-default-f4ffbb777-nqqb2" Jul 07 22:37:46.373798 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:46.373737 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/22d3bb9a-ce44-42e5-89fb-fbe379d92bd8-istiod-ca-cert\") pod \"22d3bb9a-ce44-42e5-89fb-fbe379d92bd8\" (UID: \"22d3bb9a-ce44-42e5-89fb-fbe379d92bd8\") " Jul 07 22:37:46.373798 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:46.373785 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7tvp7\" (UniqueName: \"kubernetes.io/projected/22d3bb9a-ce44-42e5-89fb-fbe379d92bd8-kube-api-access-7tvp7\") pod \"22d3bb9a-ce44-42e5-89fb-fbe379d92bd8\" (UID: \"22d3bb9a-ce44-42e5-89fb-fbe379d92bd8\") " Jul 07 22:37:46.373953 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:46.373857 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/22d3bb9a-ce44-42e5-89fb-fbe379d92bd8-credential-socket\") pod \"22d3bb9a-ce44-42e5-89fb-fbe379d92bd8\" (UID: \"22d3bb9a-ce44-42e5-89fb-fbe379d92bd8\") " Jul 07 22:37:46.373953 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:46.373889 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/22d3bb9a-ce44-42e5-89fb-fbe379d92bd8-istio-data\") pod \"22d3bb9a-ce44-42e5-89fb-fbe379d92bd8\" (UID: \"22d3bb9a-ce44-42e5-89fb-fbe379d92bd8\") " Jul 07 22:37:46.373953 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:46.373918 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/22d3bb9a-ce44-42e5-89fb-fbe379d92bd8-istio-token\") pod \"22d3bb9a-ce44-42e5-89fb-fbe379d92bd8\" (UID: \"22d3bb9a-ce44-42e5-89fb-fbe379d92bd8\") " Jul 07 22:37:46.374138 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:46.373959 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/22d3bb9a-ce44-42e5-89fb-fbe379d92bd8-istio-envoy\") pod \"22d3bb9a-ce44-42e5-89fb-fbe379d92bd8\" (UID: \"22d3bb9a-ce44-42e5-89fb-fbe379d92bd8\") " Jul 07 22:37:46.374138 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:46.373996 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/22d3bb9a-ce44-42e5-89fb-fbe379d92bd8-istio-podinfo\") pod \"22d3bb9a-ce44-42e5-89fb-fbe379d92bd8\" (UID: \"22d3bb9a-ce44-42e5-89fb-fbe379d92bd8\") " Jul 07 22:37:46.374138 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:46.374026 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/22d3bb9a-ce44-42e5-89fb-fbe379d92bd8-workload-socket\") pod \"22d3bb9a-ce44-42e5-89fb-fbe379d92bd8\" (UID: \"22d3bb9a-ce44-42e5-89fb-fbe379d92bd8\") " Jul 07 22:37:46.374138 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:46.374072 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/22d3bb9a-ce44-42e5-89fb-fbe379d92bd8-workload-certs\") pod \"22d3bb9a-ce44-42e5-89fb-fbe379d92bd8\" (UID: \"22d3bb9a-ce44-42e5-89fb-fbe379d92bd8\") " Jul 07 22:37:46.374348 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:46.374190 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/22d3bb9a-ce44-42e5-89fb-fbe379d92bd8-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "22d3bb9a-ce44-42e5-89fb-fbe379d92bd8" (UID: "22d3bb9a-ce44-42e5-89fb-fbe379d92bd8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 22:37:46.374348 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:46.374207 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/22d3bb9a-ce44-42e5-89fb-fbe379d92bd8-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "22d3bb9a-ce44-42e5-89fb-fbe379d92bd8" (UID: "22d3bb9a-ce44-42e5-89fb-fbe379d92bd8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 22:37:46.374457 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:46.374421 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/22d3bb9a-ce44-42e5-89fb-fbe379d92bd8-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "22d3bb9a-ce44-42e5-89fb-fbe379d92bd8" (UID: "22d3bb9a-ce44-42e5-89fb-fbe379d92bd8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 22:37:46.374514 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:46.374487 2545 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/22d3bb9a-ce44-42e5-89fb-fbe379d92bd8-credential-socket\") on node \"ip-10-0-131-55.ec2.internal\" DevicePath \"\"" Jul 07 22:37:46.374514 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:46.374502 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/22d3bb9a-ce44-42e5-89fb-fbe379d92bd8-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "22d3bb9a-ce44-42e5-89fb-fbe379d92bd8" (UID: "22d3bb9a-ce44-42e5-89fb-fbe379d92bd8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 22:37:46.374635 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:46.374525 2545 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/22d3bb9a-ce44-42e5-89fb-fbe379d92bd8-istiod-ca-cert\") on node \"ip-10-0-131-55.ec2.internal\" DevicePath \"\"" Jul 07 22:37:46.374890 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:46.374863 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/22d3bb9a-ce44-42e5-89fb-fbe379d92bd8-istio-data" (OuterVolumeSpecName: "istio-data") pod "22d3bb9a-ce44-42e5-89fb-fbe379d92bd8" (UID: "22d3bb9a-ce44-42e5-89fb-fbe379d92bd8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 22:37:46.376311 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:46.376286 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/22d3bb9a-ce44-42e5-89fb-fbe379d92bd8-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "22d3bb9a-ce44-42e5-89fb-fbe379d92bd8" (UID: "22d3bb9a-ce44-42e5-89fb-fbe379d92bd8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 22:37:46.376409 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:46.376378 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/22d3bb9a-ce44-42e5-89fb-fbe379d92bd8-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "22d3bb9a-ce44-42e5-89fb-fbe379d92bd8" (UID: "22d3bb9a-ce44-42e5-89fb-fbe379d92bd8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 22:37:46.376473 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:46.376442 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/22d3bb9a-ce44-42e5-89fb-fbe379d92bd8-istio-token" (OuterVolumeSpecName: "istio-token") pod "22d3bb9a-ce44-42e5-89fb-fbe379d92bd8" (UID: "22d3bb9a-ce44-42e5-89fb-fbe379d92bd8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 22:37:46.376643 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:46.376620 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/22d3bb9a-ce44-42e5-89fb-fbe379d92bd8-kube-api-access-7tvp7" (OuterVolumeSpecName: "kube-api-access-7tvp7") pod "22d3bb9a-ce44-42e5-89fb-fbe379d92bd8" (UID: "22d3bb9a-ce44-42e5-89fb-fbe379d92bd8"). InnerVolumeSpecName "kube-api-access-7tvp7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 22:37:46.475377 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:46.475348 2545 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/22d3bb9a-ce44-42e5-89fb-fbe379d92bd8-istio-data\") on node \"ip-10-0-131-55.ec2.internal\" DevicePath \"\"" Jul 07 22:37:46.475377 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:46.475373 2545 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/22d3bb9a-ce44-42e5-89fb-fbe379d92bd8-istio-token\") on node \"ip-10-0-131-55.ec2.internal\" DevicePath \"\"" Jul 07 22:37:46.475377 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:46.475382 2545 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/22d3bb9a-ce44-42e5-89fb-fbe379d92bd8-istio-envoy\") on node \"ip-10-0-131-55.ec2.internal\" DevicePath \"\"" Jul 07 22:37:46.475600 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:46.475390 2545 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/22d3bb9a-ce44-42e5-89fb-fbe379d92bd8-istio-podinfo\") on node \"ip-10-0-131-55.ec2.internal\" DevicePath \"\"" Jul 07 22:37:46.475600 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:46.475398 2545 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/22d3bb9a-ce44-42e5-89fb-fbe379d92bd8-workload-socket\") on node \"ip-10-0-131-55.ec2.internal\" DevicePath \"\"" Jul 07 22:37:46.475600 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:46.475406 2545 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/22d3bb9a-ce44-42e5-89fb-fbe379d92bd8-workload-certs\") on node \"ip-10-0-131-55.ec2.internal\" DevicePath \"\"" Jul 07 22:37:46.475600 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:46.475414 2545 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7tvp7\" (UniqueName: \"kubernetes.io/projected/22d3bb9a-ce44-42e5-89fb-fbe379d92bd8-kube-api-access-7tvp7\") on node \"ip-10-0-131-55.ec2.internal\" DevicePath \"\"" Jul 07 22:37:46.978597 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:46.978560 2545 generic.go:356] "Generic (PLEG): container finished" podID="22d3bb9a-ce44-42e5-89fb-fbe379d92bd8" containerID="1a76b5e99b6938d65153f6192e7c20127fcfb775211b38641ebfb24157f04a27" exitCode=0 Jul 07 22:37:46.978770 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:46.978613 2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-75ee45-openshift-default-f4ffbb777-nqqb2" Jul 07 22:37:46.978770 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:46.978624 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-75ee45-openshift-default-f4ffbb777-nqqb2" event={"ID":"22d3bb9a-ce44-42e5-89fb-fbe379d92bd8","Type":"ContainerDied","Data":"1a76b5e99b6938d65153f6192e7c20127fcfb775211b38641ebfb24157f04a27"} Jul 07 22:37:46.978770 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:46.978666 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-75ee45-openshift-default-f4ffbb777-nqqb2" event={"ID":"22d3bb9a-ce44-42e5-89fb-fbe379d92bd8","Type":"ContainerDied","Data":"f066553fbd6d35345361d02370a93b1174fe2fe936359571c5789f4421fce4bf"} Jul 07 22:37:46.978770 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:46.978687 2545 scope.go:117] "RemoveContainer" containerID="1a76b5e99b6938d65153f6192e7c20127fcfb775211b38641ebfb24157f04a27" Jul 07 22:37:46.987442 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:46.987428 2545 scope.go:117] "RemoveContainer" containerID="efe6cedeed4ba7f251b41a3e5e79334b56277dcc48742c8196de1a2d05aa161c" Jul 07 22:37:46.995125 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:46.995107 2545 scope.go:117] "RemoveContainer" containerID="1a76b5e99b6938d65153f6192e7c20127fcfb775211b38641ebfb24157f04a27" Jul 07 22:37:46.995368 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:37:46.995349 2545 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1a76b5e99b6938d65153f6192e7c20127fcfb775211b38641ebfb24157f04a27\": container with ID starting with 1a76b5e99b6938d65153f6192e7c20127fcfb775211b38641ebfb24157f04a27 not found: ID does not exist" containerID="1a76b5e99b6938d65153f6192e7c20127fcfb775211b38641ebfb24157f04a27" Jul 07 22:37:46.995426 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:46.995375 2545 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1a76b5e99b6938d65153f6192e7c20127fcfb775211b38641ebfb24157f04a27"} err="failed to get container status \"1a76b5e99b6938d65153f6192e7c20127fcfb775211b38641ebfb24157f04a27\": rpc error: code = NotFound desc = could not find container \"1a76b5e99b6938d65153f6192e7c20127fcfb775211b38641ebfb24157f04a27\": container with ID starting with 1a76b5e99b6938d65153f6192e7c20127fcfb775211b38641ebfb24157f04a27 not found: ID does not exist" Jul 07 22:37:46.995426 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:46.995394 2545 scope.go:117] "RemoveContainer" containerID="efe6cedeed4ba7f251b41a3e5e79334b56277dcc48742c8196de1a2d05aa161c" Jul 07 22:37:46.995641 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:37:46.995624 2545 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"efe6cedeed4ba7f251b41a3e5e79334b56277dcc48742c8196de1a2d05aa161c\": container with ID starting with efe6cedeed4ba7f251b41a3e5e79334b56277dcc48742c8196de1a2d05aa161c not found: ID does not exist" containerID="efe6cedeed4ba7f251b41a3e5e79334b56277dcc48742c8196de1a2d05aa161c" Jul 07 22:37:46.995697 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:46.995650 2545 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"efe6cedeed4ba7f251b41a3e5e79334b56277dcc48742c8196de1a2d05aa161c"} err="failed to get container status \"efe6cedeed4ba7f251b41a3e5e79334b56277dcc48742c8196de1a2d05aa161c\": rpc error: code = NotFound desc = could not find container \"efe6cedeed4ba7f251b41a3e5e79334b56277dcc48742c8196de1a2d05aa161c\": container with ID starting with efe6cedeed4ba7f251b41a3e5e79334b56277dcc48742c8196de1a2d05aa161c not found: ID does not exist" Jul 07 22:37:47.002558 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:47.002535 2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-75ee45-openshift-default-f4ffbb777-nqqb2"] Jul 07 22:37:47.009471 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:47.009442 2545 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-75ee45-openshift-default-f4ffbb777-nqqb2"] Jul 07 22:37:48.312056 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:48.312025 2545 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="22d3bb9a-ce44-42e5-89fb-fbe379d92bd8" path="/var/lib/kubelet/pods/22d3bb9a-ce44-42e5-89fb-fbe379d92bd8/volumes" Jul 07 22:37:48.952055 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:48.952020 2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-41e952-openshift-default-bf4779887-95k56"] Jul 07 22:37:48.952427 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:48.952414 2545 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="22d3bb9a-ce44-42e5-89fb-fbe379d92bd8" containerName="istio-proxy" Jul 07 22:37:48.952477 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:48.952430 2545 state_mem.go:107] "Deleted CPUSet assignment" podUID="22d3bb9a-ce44-42e5-89fb-fbe379d92bd8" containerName="istio-proxy" Jul 07 22:37:48.952477 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:48.952443 2545 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="22d3bb9a-ce44-42e5-89fb-fbe379d92bd8" containerName="istio-proxy" Jul 07 22:37:48.952477 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:48.952448 2545 state_mem.go:107] "Deleted CPUSet assignment" podUID="22d3bb9a-ce44-42e5-89fb-fbe379d92bd8" containerName="istio-proxy" Jul 07 22:37:48.952590 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:48.952509 2545 memory_manager.go:368] "RemoveStaleState removing state" podUID="22d3bb9a-ce44-42e5-89fb-fbe379d92bd8" containerName="istio-proxy" Jul 07 22:37:48.952590 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:48.952518 2545 memory_manager.go:368] "RemoveStaleState removing state" podUID="22d3bb9a-ce44-42e5-89fb-fbe379d92bd8" containerName="istio-proxy" Jul 07 22:37:48.957064 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:48.957042 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-41e952-openshift-default-bf4779887-95k56" Jul 07 22:37:48.959405 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:48.959385 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 07 22:37:48.959541 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:48.959524 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-41e952-openshift-default-dockercfg-j5q6k\"" Jul 07 22:37:48.967957 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:48.967933 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-41e952-openshift-default-bf4779887-95k56"] Jul 07 22:37:48.999514 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:48.999482 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/18408c49-ad8e-48b6-97b3-dd10ca1565e5-istiod-ca-cert\") pod \"e2e-shared-a-41e952-openshift-default-bf4779887-95k56\" (UID: \"18408c49-ad8e-48b6-97b3-dd10ca1565e5\") " pod="openshift-ingress/e2e-shared-a-41e952-openshift-default-bf4779887-95k56" Jul 07 22:37:48.999635 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:48.999541 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/18408c49-ad8e-48b6-97b3-dd10ca1565e5-credential-socket\") pod \"e2e-shared-a-41e952-openshift-default-bf4779887-95k56\" (UID: \"18408c49-ad8e-48b6-97b3-dd10ca1565e5\") " pod="openshift-ingress/e2e-shared-a-41e952-openshift-default-bf4779887-95k56" Jul 07 22:37:48.999635 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:48.999566 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/18408c49-ad8e-48b6-97b3-dd10ca1565e5-workload-socket\") pod \"e2e-shared-a-41e952-openshift-default-bf4779887-95k56\" (UID: \"18408c49-ad8e-48b6-97b3-dd10ca1565e5\") " pod="openshift-ingress/e2e-shared-a-41e952-openshift-default-bf4779887-95k56" Jul 07 22:37:48.999635 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:48.999600 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/18408c49-ad8e-48b6-97b3-dd10ca1565e5-istio-envoy\") pod \"e2e-shared-a-41e952-openshift-default-bf4779887-95k56\" (UID: \"18408c49-ad8e-48b6-97b3-dd10ca1565e5\") " pod="openshift-ingress/e2e-shared-a-41e952-openshift-default-bf4779887-95k56" Jul 07 22:37:48.999635 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:48.999616 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/18408c49-ad8e-48b6-97b3-dd10ca1565e5-istio-token\") pod \"e2e-shared-a-41e952-openshift-default-bf4779887-95k56\" (UID: \"18408c49-ad8e-48b6-97b3-dd10ca1565e5\") " pod="openshift-ingress/e2e-shared-a-41e952-openshift-default-bf4779887-95k56" Jul 07 22:37:48.999777 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:48.999673 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/18408c49-ad8e-48b6-97b3-dd10ca1565e5-istio-podinfo\") pod \"e2e-shared-a-41e952-openshift-default-bf4779887-95k56\" (UID: \"18408c49-ad8e-48b6-97b3-dd10ca1565e5\") " pod="openshift-ingress/e2e-shared-a-41e952-openshift-default-bf4779887-95k56" Jul 07 22:37:48.999777 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:48.999734 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/18408c49-ad8e-48b6-97b3-dd10ca1565e5-workload-certs\") pod \"e2e-shared-a-41e952-openshift-default-bf4779887-95k56\" (UID: \"18408c49-ad8e-48b6-97b3-dd10ca1565e5\") " pod="openshift-ingress/e2e-shared-a-41e952-openshift-default-bf4779887-95k56" Jul 07 22:37:48.999777 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:48.999769 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wv6rn\" (UniqueName: \"kubernetes.io/projected/18408c49-ad8e-48b6-97b3-dd10ca1565e5-kube-api-access-wv6rn\") pod \"e2e-shared-a-41e952-openshift-default-bf4779887-95k56\" (UID: \"18408c49-ad8e-48b6-97b3-dd10ca1565e5\") " pod="openshift-ingress/e2e-shared-a-41e952-openshift-default-bf4779887-95k56" Jul 07 22:37:48.999871 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:48.999788 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/18408c49-ad8e-48b6-97b3-dd10ca1565e5-istio-data\") pod \"e2e-shared-a-41e952-openshift-default-bf4779887-95k56\" (UID: \"18408c49-ad8e-48b6-97b3-dd10ca1565e5\") " pod="openshift-ingress/e2e-shared-a-41e952-openshift-default-bf4779887-95k56" Jul 07 22:37:49.100730 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:49.100699 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/18408c49-ad8e-48b6-97b3-dd10ca1565e5-workload-certs\") pod \"e2e-shared-a-41e952-openshift-default-bf4779887-95k56\" (UID: \"18408c49-ad8e-48b6-97b3-dd10ca1565e5\") " pod="openshift-ingress/e2e-shared-a-41e952-openshift-default-bf4779887-95k56" Jul 07 22:37:49.100918 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:49.100742 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wv6rn\" (UniqueName: \"kubernetes.io/projected/18408c49-ad8e-48b6-97b3-dd10ca1565e5-kube-api-access-wv6rn\") pod \"e2e-shared-a-41e952-openshift-default-bf4779887-95k56\" (UID: \"18408c49-ad8e-48b6-97b3-dd10ca1565e5\") " pod="openshift-ingress/e2e-shared-a-41e952-openshift-default-bf4779887-95k56" Jul 07 22:37:49.100918 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:49.100761 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/18408c49-ad8e-48b6-97b3-dd10ca1565e5-istio-data\") pod \"e2e-shared-a-41e952-openshift-default-bf4779887-95k56\" (UID: \"18408c49-ad8e-48b6-97b3-dd10ca1565e5\") " pod="openshift-ingress/e2e-shared-a-41e952-openshift-default-bf4779887-95k56" Jul 07 22:37:49.100918 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:49.100793 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/18408c49-ad8e-48b6-97b3-dd10ca1565e5-istiod-ca-cert\") pod \"e2e-shared-a-41e952-openshift-default-bf4779887-95k56\" (UID: \"18408c49-ad8e-48b6-97b3-dd10ca1565e5\") " pod="openshift-ingress/e2e-shared-a-41e952-openshift-default-bf4779887-95k56" Jul 07 22:37:49.100918 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:49.100843 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/18408c49-ad8e-48b6-97b3-dd10ca1565e5-credential-socket\") pod \"e2e-shared-a-41e952-openshift-default-bf4779887-95k56\" (UID: \"18408c49-ad8e-48b6-97b3-dd10ca1565e5\") " pod="openshift-ingress/e2e-shared-a-41e952-openshift-default-bf4779887-95k56" Jul 07 22:37:49.100918 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:49.100876 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/18408c49-ad8e-48b6-97b3-dd10ca1565e5-workload-socket\") pod \"e2e-shared-a-41e952-openshift-default-bf4779887-95k56\" (UID: \"18408c49-ad8e-48b6-97b3-dd10ca1565e5\") " pod="openshift-ingress/e2e-shared-a-41e952-openshift-default-bf4779887-95k56" Jul 07 22:37:49.100918 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:49.100912 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/18408c49-ad8e-48b6-97b3-dd10ca1565e5-istio-envoy\") pod \"e2e-shared-a-41e952-openshift-default-bf4779887-95k56\" (UID: \"18408c49-ad8e-48b6-97b3-dd10ca1565e5\") " pod="openshift-ingress/e2e-shared-a-41e952-openshift-default-bf4779887-95k56" Jul 07 22:37:49.101270 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:49.100933 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/18408c49-ad8e-48b6-97b3-dd10ca1565e5-istio-token\") pod \"e2e-shared-a-41e952-openshift-default-bf4779887-95k56\" (UID: \"18408c49-ad8e-48b6-97b3-dd10ca1565e5\") " pod="openshift-ingress/e2e-shared-a-41e952-openshift-default-bf4779887-95k56" Jul 07 22:37:49.101270 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:49.100964 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/18408c49-ad8e-48b6-97b3-dd10ca1565e5-istio-podinfo\") pod \"e2e-shared-a-41e952-openshift-default-bf4779887-95k56\" (UID: \"18408c49-ad8e-48b6-97b3-dd10ca1565e5\") " pod="openshift-ingress/e2e-shared-a-41e952-openshift-default-bf4779887-95k56" Jul 07 22:37:49.101270 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:49.101169 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/18408c49-ad8e-48b6-97b3-dd10ca1565e5-workload-certs\") pod \"e2e-shared-a-41e952-openshift-default-bf4779887-95k56\" (UID: \"18408c49-ad8e-48b6-97b3-dd10ca1565e5\") " pod="openshift-ingress/e2e-shared-a-41e952-openshift-default-bf4779887-95k56" Jul 07 22:37:49.101377 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:49.101294 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/18408c49-ad8e-48b6-97b3-dd10ca1565e5-istio-data\") pod \"e2e-shared-a-41e952-openshift-default-bf4779887-95k56\" (UID: \"18408c49-ad8e-48b6-97b3-dd10ca1565e5\") " pod="openshift-ingress/e2e-shared-a-41e952-openshift-default-bf4779887-95k56" Jul 07 22:37:49.101377 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:49.101341 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/18408c49-ad8e-48b6-97b3-dd10ca1565e5-credential-socket\") pod \"e2e-shared-a-41e952-openshift-default-bf4779887-95k56\" (UID: \"18408c49-ad8e-48b6-97b3-dd10ca1565e5\") " pod="openshift-ingress/e2e-shared-a-41e952-openshift-default-bf4779887-95k56" Jul 07 22:37:49.101377 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:49.101359 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/18408c49-ad8e-48b6-97b3-dd10ca1565e5-workload-socket\") pod \"e2e-shared-a-41e952-openshift-default-bf4779887-95k56\" (UID: \"18408c49-ad8e-48b6-97b3-dd10ca1565e5\") " pod="openshift-ingress/e2e-shared-a-41e952-openshift-default-bf4779887-95k56" Jul 07 22:37:49.101553 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:49.101536 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/18408c49-ad8e-48b6-97b3-dd10ca1565e5-istiod-ca-cert\") pod \"e2e-shared-a-41e952-openshift-default-bf4779887-95k56\" (UID: \"18408c49-ad8e-48b6-97b3-dd10ca1565e5\") " pod="openshift-ingress/e2e-shared-a-41e952-openshift-default-bf4779887-95k56" Jul 07 22:37:49.103252 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:49.103233 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/18408c49-ad8e-48b6-97b3-dd10ca1565e5-istio-envoy\") pod \"e2e-shared-a-41e952-openshift-default-bf4779887-95k56\" (UID: \"18408c49-ad8e-48b6-97b3-dd10ca1565e5\") " pod="openshift-ingress/e2e-shared-a-41e952-openshift-default-bf4779887-95k56" Jul 07 22:37:49.103484 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:49.103463 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/18408c49-ad8e-48b6-97b3-dd10ca1565e5-istio-podinfo\") pod \"e2e-shared-a-41e952-openshift-default-bf4779887-95k56\" (UID: \"18408c49-ad8e-48b6-97b3-dd10ca1565e5\") " pod="openshift-ingress/e2e-shared-a-41e952-openshift-default-bf4779887-95k56" Jul 07 22:37:49.108446 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:49.108423 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/18408c49-ad8e-48b6-97b3-dd10ca1565e5-istio-token\") pod \"e2e-shared-a-41e952-openshift-default-bf4779887-95k56\" (UID: \"18408c49-ad8e-48b6-97b3-dd10ca1565e5\") " pod="openshift-ingress/e2e-shared-a-41e952-openshift-default-bf4779887-95k56" Jul 07 22:37:49.108722 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:49.108698 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wv6rn\" (UniqueName: \"kubernetes.io/projected/18408c49-ad8e-48b6-97b3-dd10ca1565e5-kube-api-access-wv6rn\") pod \"e2e-shared-a-41e952-openshift-default-bf4779887-95k56\" (UID: \"18408c49-ad8e-48b6-97b3-dd10ca1565e5\") " pod="openshift-ingress/e2e-shared-a-41e952-openshift-default-bf4779887-95k56" Jul 07 22:37:49.268064 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:49.268029 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-41e952-openshift-default-bf4779887-95k56" Jul 07 22:37:49.402578 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:49.402550 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-41e952-openshift-default-bf4779887-95k56"] Jul 07 22:37:49.404753 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:37:49.404726 2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18408c49_ad8e_48b6_97b3_dd10ca1565e5.slice/crio-3c4a3524de43a00cbe75687655124965d7ef97ecb560dccc5ef085674ab10fb5 WatchSource:0}: Error finding container 3c4a3524de43a00cbe75687655124965d7ef97ecb560dccc5ef085674ab10fb5: Status 404 returned error can't find the container with id 3c4a3524de43a00cbe75687655124965d7ef97ecb560dccc5ef085674ab10fb5 Jul 07 22:37:49.406999 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:49.406962 2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 22:37:49.407064 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:49.407047 2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 22:37:49.407145 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:49.407127 2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 22:37:49.995797 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:49.995757 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-41e952-openshift-default-bf4779887-95k56" event={"ID":"18408c49-ad8e-48b6-97b3-dd10ca1565e5","Type":"ContainerStarted","Data":"57cd81887d6c748d26fca3aaa75406bce5ab4b963cdd3680cc22939bd6b3b167"} Jul 07 22:37:49.996018 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:49.995817 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-41e952-openshift-default-bf4779887-95k56" event={"ID":"18408c49-ad8e-48b6-97b3-dd10ca1565e5","Type":"ContainerStarted","Data":"3c4a3524de43a00cbe75687655124965d7ef97ecb560dccc5ef085674ab10fb5"} Jul 07 22:37:50.016749 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:50.016692 2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-41e952-openshift-default-bf4779887-95k56" podStartSLOduration=2.016677715 podStartE2EDuration="2.016677715s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 22:37:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:37:50.014075604 +0000 UTC m=+2052.328298648" watchObservedRunningTime="2026-07-07 22:37:50.016677715 +0000 UTC m=+2052.330900734" Jul 07 22:37:50.268331 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:50.268226 2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-41e952-openshift-default-bf4779887-95k56" Jul 07 22:37:50.273043 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:50.273018 2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-41e952-openshift-default-bf4779887-95k56" Jul 07 22:37:51.000794 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:51.000766 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-41e952-openshift-default-bf4779887-95k56" Jul 07 22:37:51.001875 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:51.001858 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-41e952-openshift-default-bf4779887-95k56" Jul 07 22:37:56.978810 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:56.978773 2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-a4bfe6-openshift-default-7f69fb97f-hqxx5"] Jul 07 22:37:56.982726 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:56.982702 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-a4bfe6-openshift-default-7f69fb97f-hqxx5" Jul 07 22:37:56.985017 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:56.984990 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-a4bfe6-openshift-default-dockercfg-xhtbg\"" Jul 07 22:37:56.995582 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:56.995560 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-a4bfe6-openshift-default-7f69fb97f-hqxx5"] Jul 07 22:37:57.075010 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:57.074971 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b15b945e-9940-4332-8f1a-064a4258b101-workload-certs\") pod \"e2e-shared-b-a4bfe6-openshift-default-7f69fb97f-hqxx5\" (UID: \"b15b945e-9940-4332-8f1a-064a4258b101\") " pod="openshift-ingress/e2e-shared-b-a4bfe6-openshift-default-7f69fb97f-hqxx5" Jul 07 22:37:57.075195 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:57.075020 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b15b945e-9940-4332-8f1a-064a4258b101-istio-data\") pod \"e2e-shared-b-a4bfe6-openshift-default-7f69fb97f-hqxx5\" (UID: \"b15b945e-9940-4332-8f1a-064a4258b101\") " pod="openshift-ingress/e2e-shared-b-a4bfe6-openshift-default-7f69fb97f-hqxx5" Jul 07 22:37:57.075195 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:57.075116 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b15b945e-9940-4332-8f1a-064a4258b101-istio-envoy\") pod \"e2e-shared-b-a4bfe6-openshift-default-7f69fb97f-hqxx5\" (UID: \"b15b945e-9940-4332-8f1a-064a4258b101\") " pod="openshift-ingress/e2e-shared-b-a4bfe6-openshift-default-7f69fb97f-hqxx5" Jul 07 22:37:57.075195 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:57.075145 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b15b945e-9940-4332-8f1a-064a4258b101-istio-podinfo\") pod \"e2e-shared-b-a4bfe6-openshift-default-7f69fb97f-hqxx5\" (UID: \"b15b945e-9940-4332-8f1a-064a4258b101\") " pod="openshift-ingress/e2e-shared-b-a4bfe6-openshift-default-7f69fb97f-hqxx5" Jul 07 22:37:57.075195 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:57.075184 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b15b945e-9940-4332-8f1a-064a4258b101-workload-socket\") pod \"e2e-shared-b-a4bfe6-openshift-default-7f69fb97f-hqxx5\" (UID: \"b15b945e-9940-4332-8f1a-064a4258b101\") " pod="openshift-ingress/e2e-shared-b-a4bfe6-openshift-default-7f69fb97f-hqxx5" Jul 07 22:37:57.075424 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:57.075260 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b15b945e-9940-4332-8f1a-064a4258b101-istio-token\") pod \"e2e-shared-b-a4bfe6-openshift-default-7f69fb97f-hqxx5\" (UID: \"b15b945e-9940-4332-8f1a-064a4258b101\") " pod="openshift-ingress/e2e-shared-b-a4bfe6-openshift-default-7f69fb97f-hqxx5" Jul 07 22:37:57.075424 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:57.075310 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b15b945e-9940-4332-8f1a-064a4258b101-istiod-ca-cert\") pod \"e2e-shared-b-a4bfe6-openshift-default-7f69fb97f-hqxx5\" (UID: \"b15b945e-9940-4332-8f1a-064a4258b101\") " pod="openshift-ingress/e2e-shared-b-a4bfe6-openshift-default-7f69fb97f-hqxx5" Jul 07 22:37:57.075424 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:57.075361 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b15b945e-9940-4332-8f1a-064a4258b101-credential-socket\") pod \"e2e-shared-b-a4bfe6-openshift-default-7f69fb97f-hqxx5\" (UID: \"b15b945e-9940-4332-8f1a-064a4258b101\") " pod="openshift-ingress/e2e-shared-b-a4bfe6-openshift-default-7f69fb97f-hqxx5" Jul 07 22:37:57.075424 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:57.075411 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-trm5j\" (UniqueName: \"kubernetes.io/projected/b15b945e-9940-4332-8f1a-064a4258b101-kube-api-access-trm5j\") pod \"e2e-shared-b-a4bfe6-openshift-default-7f69fb97f-hqxx5\" (UID: \"b15b945e-9940-4332-8f1a-064a4258b101\") " pod="openshift-ingress/e2e-shared-b-a4bfe6-openshift-default-7f69fb97f-hqxx5" Jul 07 22:37:57.176477 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:57.176440 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b15b945e-9940-4332-8f1a-064a4258b101-istio-envoy\") pod \"e2e-shared-b-a4bfe6-openshift-default-7f69fb97f-hqxx5\" (UID: \"b15b945e-9940-4332-8f1a-064a4258b101\") " pod="openshift-ingress/e2e-shared-b-a4bfe6-openshift-default-7f69fb97f-hqxx5" Jul 07 22:37:57.176477 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:57.176477 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b15b945e-9940-4332-8f1a-064a4258b101-istio-podinfo\") pod \"e2e-shared-b-a4bfe6-openshift-default-7f69fb97f-hqxx5\" (UID: \"b15b945e-9940-4332-8f1a-064a4258b101\") " pod="openshift-ingress/e2e-shared-b-a4bfe6-openshift-default-7f69fb97f-hqxx5" Jul 07 22:37:57.176728 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:57.176519 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b15b945e-9940-4332-8f1a-064a4258b101-workload-socket\") pod \"e2e-shared-b-a4bfe6-openshift-default-7f69fb97f-hqxx5\" (UID: \"b15b945e-9940-4332-8f1a-064a4258b101\") " pod="openshift-ingress/e2e-shared-b-a4bfe6-openshift-default-7f69fb97f-hqxx5" Jul 07 22:37:57.176728 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:57.176555 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b15b945e-9940-4332-8f1a-064a4258b101-istio-token\") pod \"e2e-shared-b-a4bfe6-openshift-default-7f69fb97f-hqxx5\" (UID: \"b15b945e-9940-4332-8f1a-064a4258b101\") " pod="openshift-ingress/e2e-shared-b-a4bfe6-openshift-default-7f69fb97f-hqxx5" Jul 07 22:37:57.176728 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:57.176594 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b15b945e-9940-4332-8f1a-064a4258b101-istiod-ca-cert\") pod \"e2e-shared-b-a4bfe6-openshift-default-7f69fb97f-hqxx5\" (UID: \"b15b945e-9940-4332-8f1a-064a4258b101\") " pod="openshift-ingress/e2e-shared-b-a4bfe6-openshift-default-7f69fb97f-hqxx5" Jul 07 22:37:57.176895 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:57.176728 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b15b945e-9940-4332-8f1a-064a4258b101-credential-socket\") pod \"e2e-shared-b-a4bfe6-openshift-default-7f69fb97f-hqxx5\" (UID: \"b15b945e-9940-4332-8f1a-064a4258b101\") " pod="openshift-ingress/e2e-shared-b-a4bfe6-openshift-default-7f69fb97f-hqxx5" Jul 07 22:37:57.176895 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:57.176810 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-trm5j\" (UniqueName: \"kubernetes.io/projected/b15b945e-9940-4332-8f1a-064a4258b101-kube-api-access-trm5j\") pod \"e2e-shared-b-a4bfe6-openshift-default-7f69fb97f-hqxx5\" (UID: \"b15b945e-9940-4332-8f1a-064a4258b101\") " pod="openshift-ingress/e2e-shared-b-a4bfe6-openshift-default-7f69fb97f-hqxx5" Jul 07 22:37:57.176895 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:57.176887 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b15b945e-9940-4332-8f1a-064a4258b101-workload-certs\") pod \"e2e-shared-b-a4bfe6-openshift-default-7f69fb97f-hqxx5\" (UID: \"b15b945e-9940-4332-8f1a-064a4258b101\") " pod="openshift-ingress/e2e-shared-b-a4bfe6-openshift-default-7f69fb97f-hqxx5" Jul 07 22:37:57.177066 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:57.176912 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b15b945e-9940-4332-8f1a-064a4258b101-istio-data\") pod \"e2e-shared-b-a4bfe6-openshift-default-7f69fb97f-hqxx5\" (UID: \"b15b945e-9940-4332-8f1a-064a4258b101\") " pod="openshift-ingress/e2e-shared-b-a4bfe6-openshift-default-7f69fb97f-hqxx5" Jul 07 22:37:57.177066 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:57.176961 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b15b945e-9940-4332-8f1a-064a4258b101-workload-socket\") pod \"e2e-shared-b-a4bfe6-openshift-default-7f69fb97f-hqxx5\" (UID: \"b15b945e-9940-4332-8f1a-064a4258b101\") " pod="openshift-ingress/e2e-shared-b-a4bfe6-openshift-default-7f69fb97f-hqxx5" Jul 07 22:37:57.177066 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:57.177037 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b15b945e-9940-4332-8f1a-064a4258b101-credential-socket\") pod \"e2e-shared-b-a4bfe6-openshift-default-7f69fb97f-hqxx5\" (UID: \"b15b945e-9940-4332-8f1a-064a4258b101\") " pod="openshift-ingress/e2e-shared-b-a4bfe6-openshift-default-7f69fb97f-hqxx5" Jul 07 22:37:57.177309 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:57.177276 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b15b945e-9940-4332-8f1a-064a4258b101-istio-data\") pod \"e2e-shared-b-a4bfe6-openshift-default-7f69fb97f-hqxx5\" (UID: \"b15b945e-9940-4332-8f1a-064a4258b101\") " pod="openshift-ingress/e2e-shared-b-a4bfe6-openshift-default-7f69fb97f-hqxx5" Jul 07 22:37:57.177445 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:57.177422 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b15b945e-9940-4332-8f1a-064a4258b101-workload-certs\") pod \"e2e-shared-b-a4bfe6-openshift-default-7f69fb97f-hqxx5\" (UID: \"b15b945e-9940-4332-8f1a-064a4258b101\") " pod="openshift-ingress/e2e-shared-b-a4bfe6-openshift-default-7f69fb97f-hqxx5" Jul 07 22:37:57.177542 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:57.177439 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b15b945e-9940-4332-8f1a-064a4258b101-istiod-ca-cert\") pod \"e2e-shared-b-a4bfe6-openshift-default-7f69fb97f-hqxx5\" (UID: \"b15b945e-9940-4332-8f1a-064a4258b101\") " pod="openshift-ingress/e2e-shared-b-a4bfe6-openshift-default-7f69fb97f-hqxx5" Jul 07 22:37:57.179034 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:57.179009 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b15b945e-9940-4332-8f1a-064a4258b101-istio-envoy\") pod \"e2e-shared-b-a4bfe6-openshift-default-7f69fb97f-hqxx5\" (UID: \"b15b945e-9940-4332-8f1a-064a4258b101\") " pod="openshift-ingress/e2e-shared-b-a4bfe6-openshift-default-7f69fb97f-hqxx5" Jul 07 22:37:57.179407 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:57.179387 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b15b945e-9940-4332-8f1a-064a4258b101-istio-podinfo\") pod \"e2e-shared-b-a4bfe6-openshift-default-7f69fb97f-hqxx5\" (UID: \"b15b945e-9940-4332-8f1a-064a4258b101\") " pod="openshift-ingress/e2e-shared-b-a4bfe6-openshift-default-7f69fb97f-hqxx5" Jul 07 22:37:57.185231 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:57.185203 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b15b945e-9940-4332-8f1a-064a4258b101-istio-token\") pod \"e2e-shared-b-a4bfe6-openshift-default-7f69fb97f-hqxx5\" (UID: \"b15b945e-9940-4332-8f1a-064a4258b101\") " pod="openshift-ingress/e2e-shared-b-a4bfe6-openshift-default-7f69fb97f-hqxx5" Jul 07 22:37:57.185448 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:57.185433 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-trm5j\" (UniqueName: \"kubernetes.io/projected/b15b945e-9940-4332-8f1a-064a4258b101-kube-api-access-trm5j\") pod \"e2e-shared-b-a4bfe6-openshift-default-7f69fb97f-hqxx5\" (UID: \"b15b945e-9940-4332-8f1a-064a4258b101\") " pod="openshift-ingress/e2e-shared-b-a4bfe6-openshift-default-7f69fb97f-hqxx5" Jul 07 22:37:57.294265 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:57.294179 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-a4bfe6-openshift-default-7f69fb97f-hqxx5" Jul 07 22:37:57.431979 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:57.431850 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-a4bfe6-openshift-default-7f69fb97f-hqxx5"] Jul 07 22:37:57.434357 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:37:57.434317 2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb15b945e_9940_4332_8f1a_064a4258b101.slice/crio-9c35f23d703826561abd1f32a309b600f6470f27a44792d7170da9f91020bce6 WatchSource:0}: Error finding container 9c35f23d703826561abd1f32a309b600f6470f27a44792d7170da9f91020bce6: Status 404 returned error can't find the container with id 9c35f23d703826561abd1f32a309b600f6470f27a44792d7170da9f91020bce6 Jul 07 22:37:57.436731 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:57.436701 2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 22:37:57.436790 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:57.436768 2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 22:37:57.436836 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:57.436796 2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 22:37:58.034125 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:58.034072 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-a4bfe6-openshift-default-7f69fb97f-hqxx5" event={"ID":"b15b945e-9940-4332-8f1a-064a4258b101","Type":"ContainerStarted","Data":"3255f5daa9322957268313f2f4613ce51636e4c96582468b3fa55511cf798c75"} Jul 07 22:37:58.034625 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:58.034145 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-a4bfe6-openshift-default-7f69fb97f-hqxx5" event={"ID":"b15b945e-9940-4332-8f1a-064a4258b101","Type":"ContainerStarted","Data":"9c35f23d703826561abd1f32a309b600f6470f27a44792d7170da9f91020bce6"} Jul 07 22:37:58.053730 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:58.053676 2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-a4bfe6-openshift-default-7f69fb97f-hqxx5" podStartSLOduration=2.053661504 podStartE2EDuration="2.053661504s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 22:37:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:37:58.052332858 +0000 UTC m=+2060.366555878" watchObservedRunningTime="2026-07-07 22:37:58.053661504 +0000 UTC m=+2060.367884524" Jul 07 22:37:58.295317 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:58.295230 2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-a4bfe6-openshift-default-7f69fb97f-hqxx5" Jul 07 22:37:58.300018 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:58.299992 2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-a4bfe6-openshift-default-7f69fb97f-hqxx5" Jul 07 22:37:59.039355 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:59.039321 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-a4bfe6-openshift-default-7f69fb97f-hqxx5" Jul 07 22:37:59.044211 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:37:59.044176 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-a4bfe6-openshift-default-7f69fb97f-hqxx5" Jul 07 22:38:06.001909 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:38:06.001867 2545 patch_prober.go:28] interesting pod/e2e-shared-a-41e952-openshift-default-bf4779887-95k56 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body= Jul 07 22:38:06.002366 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:38:06.001957 2545 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-41e952-openshift-default-bf4779887-95k56" podUID="18408c49-ad8e-48b6-97b3-dd10ca1565e5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" Jul 07 22:38:06.082362 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:38:06.082277 2545 generic.go:356] "Generic (PLEG): container finished" podID="18408c49-ad8e-48b6-97b3-dd10ca1565e5" containerID="57cd81887d6c748d26fca3aaa75406bce5ab4b963cdd3680cc22939bd6b3b167" exitCode=0 Jul 07 22:38:06.082362 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:38:06.082327 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-41e952-openshift-default-bf4779887-95k56" event={"ID":"18408c49-ad8e-48b6-97b3-dd10ca1565e5","Type":"ContainerDied","Data":"57cd81887d6c748d26fca3aaa75406bce5ab4b963cdd3680cc22939bd6b3b167"} Jul 07 22:38:06.082783 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:38:06.082763 2545 scope.go:117] "RemoveContainer" containerID="57cd81887d6c748d26fca3aaa75406bce5ab4b963cdd3680cc22939bd6b3b167" Jul 07 22:38:06.083797 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:38:06.083769 2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 22:38:06.083869 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:38:06.083855 2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 22:38:06.083910 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:38:06.083896 2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 22:38:07.092499 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:38:07.092459 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-41e952-openshift-default-bf4779887-95k56" event={"ID":"18408c49-ad8e-48b6-97b3-dd10ca1565e5","Type":"ContainerStarted","Data":"f73cbc580da5623177ef47d7a9dc6407d2a9656bea61f736d5042faa4cfd1b09"} Jul 07 22:38:07.268742 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:38:07.268686 2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-41e952-openshift-default-bf4779887-95k56" Jul 07 22:38:07.269173 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:38:07.269130 2545 patch_prober.go:28] interesting pod/e2e-shared-a-41e952-openshift-default-bf4779887-95k56 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body= Jul 07 22:38:07.269313 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:38:07.269211 2545 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-41e952-openshift-default-bf4779887-95k56" podUID="18408c49-ad8e-48b6-97b3-dd10ca1565e5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" Jul 07 22:38:08.269030 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:38:08.268991 2545 patch_prober.go:28] interesting pod/e2e-shared-a-41e952-openshift-default-bf4779887-95k56 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body= Jul 07 22:38:08.269519 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:38:08.269062 2545 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-41e952-openshift-default-bf4779887-95k56" podUID="18408c49-ad8e-48b6-97b3-dd10ca1565e5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" Jul 07 22:38:09.269573 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:38:09.269525 2545 patch_prober.go:28] interesting pod/e2e-shared-a-41e952-openshift-default-bf4779887-95k56 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body= Jul 07 22:38:09.270067 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:38:09.269626 2545 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-41e952-openshift-default-bf4779887-95k56" podUID="18408c49-ad8e-48b6-97b3-dd10ca1565e5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" Jul 07 22:38:10.268773 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:38:10.268733 2545 patch_prober.go:28] interesting pod/e2e-shared-a-41e952-openshift-default-bf4779887-95k56 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body= Jul 07 22:38:10.268942 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:38:10.268794 2545 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-41e952-openshift-default-bf4779887-95k56" podUID="18408c49-ad8e-48b6-97b3-dd10ca1565e5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" Jul 07 22:38:11.272650 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:38:11.272624 2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-41e952-openshift-default-bf4779887-95k56" Jul 07 22:38:11.273016 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:38:11.272972 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-41e952-openshift-default-bf4779887-95k56" Jul 07 22:38:11.273668 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:38:11.273651 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-41e952-openshift-default-bf4779887-95k56" Jul 07 22:38:12.119928 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:38:12.119898 2545 generic.go:356] "Generic (PLEG): container finished" podID="b15b945e-9940-4332-8f1a-064a4258b101" containerID="3255f5daa9322957268313f2f4613ce51636e4c96582468b3fa55511cf798c75" exitCode=0 Jul 07 22:38:12.120165 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:38:12.119982 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-a4bfe6-openshift-default-7f69fb97f-hqxx5" event={"ID":"b15b945e-9940-4332-8f1a-064a4258b101","Type":"ContainerDied","Data":"3255f5daa9322957268313f2f4613ce51636e4c96582468b3fa55511cf798c75"} Jul 07 22:38:12.120514 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:38:12.120496 2545 scope.go:117] "RemoveContainer" containerID="3255f5daa9322957268313f2f4613ce51636e4c96582468b3fa55511cf798c75" Jul 07 22:38:12.121476 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:38:12.121447 2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 22:38:12.121530 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:38:12.121516 2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 22:38:12.121566 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:38:12.121546 2545 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 07 22:38:13.127013 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:38:13.126978 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-a4bfe6-openshift-default-7f69fb97f-hqxx5" event={"ID":"b15b945e-9940-4332-8f1a-064a4258b101","Type":"ContainerStarted","Data":"7f2f4a5df5fe59e4107843bd89e5922310564f7fb3adf7888000de69551a106f"} Jul 07 22:38:13.295410 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:38:13.295374 2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-a4bfe6-openshift-default-7f69fb97f-hqxx5" Jul 07 22:38:13.296578 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:38:13.296550 2545 patch_prober.go:28] interesting pod/e2e-shared-b-a4bfe6-openshift-default-7f69fb97f-hqxx5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body= Jul 07 22:38:13.296704 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:38:13.296607 2545 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-a4bfe6-openshift-default-7f69fb97f-hqxx5" podUID="b15b945e-9940-4332-8f1a-064a4258b101" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" Jul 07 22:38:14.039835 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:38:14.039781 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-a4bfe6-openshift-default-7f69fb97f-hqxx5" Jul 07 22:38:14.295123 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:38:14.295006 2545 patch_prober.go:28] interesting pod/e2e-shared-b-a4bfe6-openshift-default-7f69fb97f-hqxx5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body= Jul 07 22:38:14.295123 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:38:14.295063 2545 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-a4bfe6-openshift-default-7f69fb97f-hqxx5" podUID="b15b945e-9940-4332-8f1a-064a4258b101" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" Jul 07 22:38:15.294557 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:38:15.294522 2545 patch_prober.go:28] interesting pod/e2e-shared-b-a4bfe6-openshift-default-7f69fb97f-hqxx5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body= Jul 07 22:38:15.294751 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:38:15.294581 2545 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-a4bfe6-openshift-default-7f69fb97f-hqxx5" podUID="b15b945e-9940-4332-8f1a-064a4258b101" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" Jul 07 22:38:16.294585 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:38:16.294547 2545 patch_prober.go:28] interesting pod/e2e-shared-b-a4bfe6-openshift-default-7f69fb97f-hqxx5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body= Jul 07 22:38:16.295005 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:38:16.294605 2545 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-a4bfe6-openshift-default-7f69fb97f-hqxx5" podUID="b15b945e-9940-4332-8f1a-064a4258b101" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" Jul 07 22:38:17.299761 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:38:17.299725 2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-a4bfe6-openshift-default-7f69fb97f-hqxx5" Jul 07 22:38:17.300739 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:38:17.300714 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-a4bfe6-openshift-default-7f69fb97f-hqxx5" Jul 07 22:38:38.315445 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:38:38.315415 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-rhg5r_71f36bfa-4b8a-4572-b522-57c16392aa0b/console-operator/1.log" Jul 07 22:38:38.318545 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:38:38.318523 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-rhg5r_71f36bfa-4b8a-4572-b522-57c16392aa0b/console-operator/1.log" Jul 07 22:38:51.116147 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:38:51.116108 2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-a4bfe6/auth-test-model-a4bfe6-kserve-665bc74f5f-rj7z5"] Jul 07 22:38:51.119718 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:38:51.119690 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-a4bfe6/auth-test-model-a4bfe6-kserve-665bc74f5f-rj7z5" Jul 07 22:38:51.122257 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:38:51.122224 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-a4bfe6\"/\"openshift-service-ca.crt\"" Jul 07 22:38:51.123393 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:38:51.123362 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-a4bfe6\"/\"kube-root-ca.crt\"" Jul 07 22:38:51.123502 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:38:51.123437 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-a4bfe6\"/\"auth-test-model-a4bfe6-kserve-self-signed-certs\"" Jul 07 22:38:51.123574 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:38:51.123495 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-a4bfe6\"/\"default-dockercfg-26nnx\"" Jul 07 22:38:51.129375 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:38:51.129322 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-a4bfe6/auth-test-model-a4bfe6-kserve-665bc74f5f-rj7z5"] Jul 07 22:38:51.215314 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:38:51.215278 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8hrf4\" (UniqueName: \"kubernetes.io/projected/e3d08539-a6af-4015-94e9-5697df7593d0-kube-api-access-8hrf4\") pod \"auth-test-model-a4bfe6-kserve-665bc74f5f-rj7z5\" (UID: \"e3d08539-a6af-4015-94e9-5697df7593d0\") " pod="ai-tenant-e2e-shared-b-a4bfe6/auth-test-model-a4bfe6-kserve-665bc74f5f-rj7z5" Jul 07 22:38:51.215501 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:38:51.215410 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e3d08539-a6af-4015-94e9-5697df7593d0-model-cache\") pod \"auth-test-model-a4bfe6-kserve-665bc74f5f-rj7z5\" (UID: \"e3d08539-a6af-4015-94e9-5697df7593d0\") " pod="ai-tenant-e2e-shared-b-a4bfe6/auth-test-model-a4bfe6-kserve-665bc74f5f-rj7z5" Jul 07 22:38:51.215501 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:38:51.215447 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e3d08539-a6af-4015-94e9-5697df7593d0-dshm\") pod \"auth-test-model-a4bfe6-kserve-665bc74f5f-rj7z5\" (UID: \"e3d08539-a6af-4015-94e9-5697df7593d0\") " pod="ai-tenant-e2e-shared-b-a4bfe6/auth-test-model-a4bfe6-kserve-665bc74f5f-rj7z5" Jul 07 22:38:51.215501 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:38:51.215462 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e3d08539-a6af-4015-94e9-5697df7593d0-tmp-dir\") pod \"auth-test-model-a4bfe6-kserve-665bc74f5f-rj7z5\" (UID: \"e3d08539-a6af-4015-94e9-5697df7593d0\") " pod="ai-tenant-e2e-shared-b-a4bfe6/auth-test-model-a4bfe6-kserve-665bc74f5f-rj7z5" Jul 07 22:38:51.215501 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:38:51.215483 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e3d08539-a6af-4015-94e9-5697df7593d0-home\") pod \"auth-test-model-a4bfe6-kserve-665bc74f5f-rj7z5\" (UID: \"e3d08539-a6af-4015-94e9-5697df7593d0\") " pod="ai-tenant-e2e-shared-b-a4bfe6/auth-test-model-a4bfe6-kserve-665bc74f5f-rj7z5" Jul 07 22:38:51.215646 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:38:51.215545 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e3d08539-a6af-4015-94e9-5697df7593d0-tls-certs\") pod \"auth-test-model-a4bfe6-kserve-665bc74f5f-rj7z5\" (UID: \"e3d08539-a6af-4015-94e9-5697df7593d0\") " pod="ai-tenant-e2e-shared-b-a4bfe6/auth-test-model-a4bfe6-kserve-665bc74f5f-rj7z5" Jul 07 22:38:51.316036 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:38:51.316004 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8hrf4\" (UniqueName: \"kubernetes.io/projected/e3d08539-a6af-4015-94e9-5697df7593d0-kube-api-access-8hrf4\") pod \"auth-test-model-a4bfe6-kserve-665bc74f5f-rj7z5\" (UID: \"e3d08539-a6af-4015-94e9-5697df7593d0\") " pod="ai-tenant-e2e-shared-b-a4bfe6/auth-test-model-a4bfe6-kserve-665bc74f5f-rj7z5" Jul 07 22:38:51.316224 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:38:51.316111 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e3d08539-a6af-4015-94e9-5697df7593d0-model-cache\") pod \"auth-test-model-a4bfe6-kserve-665bc74f5f-rj7z5\" (UID: \"e3d08539-a6af-4015-94e9-5697df7593d0\") " pod="ai-tenant-e2e-shared-b-a4bfe6/auth-test-model-a4bfe6-kserve-665bc74f5f-rj7z5" Jul 07 22:38:51.316224 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:38:51.316141 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e3d08539-a6af-4015-94e9-5697df7593d0-dshm\") pod \"auth-test-model-a4bfe6-kserve-665bc74f5f-rj7z5\" (UID: \"e3d08539-a6af-4015-94e9-5697df7593d0\") " pod="ai-tenant-e2e-shared-b-a4bfe6/auth-test-model-a4bfe6-kserve-665bc74f5f-rj7z5" Jul 07 22:38:51.316224 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:38:51.316157 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e3d08539-a6af-4015-94e9-5697df7593d0-tmp-dir\") pod \"auth-test-model-a4bfe6-kserve-665bc74f5f-rj7z5\" (UID: \"e3d08539-a6af-4015-94e9-5697df7593d0\") " pod="ai-tenant-e2e-shared-b-a4bfe6/auth-test-model-a4bfe6-kserve-665bc74f5f-rj7z5" Jul 07 22:38:51.316224 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:38:51.316174 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e3d08539-a6af-4015-94e9-5697df7593d0-home\") pod \"auth-test-model-a4bfe6-kserve-665bc74f5f-rj7z5\" (UID: \"e3d08539-a6af-4015-94e9-5697df7593d0\") " pod="ai-tenant-e2e-shared-b-a4bfe6/auth-test-model-a4bfe6-kserve-665bc74f5f-rj7z5" Jul 07 22:38:51.316224 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:38:51.316196 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e3d08539-a6af-4015-94e9-5697df7593d0-tls-certs\") pod \"auth-test-model-a4bfe6-kserve-665bc74f5f-rj7z5\" (UID: \"e3d08539-a6af-4015-94e9-5697df7593d0\") " pod="ai-tenant-e2e-shared-b-a4bfe6/auth-test-model-a4bfe6-kserve-665bc74f5f-rj7z5" Jul 07 22:38:51.316605 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:38:51.316578 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e3d08539-a6af-4015-94e9-5697df7593d0-model-cache\") pod \"auth-test-model-a4bfe6-kserve-665bc74f5f-rj7z5\" (UID: \"e3d08539-a6af-4015-94e9-5697df7593d0\") " pod="ai-tenant-e2e-shared-b-a4bfe6/auth-test-model-a4bfe6-kserve-665bc74f5f-rj7z5" Jul 07 22:38:51.316707 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:38:51.316626 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e3d08539-a6af-4015-94e9-5697df7593d0-tmp-dir\") pod \"auth-test-model-a4bfe6-kserve-665bc74f5f-rj7z5\" (UID: \"e3d08539-a6af-4015-94e9-5697df7593d0\") " pod="ai-tenant-e2e-shared-b-a4bfe6/auth-test-model-a4bfe6-kserve-665bc74f5f-rj7z5" Jul 07 22:38:51.316707 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:38:51.316680 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e3d08539-a6af-4015-94e9-5697df7593d0-home\") pod \"auth-test-model-a4bfe6-kserve-665bc74f5f-rj7z5\" (UID: \"e3d08539-a6af-4015-94e9-5697df7593d0\") " pod="ai-tenant-e2e-shared-b-a4bfe6/auth-test-model-a4bfe6-kserve-665bc74f5f-rj7z5" Jul 07 22:38:51.318767 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:38:51.318737 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e3d08539-a6af-4015-94e9-5697df7593d0-tls-certs\") pod \"auth-test-model-a4bfe6-kserve-665bc74f5f-rj7z5\" (UID: \"e3d08539-a6af-4015-94e9-5697df7593d0\") " pod="ai-tenant-e2e-shared-b-a4bfe6/auth-test-model-a4bfe6-kserve-665bc74f5f-rj7z5" Jul 07 22:38:51.318891 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:38:51.318869 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e3d08539-a6af-4015-94e9-5697df7593d0-dshm\") pod \"auth-test-model-a4bfe6-kserve-665bc74f5f-rj7z5\" (UID: \"e3d08539-a6af-4015-94e9-5697df7593d0\") " pod="ai-tenant-e2e-shared-b-a4bfe6/auth-test-model-a4bfe6-kserve-665bc74f5f-rj7z5" Jul 07 22:38:51.323606 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:38:51.323585 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8hrf4\" (UniqueName: \"kubernetes.io/projected/e3d08539-a6af-4015-94e9-5697df7593d0-kube-api-access-8hrf4\") pod \"auth-test-model-a4bfe6-kserve-665bc74f5f-rj7z5\" (UID: \"e3d08539-a6af-4015-94e9-5697df7593d0\") " pod="ai-tenant-e2e-shared-b-a4bfe6/auth-test-model-a4bfe6-kserve-665bc74f5f-rj7z5" Jul 07 22:38:51.431498 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:38:51.431413 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-a4bfe6/auth-test-model-a4bfe6-kserve-665bc74f5f-rj7z5" Jul 07 22:38:51.563135 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:38:51.563110 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-a4bfe6/auth-test-model-a4bfe6-kserve-665bc74f5f-rj7z5"] Jul 07 22:38:51.565539 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:38:51.565510 2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode3d08539_a6af_4015_94e9_5697df7593d0.slice/crio-4799013bc95e34ae41e073c1cfa0e3db4bc4ba101b6797ded1699fbc7794239f WatchSource:0}: Error finding container 4799013bc95e34ae41e073c1cfa0e3db4bc4ba101b6797ded1699fbc7794239f: Status 404 returned error can't find the container with id 4799013bc95e34ae41e073c1cfa0e3db4bc4ba101b6797ded1699fbc7794239f Jul 07 22:38:52.325479 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:38:52.325440 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-a4bfe6/auth-test-model-a4bfe6-kserve-665bc74f5f-rj7z5" event={"ID":"e3d08539-a6af-4015-94e9-5697df7593d0","Type":"ContainerStarted","Data":"027839f0b40d3beda339bd91c9faab69047072d7c07fcc72345a1d3b3476c09a"} Jul 07 22:38:52.325865 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:38:52.325487 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-a4bfe6/auth-test-model-a4bfe6-kserve-665bc74f5f-rj7z5" event={"ID":"e3d08539-a6af-4015-94e9-5697df7593d0","Type":"ContainerStarted","Data":"4799013bc95e34ae41e073c1cfa0e3db4bc4ba101b6797ded1699fbc7794239f"} Jul 07 22:38:52.340557 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:38:52.340515 2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-a4bfe6/auth-test-model-a4bfe6-kserve-665bc74f5f-rj7z5" podStartSLOduration=1.340499241 podStartE2EDuration="1.340499241s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 22:38:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:38:52.340162959 +0000 UTC m=+2114.654385979" watchObservedRunningTime="2026-07-07 22:38:52.340499241 +0000 UTC m=+2114.654722261" Jul 07 22:39:01.432106 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:39:01.432059 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-a4bfe6/auth-test-model-a4bfe6-kserve-665bc74f5f-rj7z5" Jul 07 22:39:01.432490 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:39:01.432122 2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-a4bfe6/auth-test-model-a4bfe6-kserve-665bc74f5f-rj7z5" Jul 07 22:39:01.447452 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:39:01.447415 2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-a4bfe6/auth-test-model-a4bfe6-kserve-665bc74f5f-rj7z5" Jul 07 22:39:02.381773 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:39:02.381741 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-a4bfe6/auth-test-model-a4bfe6-kserve-665bc74f5f-rj7z5" Jul 07 22:40:45.296263 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:40:45.296225 2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-a4bfe6/auth-test-model-a4bfe6-kserve-665bc74f5f-rj7z5"] Jul 07 22:40:59.718259 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:40:59.718169 2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-41e952/sub-test-model-41e952-kserve-6b89c984c-pbvb6"] Jul 07 22:40:59.721336 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:40:59.721313 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-41e952/sub-test-model-41e952-kserve-6b89c984c-pbvb6" Jul 07 22:40:59.724485 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:40:59.724456 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-41e952\"/\"kube-root-ca.crt\"" Jul 07 22:40:59.724485 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:40:59.724471 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-41e952\"/\"sub-test-model-41e952-kserve-self-signed-certs\"" Jul 07 22:40:59.724694 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:40:59.724500 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-41e952\"/\"openshift-service-ca.crt\"" Jul 07 22:40:59.724694 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:40:59.724509 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-41e952\"/\"default-dockercfg-ntw89\"" Jul 07 22:40:59.728646 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:40:59.728621 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-41e952/sub-test-model-41e952-kserve-6b89c984c-pbvb6"] Jul 07 22:40:59.768550 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:40:59.768519 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2fa5830e-a011-4ed7-96ee-1f6b2da5f000-dshm\") pod \"sub-test-model-41e952-kserve-6b89c984c-pbvb6\" (UID: \"2fa5830e-a011-4ed7-96ee-1f6b2da5f000\") " pod="ai-tenant-e2e-shared-a-41e952/sub-test-model-41e952-kserve-6b89c984c-pbvb6" Jul 07 22:40:59.768734 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:40:59.768579 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2fa5830e-a011-4ed7-96ee-1f6b2da5f000-home\") pod \"sub-test-model-41e952-kserve-6b89c984c-pbvb6\" (UID: \"2fa5830e-a011-4ed7-96ee-1f6b2da5f000\") " pod="ai-tenant-e2e-shared-a-41e952/sub-test-model-41e952-kserve-6b89c984c-pbvb6" Jul 07 22:40:59.768734 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:40:59.768638 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2fa5830e-a011-4ed7-96ee-1f6b2da5f000-tmp-dir\") pod \"sub-test-model-41e952-kserve-6b89c984c-pbvb6\" (UID: \"2fa5830e-a011-4ed7-96ee-1f6b2da5f000\") " pod="ai-tenant-e2e-shared-a-41e952/sub-test-model-41e952-kserve-6b89c984c-pbvb6" Jul 07 22:40:59.768734 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:40:59.768705 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2fa5830e-a011-4ed7-96ee-1f6b2da5f000-model-cache\") pod \"sub-test-model-41e952-kserve-6b89c984c-pbvb6\" (UID: \"2fa5830e-a011-4ed7-96ee-1f6b2da5f000\") " pod="ai-tenant-e2e-shared-a-41e952/sub-test-model-41e952-kserve-6b89c984c-pbvb6" Jul 07 22:40:59.768925 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:40:59.768764 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2fa5830e-a011-4ed7-96ee-1f6b2da5f000-tls-certs\") pod \"sub-test-model-41e952-kserve-6b89c984c-pbvb6\" (UID: \"2fa5830e-a011-4ed7-96ee-1f6b2da5f000\") " pod="ai-tenant-e2e-shared-a-41e952/sub-test-model-41e952-kserve-6b89c984c-pbvb6" Jul 07 22:40:59.768925 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:40:59.768842 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5ndhq\" (UniqueName: \"kubernetes.io/projected/2fa5830e-a011-4ed7-96ee-1f6b2da5f000-kube-api-access-5ndhq\") pod \"sub-test-model-41e952-kserve-6b89c984c-pbvb6\" (UID: \"2fa5830e-a011-4ed7-96ee-1f6b2da5f000\") " pod="ai-tenant-e2e-shared-a-41e952/sub-test-model-41e952-kserve-6b89c984c-pbvb6" Jul 07 22:40:59.870268 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:40:59.870227 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2fa5830e-a011-4ed7-96ee-1f6b2da5f000-tmp-dir\") pod \"sub-test-model-41e952-kserve-6b89c984c-pbvb6\" (UID: \"2fa5830e-a011-4ed7-96ee-1f6b2da5f000\") " pod="ai-tenant-e2e-shared-a-41e952/sub-test-model-41e952-kserve-6b89c984c-pbvb6" Jul 07 22:40:59.870468 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:40:59.870279 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2fa5830e-a011-4ed7-96ee-1f6b2da5f000-model-cache\") pod \"sub-test-model-41e952-kserve-6b89c984c-pbvb6\" (UID: \"2fa5830e-a011-4ed7-96ee-1f6b2da5f000\") " pod="ai-tenant-e2e-shared-a-41e952/sub-test-model-41e952-kserve-6b89c984c-pbvb6" Jul 07 22:40:59.870468 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:40:59.870306 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2fa5830e-a011-4ed7-96ee-1f6b2da5f000-tls-certs\") pod \"sub-test-model-41e952-kserve-6b89c984c-pbvb6\" (UID: \"2fa5830e-a011-4ed7-96ee-1f6b2da5f000\") " pod="ai-tenant-e2e-shared-a-41e952/sub-test-model-41e952-kserve-6b89c984c-pbvb6" Jul 07 22:40:59.870468 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:40:59.870355 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5ndhq\" (UniqueName: \"kubernetes.io/projected/2fa5830e-a011-4ed7-96ee-1f6b2da5f000-kube-api-access-5ndhq\") pod \"sub-test-model-41e952-kserve-6b89c984c-pbvb6\" (UID: \"2fa5830e-a011-4ed7-96ee-1f6b2da5f000\") " pod="ai-tenant-e2e-shared-a-41e952/sub-test-model-41e952-kserve-6b89c984c-pbvb6" Jul 07 22:40:59.870468 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:40:59.870400 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2fa5830e-a011-4ed7-96ee-1f6b2da5f000-dshm\") pod \"sub-test-model-41e952-kserve-6b89c984c-pbvb6\" (UID: \"2fa5830e-a011-4ed7-96ee-1f6b2da5f000\") " pod="ai-tenant-e2e-shared-a-41e952/sub-test-model-41e952-kserve-6b89c984c-pbvb6" Jul 07 22:40:59.870468 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:40:59.870470 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2fa5830e-a011-4ed7-96ee-1f6b2da5f000-home\") pod \"sub-test-model-41e952-kserve-6b89c984c-pbvb6\" (UID: \"2fa5830e-a011-4ed7-96ee-1f6b2da5f000\") " pod="ai-tenant-e2e-shared-a-41e952/sub-test-model-41e952-kserve-6b89c984c-pbvb6" Jul 07 22:40:59.870756 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:40:59.870710 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2fa5830e-a011-4ed7-96ee-1f6b2da5f000-tmp-dir\") pod \"sub-test-model-41e952-kserve-6b89c984c-pbvb6\" (UID: \"2fa5830e-a011-4ed7-96ee-1f6b2da5f000\") " pod="ai-tenant-e2e-shared-a-41e952/sub-test-model-41e952-kserve-6b89c984c-pbvb6" Jul 07 22:40:59.870756 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:40:59.870727 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2fa5830e-a011-4ed7-96ee-1f6b2da5f000-model-cache\") pod \"sub-test-model-41e952-kserve-6b89c984c-pbvb6\" (UID: \"2fa5830e-a011-4ed7-96ee-1f6b2da5f000\") " pod="ai-tenant-e2e-shared-a-41e952/sub-test-model-41e952-kserve-6b89c984c-pbvb6" Jul 07 22:40:59.870832 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:40:59.870795 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2fa5830e-a011-4ed7-96ee-1f6b2da5f000-home\") pod \"sub-test-model-41e952-kserve-6b89c984c-pbvb6\" (UID: \"2fa5830e-a011-4ed7-96ee-1f6b2da5f000\") " pod="ai-tenant-e2e-shared-a-41e952/sub-test-model-41e952-kserve-6b89c984c-pbvb6" Jul 07 22:40:59.872636 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:40:59.872615 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2fa5830e-a011-4ed7-96ee-1f6b2da5f000-dshm\") pod \"sub-test-model-41e952-kserve-6b89c984c-pbvb6\" (UID: \"2fa5830e-a011-4ed7-96ee-1f6b2da5f000\") " pod="ai-tenant-e2e-shared-a-41e952/sub-test-model-41e952-kserve-6b89c984c-pbvb6" Jul 07 22:40:59.872895 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:40:59.872879 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2fa5830e-a011-4ed7-96ee-1f6b2da5f000-tls-certs\") pod \"sub-test-model-41e952-kserve-6b89c984c-pbvb6\" (UID: \"2fa5830e-a011-4ed7-96ee-1f6b2da5f000\") " pod="ai-tenant-e2e-shared-a-41e952/sub-test-model-41e952-kserve-6b89c984c-pbvb6" Jul 07 22:40:59.878638 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:40:59.878620 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5ndhq\" (UniqueName: \"kubernetes.io/projected/2fa5830e-a011-4ed7-96ee-1f6b2da5f000-kube-api-access-5ndhq\") pod \"sub-test-model-41e952-kserve-6b89c984c-pbvb6\" (UID: \"2fa5830e-a011-4ed7-96ee-1f6b2da5f000\") " pod="ai-tenant-e2e-shared-a-41e952/sub-test-model-41e952-kserve-6b89c984c-pbvb6" Jul 07 22:41:00.032594 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:41:00.032498 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-41e952/sub-test-model-41e952-kserve-6b89c984c-pbvb6" Jul 07 22:41:00.167368 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:41:00.167328 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-41e952/sub-test-model-41e952-kserve-6b89c984c-pbvb6"] Jul 07 22:41:00.169571 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:41:00.169541 2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2fa5830e_a011_4ed7_96ee_1f6b2da5f000.slice/crio-ec1cf4bc7db7491eb9b2e751bf82225ee1a5462bddce585545bd60f6c525000a WatchSource:0}: Error finding container ec1cf4bc7db7491eb9b2e751bf82225ee1a5462bddce585545bd60f6c525000a: Status 404 returned error can't find the container with id ec1cf4bc7db7491eb9b2e751bf82225ee1a5462bddce585545bd60f6c525000a Jul 07 22:41:00.317159 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:41:00.317036 2545 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-a4bfe6/auth-test-model-a4bfe6-kserve-665bc74f5f-rj7z5" podUID="e3d08539-a6af-4015-94e9-5697df7593d0" containerName="main" containerID="cri-o://027839f0b40d3beda339bd91c9faab69047072d7c07fcc72345a1d3b3476c09a" gracePeriod=45 Jul 07 22:41:00.552278 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:41:00.552248 2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-a4bfe6/auth-test-model-a4bfe6-kserve-665bc74f5f-rj7z5" Jul 07 22:41:00.577069 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:41:00.576996 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e3d08539-a6af-4015-94e9-5697df7593d0-dshm\") pod \"e3d08539-a6af-4015-94e9-5697df7593d0\" (UID: \"e3d08539-a6af-4015-94e9-5697df7593d0\") " Jul 07 22:41:00.577069 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:41:00.577038 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e3d08539-a6af-4015-94e9-5697df7593d0-tmp-dir\") pod \"e3d08539-a6af-4015-94e9-5697df7593d0\" (UID: \"e3d08539-a6af-4015-94e9-5697df7593d0\") " Jul 07 22:41:00.577299 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:41:00.577102 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e3d08539-a6af-4015-94e9-5697df7593d0-home\") pod \"e3d08539-a6af-4015-94e9-5697df7593d0\" (UID: \"e3d08539-a6af-4015-94e9-5697df7593d0\") " Jul 07 22:41:00.577299 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:41:00.577163 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e3d08539-a6af-4015-94e9-5697df7593d0-model-cache\") pod \"e3d08539-a6af-4015-94e9-5697df7593d0\" (UID: \"e3d08539-a6af-4015-94e9-5697df7593d0\") " Jul 07 22:41:00.577299 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:41:00.577181 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8hrf4\" (UniqueName: \"kubernetes.io/projected/e3d08539-a6af-4015-94e9-5697df7593d0-kube-api-access-8hrf4\") pod \"e3d08539-a6af-4015-94e9-5697df7593d0\" (UID: \"e3d08539-a6af-4015-94e9-5697df7593d0\") " Jul 07 22:41:00.577299 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:41:00.577205 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e3d08539-a6af-4015-94e9-5697df7593d0-tls-certs\") pod \"e3d08539-a6af-4015-94e9-5697df7593d0\" (UID: \"e3d08539-a6af-4015-94e9-5697df7593d0\") " Jul 07 22:41:00.577521 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:41:00.577365 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3d08539-a6af-4015-94e9-5697df7593d0-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e3d08539-a6af-4015-94e9-5697df7593d0" (UID: "e3d08539-a6af-4015-94e9-5697df7593d0"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 22:41:00.577521 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:41:00.577404 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3d08539-a6af-4015-94e9-5697df7593d0-model-cache" (OuterVolumeSpecName: "model-cache") pod "e3d08539-a6af-4015-94e9-5697df7593d0" (UID: "e3d08539-a6af-4015-94e9-5697df7593d0"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 22:41:00.577521 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:41:00.577420 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3d08539-a6af-4015-94e9-5697df7593d0-home" (OuterVolumeSpecName: "home") pod "e3d08539-a6af-4015-94e9-5697df7593d0" (UID: "e3d08539-a6af-4015-94e9-5697df7593d0"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 22:41:00.577660 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:41:00.577645 2545 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e3d08539-a6af-4015-94e9-5697df7593d0-tmp-dir\") on node \"ip-10-0-131-55.ec2.internal\" DevicePath \"\"" Jul 07 22:41:00.577711 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:41:00.577665 2545 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e3d08539-a6af-4015-94e9-5697df7593d0-home\") on node \"ip-10-0-131-55.ec2.internal\" DevicePath \"\"" Jul 07 22:41:00.577711 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:41:00.577679 2545 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e3d08539-a6af-4015-94e9-5697df7593d0-model-cache\") on node \"ip-10-0-131-55.ec2.internal\" DevicePath \"\"" Jul 07 22:41:00.579420 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:41:00.579395 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e3d08539-a6af-4015-94e9-5697df7593d0-kube-api-access-8hrf4" (OuterVolumeSpecName: "kube-api-access-8hrf4") pod "e3d08539-a6af-4015-94e9-5697df7593d0" (UID: "e3d08539-a6af-4015-94e9-5697df7593d0"). InnerVolumeSpecName "kube-api-access-8hrf4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 22:41:00.579621 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:41:00.579599 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e3d08539-a6af-4015-94e9-5697df7593d0-dshm" (OuterVolumeSpecName: "dshm") pod "e3d08539-a6af-4015-94e9-5697df7593d0" (UID: "e3d08539-a6af-4015-94e9-5697df7593d0"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 22:41:00.579717 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:41:00.579599 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e3d08539-a6af-4015-94e9-5697df7593d0-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e3d08539-a6af-4015-94e9-5697df7593d0" (UID: "e3d08539-a6af-4015-94e9-5697df7593d0"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 22:41:00.678459 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:41:00.678424 2545 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e3d08539-a6af-4015-94e9-5697df7593d0-dshm\") on node \"ip-10-0-131-55.ec2.internal\" DevicePath \"\"" Jul 07 22:41:00.678459 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:41:00.678453 2545 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8hrf4\" (UniqueName: \"kubernetes.io/projected/e3d08539-a6af-4015-94e9-5697df7593d0-kube-api-access-8hrf4\") on node \"ip-10-0-131-55.ec2.internal\" DevicePath \"\"" Jul 07 22:41:00.678459 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:41:00.678464 2545 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e3d08539-a6af-4015-94e9-5697df7593d0-tls-certs\") on node \"ip-10-0-131-55.ec2.internal\" DevicePath \"\"" Jul 07 22:41:00.939426 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:41:00.939391 2545 generic.go:356] "Generic (PLEG): container finished" podID="e3d08539-a6af-4015-94e9-5697df7593d0" containerID="027839f0b40d3beda339bd91c9faab69047072d7c07fcc72345a1d3b3476c09a" exitCode=0 Jul 07 22:41:00.939842 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:41:00.939441 2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-a4bfe6/auth-test-model-a4bfe6-kserve-665bc74f5f-rj7z5" Jul 07 22:41:00.939842 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:41:00.939477 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-a4bfe6/auth-test-model-a4bfe6-kserve-665bc74f5f-rj7z5" event={"ID":"e3d08539-a6af-4015-94e9-5697df7593d0","Type":"ContainerDied","Data":"027839f0b40d3beda339bd91c9faab69047072d7c07fcc72345a1d3b3476c09a"} Jul 07 22:41:00.939842 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:41:00.939522 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-a4bfe6/auth-test-model-a4bfe6-kserve-665bc74f5f-rj7z5" event={"ID":"e3d08539-a6af-4015-94e9-5697df7593d0","Type":"ContainerDied","Data":"4799013bc95e34ae41e073c1cfa0e3db4bc4ba101b6797ded1699fbc7794239f"} Jul 07 22:41:00.939842 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:41:00.939540 2545 scope.go:117] "RemoveContainer" containerID="027839f0b40d3beda339bd91c9faab69047072d7c07fcc72345a1d3b3476c09a" Jul 07 22:41:00.941312 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:41:00.941286 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-41e952/sub-test-model-41e952-kserve-6b89c984c-pbvb6" event={"ID":"2fa5830e-a011-4ed7-96ee-1f6b2da5f000","Type":"ContainerStarted","Data":"b02a2a0036396a629a3d50f1eecd4455f79c8e60855ae62f8df5f13cbb5f2d06"} Jul 07 22:41:00.941411 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:41:00.941320 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-41e952/sub-test-model-41e952-kserve-6b89c984c-pbvb6" event={"ID":"2fa5830e-a011-4ed7-96ee-1f6b2da5f000","Type":"ContainerStarted","Data":"ec1cf4bc7db7491eb9b2e751bf82225ee1a5462bddce585545bd60f6c525000a"} Jul 07 22:41:00.948491 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:41:00.948476 2545 scope.go:117] "RemoveContainer" containerID="027839f0b40d3beda339bd91c9faab69047072d7c07fcc72345a1d3b3476c09a" Jul 07 22:41:00.948719 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:41:00.948700 2545 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"027839f0b40d3beda339bd91c9faab69047072d7c07fcc72345a1d3b3476c09a\": container with ID starting with 027839f0b40d3beda339bd91c9faab69047072d7c07fcc72345a1d3b3476c09a not found: ID does not exist" containerID="027839f0b40d3beda339bd91c9faab69047072d7c07fcc72345a1d3b3476c09a" Jul 07 22:41:00.948774 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:41:00.948726 2545 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"027839f0b40d3beda339bd91c9faab69047072d7c07fcc72345a1d3b3476c09a"} err="failed to get container status \"027839f0b40d3beda339bd91c9faab69047072d7c07fcc72345a1d3b3476c09a\": rpc error: code = NotFound desc = could not find container \"027839f0b40d3beda339bd91c9faab69047072d7c07fcc72345a1d3b3476c09a\": container with ID starting with 027839f0b40d3beda339bd91c9faab69047072d7c07fcc72345a1d3b3476c09a not found: ID does not exist" Jul 07 22:41:00.960613 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:41:00.960571 2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-41e952/sub-test-model-41e952-kserve-6b89c984c-pbvb6" podStartSLOduration=1.960559494 podStartE2EDuration="1.960559494s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 22:40:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:41:00.958762353 +0000 UTC m=+2243.272985372" watchObservedRunningTime="2026-07-07 22:41:00.960559494 +0000 UTC m=+2243.274782513" Jul 07 22:41:00.971970 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:41:00.971949 2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-a4bfe6/auth-test-model-a4bfe6-kserve-665bc74f5f-rj7z5"] Jul 07 22:41:00.976089 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:41:00.976062 2545 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-a4bfe6/auth-test-model-a4bfe6-kserve-665bc74f5f-rj7z5"] Jul 07 22:41:02.311504 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:41:02.311465 2545 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e3d08539-a6af-4015-94e9-5697df7593d0" path="/var/lib/kubelet/pods/e3d08539-a6af-4015-94e9-5697df7593d0/volumes" Jul 07 22:41:10.032892 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:41:10.032852 2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-41e952/sub-test-model-41e952-kserve-6b89c984c-pbvb6" Jul 07 22:41:10.032892 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:41:10.032899 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-41e952/sub-test-model-41e952-kserve-6b89c984c-pbvb6" Jul 07 22:41:10.045626 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:41:10.045601 2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-41e952/sub-test-model-41e952-kserve-6b89c984c-pbvb6" Jul 07 22:41:11.001145 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:41:11.001117 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-41e952/sub-test-model-41e952-kserve-6b89c984c-pbvb6" Jul 07 22:42:19.534095 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:42:19.534043 2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-41e952/sub-test-model-41e952-kserve-6b89c984c-pbvb6"] Jul 07 22:42:33.709917 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:42:33.709883 2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-41e952/rate-test-model-41e952-kserve-6957b7d7bc-k4hdx"] Jul 07 22:42:33.710392 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:42:33.710266 2545 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e3d08539-a6af-4015-94e9-5697df7593d0" containerName="main" Jul 07 22:42:33.710392 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:42:33.710278 2545 state_mem.go:107] "Deleted CPUSet assignment" podUID="e3d08539-a6af-4015-94e9-5697df7593d0" containerName="main" Jul 07 22:42:33.710392 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:42:33.710358 2545 memory_manager.go:368] "RemoveStaleState removing state" podUID="e3d08539-a6af-4015-94e9-5697df7593d0" containerName="main" Jul 07 22:42:33.713808 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:42:33.713791 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-41e952/rate-test-model-41e952-kserve-6957b7d7bc-k4hdx" Jul 07 22:42:33.716136 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:42:33.716113 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-41e952\"/\"rate-test-model-41e952-kserve-self-signed-certs\"" Jul 07 22:42:33.722549 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:42:33.722514 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-41e952/rate-test-model-41e952-kserve-6957b7d7bc-k4hdx"] Jul 07 22:42:33.789520 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:42:33.789483 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8ad20429-54bd-4414-92fb-dc0554db927b-tls-certs\") pod \"rate-test-model-41e952-kserve-6957b7d7bc-k4hdx\" (UID: \"8ad20429-54bd-4414-92fb-dc0554db927b\") " pod="ai-tenant-e2e-shared-a-41e952/rate-test-model-41e952-kserve-6957b7d7bc-k4hdx" Jul 07 22:42:33.789691 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:42:33.789542 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8ad20429-54bd-4414-92fb-dc0554db927b-home\") pod \"rate-test-model-41e952-kserve-6957b7d7bc-k4hdx\" (UID: \"8ad20429-54bd-4414-92fb-dc0554db927b\") " pod="ai-tenant-e2e-shared-a-41e952/rate-test-model-41e952-kserve-6957b7d7bc-k4hdx" Jul 07 22:42:33.789691 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:42:33.789593 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8ad20429-54bd-4414-92fb-dc0554db927b-tmp-dir\") pod \"rate-test-model-41e952-kserve-6957b7d7bc-k4hdx\" (UID: \"8ad20429-54bd-4414-92fb-dc0554db927b\") " pod="ai-tenant-e2e-shared-a-41e952/rate-test-model-41e952-kserve-6957b7d7bc-k4hdx" Jul 07 22:42:33.789691 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:42:33.789668 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xlx2m\" (UniqueName: \"kubernetes.io/projected/8ad20429-54bd-4414-92fb-dc0554db927b-kube-api-access-xlx2m\") pod \"rate-test-model-41e952-kserve-6957b7d7bc-k4hdx\" (UID: \"8ad20429-54bd-4414-92fb-dc0554db927b\") " pod="ai-tenant-e2e-shared-a-41e952/rate-test-model-41e952-kserve-6957b7d7bc-k4hdx" Jul 07 22:42:33.789844 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:42:33.789739 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8ad20429-54bd-4414-92fb-dc0554db927b-model-cache\") pod \"rate-test-model-41e952-kserve-6957b7d7bc-k4hdx\" (UID: \"8ad20429-54bd-4414-92fb-dc0554db927b\") " pod="ai-tenant-e2e-shared-a-41e952/rate-test-model-41e952-kserve-6957b7d7bc-k4hdx" Jul 07 22:42:33.789844 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:42:33.789768 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8ad20429-54bd-4414-92fb-dc0554db927b-dshm\") pod \"rate-test-model-41e952-kserve-6957b7d7bc-k4hdx\" (UID: \"8ad20429-54bd-4414-92fb-dc0554db927b\") " pod="ai-tenant-e2e-shared-a-41e952/rate-test-model-41e952-kserve-6957b7d7bc-k4hdx" Jul 07 22:42:33.890960 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:42:33.890923 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8ad20429-54bd-4414-92fb-dc0554db927b-model-cache\") pod \"rate-test-model-41e952-kserve-6957b7d7bc-k4hdx\" (UID: \"8ad20429-54bd-4414-92fb-dc0554db927b\") " pod="ai-tenant-e2e-shared-a-41e952/rate-test-model-41e952-kserve-6957b7d7bc-k4hdx" Jul 07 22:42:33.890960 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:42:33.890963 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8ad20429-54bd-4414-92fb-dc0554db927b-dshm\") pod \"rate-test-model-41e952-kserve-6957b7d7bc-k4hdx\" (UID: \"8ad20429-54bd-4414-92fb-dc0554db927b\") " pod="ai-tenant-e2e-shared-a-41e952/rate-test-model-41e952-kserve-6957b7d7bc-k4hdx" Jul 07 22:42:33.891242 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:42:33.890995 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8ad20429-54bd-4414-92fb-dc0554db927b-tls-certs\") pod \"rate-test-model-41e952-kserve-6957b7d7bc-k4hdx\" (UID: \"8ad20429-54bd-4414-92fb-dc0554db927b\") " pod="ai-tenant-e2e-shared-a-41e952/rate-test-model-41e952-kserve-6957b7d7bc-k4hdx" Jul 07 22:42:33.891242 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:42:33.891034 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8ad20429-54bd-4414-92fb-dc0554db927b-home\") pod \"rate-test-model-41e952-kserve-6957b7d7bc-k4hdx\" (UID: \"8ad20429-54bd-4414-92fb-dc0554db927b\") " pod="ai-tenant-e2e-shared-a-41e952/rate-test-model-41e952-kserve-6957b7d7bc-k4hdx" Jul 07 22:42:33.891242 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:42:33.891122 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8ad20429-54bd-4414-92fb-dc0554db927b-tmp-dir\") pod \"rate-test-model-41e952-kserve-6957b7d7bc-k4hdx\" (UID: \"8ad20429-54bd-4414-92fb-dc0554db927b\") " pod="ai-tenant-e2e-shared-a-41e952/rate-test-model-41e952-kserve-6957b7d7bc-k4hdx" Jul 07 22:42:33.891242 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:42:33.891191 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xlx2m\" (UniqueName: \"kubernetes.io/projected/8ad20429-54bd-4414-92fb-dc0554db927b-kube-api-access-xlx2m\") pod \"rate-test-model-41e952-kserve-6957b7d7bc-k4hdx\" (UID: \"8ad20429-54bd-4414-92fb-dc0554db927b\") " pod="ai-tenant-e2e-shared-a-41e952/rate-test-model-41e952-kserve-6957b7d7bc-k4hdx" Jul 07 22:42:33.891494 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:42:33.891464 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8ad20429-54bd-4414-92fb-dc0554db927b-home\") pod \"rate-test-model-41e952-kserve-6957b7d7bc-k4hdx\" (UID: \"8ad20429-54bd-4414-92fb-dc0554db927b\") " pod="ai-tenant-e2e-shared-a-41e952/rate-test-model-41e952-kserve-6957b7d7bc-k4hdx" Jul 07 22:42:33.891563 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:42:33.891517 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8ad20429-54bd-4414-92fb-dc0554db927b-model-cache\") pod \"rate-test-model-41e952-kserve-6957b7d7bc-k4hdx\" (UID: \"8ad20429-54bd-4414-92fb-dc0554db927b\") " pod="ai-tenant-e2e-shared-a-41e952/rate-test-model-41e952-kserve-6957b7d7bc-k4hdx" Jul 07 22:42:33.891563 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:42:33.891539 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8ad20429-54bd-4414-92fb-dc0554db927b-tmp-dir\") pod \"rate-test-model-41e952-kserve-6957b7d7bc-k4hdx\" (UID: \"8ad20429-54bd-4414-92fb-dc0554db927b\") " pod="ai-tenant-e2e-shared-a-41e952/rate-test-model-41e952-kserve-6957b7d7bc-k4hdx" Jul 07 22:42:33.893265 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:42:33.893242 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8ad20429-54bd-4414-92fb-dc0554db927b-dshm\") pod \"rate-test-model-41e952-kserve-6957b7d7bc-k4hdx\" (UID: \"8ad20429-54bd-4414-92fb-dc0554db927b\") " pod="ai-tenant-e2e-shared-a-41e952/rate-test-model-41e952-kserve-6957b7d7bc-k4hdx" Jul 07 22:42:33.893608 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:42:33.893592 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8ad20429-54bd-4414-92fb-dc0554db927b-tls-certs\") pod \"rate-test-model-41e952-kserve-6957b7d7bc-k4hdx\" (UID: \"8ad20429-54bd-4414-92fb-dc0554db927b\") " pod="ai-tenant-e2e-shared-a-41e952/rate-test-model-41e952-kserve-6957b7d7bc-k4hdx" Jul 07 22:42:33.898921 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:42:33.898896 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xlx2m\" (UniqueName: \"kubernetes.io/projected/8ad20429-54bd-4414-92fb-dc0554db927b-kube-api-access-xlx2m\") pod \"rate-test-model-41e952-kserve-6957b7d7bc-k4hdx\" (UID: \"8ad20429-54bd-4414-92fb-dc0554db927b\") " pod="ai-tenant-e2e-shared-a-41e952/rate-test-model-41e952-kserve-6957b7d7bc-k4hdx" Jul 07 22:42:34.024452 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:42:34.024416 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-41e952/rate-test-model-41e952-kserve-6957b7d7bc-k4hdx" Jul 07 22:42:34.153672 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:42:34.153641 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-41e952/rate-test-model-41e952-kserve-6957b7d7bc-k4hdx"] Jul 07 22:42:34.155099 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:42:34.155052 2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8ad20429_54bd_4414_92fb_dc0554db927b.slice/crio-8175e53a09b6b65e959ee121ddbd1e183698cdf3e931f6432a4d8abde5bcf1c2 WatchSource:0}: Error finding container 8175e53a09b6b65e959ee121ddbd1e183698cdf3e931f6432a4d8abde5bcf1c2: Status 404 returned error can't find the container with id 8175e53a09b6b65e959ee121ddbd1e183698cdf3e931f6432a4d8abde5bcf1c2 Jul 07 22:42:34.395908 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:42:34.395826 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-41e952/rate-test-model-41e952-kserve-6957b7d7bc-k4hdx" event={"ID":"8ad20429-54bd-4414-92fb-dc0554db927b","Type":"ContainerStarted","Data":"bde32d9ddbfb2393c6246dd9cd2fe079b2010eb1d8c11b8ed8a66dacc19438e6"} Jul 07 22:42:34.395908 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:42:34.395869 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-41e952/rate-test-model-41e952-kserve-6957b7d7bc-k4hdx" event={"ID":"8ad20429-54bd-4414-92fb-dc0554db927b","Type":"ContainerStarted","Data":"8175e53a09b6b65e959ee121ddbd1e183698cdf3e931f6432a4d8abde5bcf1c2"} Jul 07 22:42:34.412398 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:42:34.412351 2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-41e952/rate-test-model-41e952-kserve-6957b7d7bc-k4hdx" podStartSLOduration=1.412335417 podStartE2EDuration="1.412335417s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 22:42:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:42:34.410576408 +0000 UTC m=+2336.724799427" watchObservedRunningTime="2026-07-07 22:42:34.412335417 +0000 UTC m=+2336.726558435" Jul 07 22:42:34.556024 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:42:34.555986 2545 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-41e952/sub-test-model-41e952-kserve-6b89c984c-pbvb6" podUID="2fa5830e-a011-4ed7-96ee-1f6b2da5f000" containerName="main" containerID="cri-o://b02a2a0036396a629a3d50f1eecd4455f79c8e60855ae62f8df5f13cbb5f2d06" gracePeriod=45 Jul 07 22:42:34.787612 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:42:34.787583 2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-41e952/sub-test-model-41e952-kserve-6b89c984c-pbvb6" Jul 07 22:42:34.901851 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:42:34.901820 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2fa5830e-a011-4ed7-96ee-1f6b2da5f000-home\") pod \"2fa5830e-a011-4ed7-96ee-1f6b2da5f000\" (UID: \"2fa5830e-a011-4ed7-96ee-1f6b2da5f000\") " Jul 07 22:42:34.901984 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:42:34.901869 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2fa5830e-a011-4ed7-96ee-1f6b2da5f000-tmp-dir\") pod \"2fa5830e-a011-4ed7-96ee-1f6b2da5f000\" (UID: \"2fa5830e-a011-4ed7-96ee-1f6b2da5f000\") " Jul 07 22:42:34.901984 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:42:34.901961 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5ndhq\" (UniqueName: \"kubernetes.io/projected/2fa5830e-a011-4ed7-96ee-1f6b2da5f000-kube-api-access-5ndhq\") pod \"2fa5830e-a011-4ed7-96ee-1f6b2da5f000\" (UID: \"2fa5830e-a011-4ed7-96ee-1f6b2da5f000\") " Jul 07 22:42:34.902115 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:42:34.902009 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2fa5830e-a011-4ed7-96ee-1f6b2da5f000-tls-certs\") pod \"2fa5830e-a011-4ed7-96ee-1f6b2da5f000\" (UID: \"2fa5830e-a011-4ed7-96ee-1f6b2da5f000\") " Jul 07 22:42:34.902115 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:42:34.902040 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2fa5830e-a011-4ed7-96ee-1f6b2da5f000-dshm\") pod \"2fa5830e-a011-4ed7-96ee-1f6b2da5f000\" (UID: \"2fa5830e-a011-4ed7-96ee-1f6b2da5f000\") " Jul 07 22:42:34.902115 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:42:34.902093 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2fa5830e-a011-4ed7-96ee-1f6b2da5f000-model-cache\") pod \"2fa5830e-a011-4ed7-96ee-1f6b2da5f000\" (UID: \"2fa5830e-a011-4ed7-96ee-1f6b2da5f000\") " Jul 07 22:42:34.902276 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:42:34.902189 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2fa5830e-a011-4ed7-96ee-1f6b2da5f000-home" (OuterVolumeSpecName: "home") pod "2fa5830e-a011-4ed7-96ee-1f6b2da5f000" (UID: "2fa5830e-a011-4ed7-96ee-1f6b2da5f000"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 22:42:34.902276 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:42:34.902200 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2fa5830e-a011-4ed7-96ee-1f6b2da5f000-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "2fa5830e-a011-4ed7-96ee-1f6b2da5f000" (UID: "2fa5830e-a011-4ed7-96ee-1f6b2da5f000"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 22:42:34.902432 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:42:34.902408 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2fa5830e-a011-4ed7-96ee-1f6b2da5f000-model-cache" (OuterVolumeSpecName: "model-cache") pod "2fa5830e-a011-4ed7-96ee-1f6b2da5f000" (UID: "2fa5830e-a011-4ed7-96ee-1f6b2da5f000"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 22:42:34.902558 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:42:34.902539 2545 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2fa5830e-a011-4ed7-96ee-1f6b2da5f000-home\") on node \"ip-10-0-131-55.ec2.internal\" DevicePath \"\"" Jul 07 22:42:34.902558 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:42:34.902554 2545 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2fa5830e-a011-4ed7-96ee-1f6b2da5f000-tmp-dir\") on node \"ip-10-0-131-55.ec2.internal\" DevicePath \"\"" Jul 07 22:42:34.902673 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:42:34.902568 2545 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2fa5830e-a011-4ed7-96ee-1f6b2da5f000-model-cache\") on node \"ip-10-0-131-55.ec2.internal\" DevicePath \"\"" Jul 07 22:42:34.904515 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:42:34.904474 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2fa5830e-a011-4ed7-96ee-1f6b2da5f000-dshm" (OuterVolumeSpecName: "dshm") pod "2fa5830e-a011-4ed7-96ee-1f6b2da5f000" (UID: "2fa5830e-a011-4ed7-96ee-1f6b2da5f000"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 22:42:34.904515 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:42:34.904483 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2fa5830e-a011-4ed7-96ee-1f6b2da5f000-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "2fa5830e-a011-4ed7-96ee-1f6b2da5f000" (UID: "2fa5830e-a011-4ed7-96ee-1f6b2da5f000"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 22:42:34.904661 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:42:34.904591 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2fa5830e-a011-4ed7-96ee-1f6b2da5f000-kube-api-access-5ndhq" (OuterVolumeSpecName: "kube-api-access-5ndhq") pod "2fa5830e-a011-4ed7-96ee-1f6b2da5f000" (UID: "2fa5830e-a011-4ed7-96ee-1f6b2da5f000"). InnerVolumeSpecName "kube-api-access-5ndhq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 22:42:35.004169 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:42:35.004133 2545 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5ndhq\" (UniqueName: \"kubernetes.io/projected/2fa5830e-a011-4ed7-96ee-1f6b2da5f000-kube-api-access-5ndhq\") on node \"ip-10-0-131-55.ec2.internal\" DevicePath \"\"" Jul 07 22:42:35.004320 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:42:35.004170 2545 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2fa5830e-a011-4ed7-96ee-1f6b2da5f000-tls-certs\") on node \"ip-10-0-131-55.ec2.internal\" DevicePath \"\"" Jul 07 22:42:35.004320 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:42:35.004185 2545 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2fa5830e-a011-4ed7-96ee-1f6b2da5f000-dshm\") on node \"ip-10-0-131-55.ec2.internal\" DevicePath \"\"" Jul 07 22:42:35.402402 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:42:35.402309 2545 generic.go:356] "Generic (PLEG): container finished" podID="2fa5830e-a011-4ed7-96ee-1f6b2da5f000" containerID="b02a2a0036396a629a3d50f1eecd4455f79c8e60855ae62f8df5f13cbb5f2d06" exitCode=0 Jul 07 22:42:35.402402 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:42:35.402367 2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-41e952/sub-test-model-41e952-kserve-6b89c984c-pbvb6" Jul 07 22:42:35.402402 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:42:35.402371 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-41e952/sub-test-model-41e952-kserve-6b89c984c-pbvb6" event={"ID":"2fa5830e-a011-4ed7-96ee-1f6b2da5f000","Type":"ContainerDied","Data":"b02a2a0036396a629a3d50f1eecd4455f79c8e60855ae62f8df5f13cbb5f2d06"} Jul 07 22:42:35.402649 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:42:35.402417 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-41e952/sub-test-model-41e952-kserve-6b89c984c-pbvb6" event={"ID":"2fa5830e-a011-4ed7-96ee-1f6b2da5f000","Type":"ContainerDied","Data":"ec1cf4bc7db7491eb9b2e751bf82225ee1a5462bddce585545bd60f6c525000a"} Jul 07 22:42:35.402649 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:42:35.402433 2545 scope.go:117] "RemoveContainer" containerID="b02a2a0036396a629a3d50f1eecd4455f79c8e60855ae62f8df5f13cbb5f2d06" Jul 07 22:42:35.411398 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:42:35.411380 2545 scope.go:117] "RemoveContainer" containerID="b02a2a0036396a629a3d50f1eecd4455f79c8e60855ae62f8df5f13cbb5f2d06" Jul 07 22:42:35.411642 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:42:35.411622 2545 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b02a2a0036396a629a3d50f1eecd4455f79c8e60855ae62f8df5f13cbb5f2d06\": container with ID starting with b02a2a0036396a629a3d50f1eecd4455f79c8e60855ae62f8df5f13cbb5f2d06 not found: ID does not exist" containerID="b02a2a0036396a629a3d50f1eecd4455f79c8e60855ae62f8df5f13cbb5f2d06" Jul 07 22:42:35.411678 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:42:35.411652 2545 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b02a2a0036396a629a3d50f1eecd4455f79c8e60855ae62f8df5f13cbb5f2d06"} err="failed to get container status \"b02a2a0036396a629a3d50f1eecd4455f79c8e60855ae62f8df5f13cbb5f2d06\": rpc error: code = NotFound desc = could not find container \"b02a2a0036396a629a3d50f1eecd4455f79c8e60855ae62f8df5f13cbb5f2d06\": container with ID starting with b02a2a0036396a629a3d50f1eecd4455f79c8e60855ae62f8df5f13cbb5f2d06 not found: ID does not exist" Jul 07 22:42:35.422224 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:42:35.422201 2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-41e952/sub-test-model-41e952-kserve-6b89c984c-pbvb6"] Jul 07 22:42:35.427344 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:42:35.427325 2545 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-41e952/sub-test-model-41e952-kserve-6b89c984c-pbvb6"] Jul 07 22:42:36.312349 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:42:36.312317 2545 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2fa5830e-a011-4ed7-96ee-1f6b2da5f000" path="/var/lib/kubelet/pods/2fa5830e-a011-4ed7-96ee-1f6b2da5f000/volumes" Jul 07 22:42:44.024947 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:42:44.024897 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-41e952/rate-test-model-41e952-kserve-6957b7d7bc-k4hdx" Jul 07 22:42:44.024947 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:42:44.024963 2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-41e952/rate-test-model-41e952-kserve-6957b7d7bc-k4hdx" Jul 07 22:42:44.037381 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:42:44.037355 2545 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-41e952/rate-test-model-41e952-kserve-6957b7d7bc-k4hdx" Jul 07 22:42:44.461103 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:42:44.461004 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-41e952/rate-test-model-41e952-kserve-6957b7d7bc-k4hdx" Jul 07 22:43:33.434437 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:43:33.434402 2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-41e952/rate-test-model-41e952-kserve-6957b7d7bc-k4hdx"] Jul 07 22:43:38.348187 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:43:38.348160 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-rhg5r_71f36bfa-4b8a-4572-b522-57c16392aa0b/console-operator/1.log" Jul 07 22:43:38.351162 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:43:38.351133 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-rhg5r_71f36bfa-4b8a-4572-b522-57c16392aa0b/console-operator/1.log" Jul 07 22:43:48.454950 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:43:48.454908 2545 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-41e952/rate-test-model-41e952-kserve-6957b7d7bc-k4hdx" podUID="8ad20429-54bd-4414-92fb-dc0554db927b" containerName="main" containerID="cri-o://bde32d9ddbfb2393c6246dd9cd2fe079b2010eb1d8c11b8ed8a66dacc19438e6" gracePeriod=45 Jul 07 22:43:48.699585 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:43:48.699560 2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-41e952/rate-test-model-41e952-kserve-6957b7d7bc-k4hdx" Jul 07 22:43:48.776850 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:43:48.776814 2545 generic.go:356] "Generic (PLEG): container finished" podID="8ad20429-54bd-4414-92fb-dc0554db927b" containerID="bde32d9ddbfb2393c6246dd9cd2fe079b2010eb1d8c11b8ed8a66dacc19438e6" exitCode=0 Jul 07 22:43:48.777024 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:43:48.776861 2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-41e952/rate-test-model-41e952-kserve-6957b7d7bc-k4hdx" Jul 07 22:43:48.777024 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:43:48.776903 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-41e952/rate-test-model-41e952-kserve-6957b7d7bc-k4hdx" event={"ID":"8ad20429-54bd-4414-92fb-dc0554db927b","Type":"ContainerDied","Data":"bde32d9ddbfb2393c6246dd9cd2fe079b2010eb1d8c11b8ed8a66dacc19438e6"} Jul 07 22:43:48.777024 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:43:48.776951 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-41e952/rate-test-model-41e952-kserve-6957b7d7bc-k4hdx" event={"ID":"8ad20429-54bd-4414-92fb-dc0554db927b","Type":"ContainerDied","Data":"8175e53a09b6b65e959ee121ddbd1e183698cdf3e931f6432a4d8abde5bcf1c2"} Jul 07 22:43:48.777024 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:43:48.776967 2545 scope.go:117] "RemoveContainer" containerID="bde32d9ddbfb2393c6246dd9cd2fe079b2010eb1d8c11b8ed8a66dacc19438e6" Jul 07 22:43:48.786071 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:43:48.786056 2545 scope.go:117] "RemoveContainer" containerID="bde32d9ddbfb2393c6246dd9cd2fe079b2010eb1d8c11b8ed8a66dacc19438e6" Jul 07 22:43:48.786379 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:43:48.786359 2545 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bde32d9ddbfb2393c6246dd9cd2fe079b2010eb1d8c11b8ed8a66dacc19438e6\": container with ID starting with bde32d9ddbfb2393c6246dd9cd2fe079b2010eb1d8c11b8ed8a66dacc19438e6 not found: ID does not exist" containerID="bde32d9ddbfb2393c6246dd9cd2fe079b2010eb1d8c11b8ed8a66dacc19438e6" Jul 07 22:43:48.786450 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:43:48.786387 2545 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bde32d9ddbfb2393c6246dd9cd2fe079b2010eb1d8c11b8ed8a66dacc19438e6"} err="failed to get container status \"bde32d9ddbfb2393c6246dd9cd2fe079b2010eb1d8c11b8ed8a66dacc19438e6\": rpc error: code = NotFound desc = could not find container \"bde32d9ddbfb2393c6246dd9cd2fe079b2010eb1d8c11b8ed8a66dacc19438e6\": container with ID starting with bde32d9ddbfb2393c6246dd9cd2fe079b2010eb1d8c11b8ed8a66dacc19438e6 not found: ID does not exist" Jul 07 22:43:48.822415 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:43:48.822390 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8ad20429-54bd-4414-92fb-dc0554db927b-dshm\") pod \"8ad20429-54bd-4414-92fb-dc0554db927b\" (UID: \"8ad20429-54bd-4414-92fb-dc0554db927b\") " Jul 07 22:43:48.822605 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:43:48.822442 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8ad20429-54bd-4414-92fb-dc0554db927b-tmp-dir\") pod \"8ad20429-54bd-4414-92fb-dc0554db927b\" (UID: \"8ad20429-54bd-4414-92fb-dc0554db927b\") " Jul 07 22:43:48.822605 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:43:48.822490 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8ad20429-54bd-4414-92fb-dc0554db927b-tls-certs\") pod \"8ad20429-54bd-4414-92fb-dc0554db927b\" (UID: \"8ad20429-54bd-4414-92fb-dc0554db927b\") " Jul 07 22:43:48.822605 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:43:48.822516 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xlx2m\" (UniqueName: \"kubernetes.io/projected/8ad20429-54bd-4414-92fb-dc0554db927b-kube-api-access-xlx2m\") pod \"8ad20429-54bd-4414-92fb-dc0554db927b\" (UID: \"8ad20429-54bd-4414-92fb-dc0554db927b\") " Jul 07 22:43:48.822783 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:43:48.822642 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8ad20429-54bd-4414-92fb-dc0554db927b-home\") pod \"8ad20429-54bd-4414-92fb-dc0554db927b\" (UID: \"8ad20429-54bd-4414-92fb-dc0554db927b\") " Jul 07 22:43:48.822783 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:43:48.822678 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8ad20429-54bd-4414-92fb-dc0554db927b-model-cache\") pod \"8ad20429-54bd-4414-92fb-dc0554db927b\" (UID: \"8ad20429-54bd-4414-92fb-dc0554db927b\") " Jul 07 22:43:48.822783 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:43:48.822757 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ad20429-54bd-4414-92fb-dc0554db927b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "8ad20429-54bd-4414-92fb-dc0554db927b" (UID: "8ad20429-54bd-4414-92fb-dc0554db927b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 22:43:48.822997 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:43:48.822975 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ad20429-54bd-4414-92fb-dc0554db927b-model-cache" (OuterVolumeSpecName: "model-cache") pod "8ad20429-54bd-4414-92fb-dc0554db927b" (UID: "8ad20429-54bd-4414-92fb-dc0554db927b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 22:43:48.823144 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:43:48.823128 2545 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8ad20429-54bd-4414-92fb-dc0554db927b-model-cache\") on node \"ip-10-0-131-55.ec2.internal\" DevicePath \"\"" Jul 07 22:43:48.823223 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:43:48.823134 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ad20429-54bd-4414-92fb-dc0554db927b-home" (OuterVolumeSpecName: "home") pod "8ad20429-54bd-4414-92fb-dc0554db927b" (UID: "8ad20429-54bd-4414-92fb-dc0554db927b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 22:43:48.823223 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:43:48.823151 2545 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8ad20429-54bd-4414-92fb-dc0554db927b-tmp-dir\") on node \"ip-10-0-131-55.ec2.internal\" DevicePath \"\"" Jul 07 22:43:48.824864 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:43:48.824841 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ad20429-54bd-4414-92fb-dc0554db927b-dshm" (OuterVolumeSpecName: "dshm") pod "8ad20429-54bd-4414-92fb-dc0554db927b" (UID: "8ad20429-54bd-4414-92fb-dc0554db927b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 22:43:48.824973 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:43:48.824888 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8ad20429-54bd-4414-92fb-dc0554db927b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "8ad20429-54bd-4414-92fb-dc0554db927b" (UID: "8ad20429-54bd-4414-92fb-dc0554db927b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 07 22:43:48.824973 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:43:48.824928 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8ad20429-54bd-4414-92fb-dc0554db927b-kube-api-access-xlx2m" (OuterVolumeSpecName: "kube-api-access-xlx2m") pod "8ad20429-54bd-4414-92fb-dc0554db927b" (UID: "8ad20429-54bd-4414-92fb-dc0554db927b"). InnerVolumeSpecName "kube-api-access-xlx2m". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 22:43:48.924001 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:43:48.923962 2545 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8ad20429-54bd-4414-92fb-dc0554db927b-home\") on node \"ip-10-0-131-55.ec2.internal\" DevicePath \"\"" Jul 07 22:43:48.924001 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:43:48.923994 2545 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8ad20429-54bd-4414-92fb-dc0554db927b-dshm\") on node \"ip-10-0-131-55.ec2.internal\" DevicePath \"\"" Jul 07 22:43:48.924001 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:43:48.924002 2545 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8ad20429-54bd-4414-92fb-dc0554db927b-tls-certs\") on node \"ip-10-0-131-55.ec2.internal\" DevicePath \"\"" Jul 07 22:43:48.924273 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:43:48.924013 2545 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xlx2m\" (UniqueName: \"kubernetes.io/projected/8ad20429-54bd-4414-92fb-dc0554db927b-kube-api-access-xlx2m\") on node \"ip-10-0-131-55.ec2.internal\" DevicePath \"\"" Jul 07 22:43:49.100322 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:43:49.100297 2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-41e952/rate-test-model-41e952-kserve-6957b7d7bc-k4hdx"] Jul 07 22:43:49.102794 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:43:49.102770 2545 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-41e952/rate-test-model-41e952-kserve-6957b7d7bc-k4hdx"] Jul 07 22:43:50.311609 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:43:50.311569 2545 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8ad20429-54bd-4414-92fb-dc0554db927b" path="/var/lib/kubelet/pods/8ad20429-54bd-4414-92fb-dc0554db927b/volumes" Jul 07 22:44:17.892018 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:17.891939 2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-41e952-openshift-default-bf4779887-95k56"] Jul 07 22:44:17.892557 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:17.892191 2545 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-41e952-openshift-default-bf4779887-95k56" podUID="18408c49-ad8e-48b6-97b3-dd10ca1565e5" containerName="istio-proxy" containerID="cri-o://f73cbc580da5623177ef47d7a9dc6407d2a9656bea61f736d5042faa4cfd1b09" gracePeriod=30 Jul 07 22:44:23.148124 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:23.148094 2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-41e952-openshift-default-bf4779887-95k56" Jul 07 22:44:23.245233 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:23.245200 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/18408c49-ad8e-48b6-97b3-dd10ca1565e5-istio-data\") pod \"18408c49-ad8e-48b6-97b3-dd10ca1565e5\" (UID: \"18408c49-ad8e-48b6-97b3-dd10ca1565e5\") " Jul 07 22:44:23.245431 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:23.245306 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/18408c49-ad8e-48b6-97b3-dd10ca1565e5-credential-socket\") pod \"18408c49-ad8e-48b6-97b3-dd10ca1565e5\" (UID: \"18408c49-ad8e-48b6-97b3-dd10ca1565e5\") " Jul 07 22:44:23.245431 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:23.245341 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/18408c49-ad8e-48b6-97b3-dd10ca1565e5-istiod-ca-cert\") pod \"18408c49-ad8e-48b6-97b3-dd10ca1565e5\" (UID: \"18408c49-ad8e-48b6-97b3-dd10ca1565e5\") " Jul 07 22:44:23.245431 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:23.245362 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/18408c49-ad8e-48b6-97b3-dd10ca1565e5-workload-certs\") pod \"18408c49-ad8e-48b6-97b3-dd10ca1565e5\" (UID: \"18408c49-ad8e-48b6-97b3-dd10ca1565e5\") " Jul 07 22:44:23.245669 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:23.245447 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/18408c49-ad8e-48b6-97b3-dd10ca1565e5-istio-token\") pod \"18408c49-ad8e-48b6-97b3-dd10ca1565e5\" (UID: \"18408c49-ad8e-48b6-97b3-dd10ca1565e5\") " Jul 07 22:44:23.245669 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:23.245471 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wv6rn\" (UniqueName: \"kubernetes.io/projected/18408c49-ad8e-48b6-97b3-dd10ca1565e5-kube-api-access-wv6rn\") pod \"18408c49-ad8e-48b6-97b3-dd10ca1565e5\" (UID: \"18408c49-ad8e-48b6-97b3-dd10ca1565e5\") " Jul 07 22:44:23.245669 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:23.245504 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/18408c49-ad8e-48b6-97b3-dd10ca1565e5-istio-podinfo\") pod \"18408c49-ad8e-48b6-97b3-dd10ca1565e5\" (UID: \"18408c49-ad8e-48b6-97b3-dd10ca1565e5\") " Jul 07 22:44:23.245669 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:23.245544 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18408c49-ad8e-48b6-97b3-dd10ca1565e5-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "18408c49-ad8e-48b6-97b3-dd10ca1565e5" (UID: "18408c49-ad8e-48b6-97b3-dd10ca1565e5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 22:44:23.245901 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:23.245741 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/18408c49-ad8e-48b6-97b3-dd10ca1565e5-workload-socket\") pod \"18408c49-ad8e-48b6-97b3-dd10ca1565e5\" (UID: \"18408c49-ad8e-48b6-97b3-dd10ca1565e5\") " Jul 07 22:44:23.245901 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:23.245750 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/18408c49-ad8e-48b6-97b3-dd10ca1565e5-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "18408c49-ad8e-48b6-97b3-dd10ca1565e5" (UID: "18408c49-ad8e-48b6-97b3-dd10ca1565e5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 22:44:23.245901 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:23.245758 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18408c49-ad8e-48b6-97b3-dd10ca1565e5-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "18408c49-ad8e-48b6-97b3-dd10ca1565e5" (UID: "18408c49-ad8e-48b6-97b3-dd10ca1565e5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 22:44:23.245901 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:23.245780 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/18408c49-ad8e-48b6-97b3-dd10ca1565e5-istio-envoy\") pod \"18408c49-ad8e-48b6-97b3-dd10ca1565e5\" (UID: \"18408c49-ad8e-48b6-97b3-dd10ca1565e5\") " Jul 07 22:44:23.246131 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:23.245996 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18408c49-ad8e-48b6-97b3-dd10ca1565e5-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "18408c49-ad8e-48b6-97b3-dd10ca1565e5" (UID: "18408c49-ad8e-48b6-97b3-dd10ca1565e5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 22:44:23.246206 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:23.246165 2545 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/18408c49-ad8e-48b6-97b3-dd10ca1565e5-workload-socket\") on node \"ip-10-0-131-55.ec2.internal\" DevicePath \"\"" Jul 07 22:44:23.246206 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:23.246182 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18408c49-ad8e-48b6-97b3-dd10ca1565e5-istio-data" (OuterVolumeSpecName: "istio-data") pod "18408c49-ad8e-48b6-97b3-dd10ca1565e5" (UID: "18408c49-ad8e-48b6-97b3-dd10ca1565e5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 22:44:23.246206 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:23.246187 2545 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/18408c49-ad8e-48b6-97b3-dd10ca1565e5-credential-socket\") on node \"ip-10-0-131-55.ec2.internal\" DevicePath \"\"" Jul 07 22:44:23.246362 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:23.246221 2545 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/18408c49-ad8e-48b6-97b3-dd10ca1565e5-istiod-ca-cert\") on node \"ip-10-0-131-55.ec2.internal\" DevicePath \"\"" Jul 07 22:44:23.246362 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:23.246232 2545 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/18408c49-ad8e-48b6-97b3-dd10ca1565e5-workload-certs\") on node \"ip-10-0-131-55.ec2.internal\" DevicePath \"\"" Jul 07 22:44:23.247802 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:23.247767 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/18408c49-ad8e-48b6-97b3-dd10ca1565e5-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "18408c49-ad8e-48b6-97b3-dd10ca1565e5" (UID: "18408c49-ad8e-48b6-97b3-dd10ca1565e5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 22:44:23.247802 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:23.247772 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18408c49-ad8e-48b6-97b3-dd10ca1565e5-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "18408c49-ad8e-48b6-97b3-dd10ca1565e5" (UID: "18408c49-ad8e-48b6-97b3-dd10ca1565e5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 22:44:23.248026 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:23.248009 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/18408c49-ad8e-48b6-97b3-dd10ca1565e5-istio-token" (OuterVolumeSpecName: "istio-token") pod "18408c49-ad8e-48b6-97b3-dd10ca1565e5" (UID: "18408c49-ad8e-48b6-97b3-dd10ca1565e5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 22:44:23.248157 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:23.248142 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/18408c49-ad8e-48b6-97b3-dd10ca1565e5-kube-api-access-wv6rn" (OuterVolumeSpecName: "kube-api-access-wv6rn") pod "18408c49-ad8e-48b6-97b3-dd10ca1565e5" (UID: "18408c49-ad8e-48b6-97b3-dd10ca1565e5"). InnerVolumeSpecName "kube-api-access-wv6rn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 22:44:23.346854 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:23.346825 2545 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/18408c49-ad8e-48b6-97b3-dd10ca1565e5-istio-token\") on node \"ip-10-0-131-55.ec2.internal\" DevicePath \"\"" Jul 07 22:44:23.346854 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:23.346850 2545 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wv6rn\" (UniqueName: \"kubernetes.io/projected/18408c49-ad8e-48b6-97b3-dd10ca1565e5-kube-api-access-wv6rn\") on node \"ip-10-0-131-55.ec2.internal\" DevicePath \"\"" Jul 07 22:44:23.346854 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:23.346860 2545 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/18408c49-ad8e-48b6-97b3-dd10ca1565e5-istio-podinfo\") on node \"ip-10-0-131-55.ec2.internal\" DevicePath \"\"" Jul 07 22:44:23.347066 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:23.346870 2545 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/18408c49-ad8e-48b6-97b3-dd10ca1565e5-istio-envoy\") on node \"ip-10-0-131-55.ec2.internal\" DevicePath \"\"" Jul 07 22:44:23.347066 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:23.346880 2545 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/18408c49-ad8e-48b6-97b3-dd10ca1565e5-istio-data\") on node \"ip-10-0-131-55.ec2.internal\" DevicePath \"\"" Jul 07 22:44:23.953469 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:23.953439 2545 generic.go:356] "Generic (PLEG): container finished" podID="18408c49-ad8e-48b6-97b3-dd10ca1565e5" containerID="f73cbc580da5623177ef47d7a9dc6407d2a9656bea61f736d5042faa4cfd1b09" exitCode=0 Jul 07 22:44:23.953651 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:23.953487 2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-41e952-openshift-default-bf4779887-95k56" Jul 07 22:44:23.953651 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:23.953532 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-41e952-openshift-default-bf4779887-95k56" event={"ID":"18408c49-ad8e-48b6-97b3-dd10ca1565e5","Type":"ContainerDied","Data":"f73cbc580da5623177ef47d7a9dc6407d2a9656bea61f736d5042faa4cfd1b09"} Jul 07 22:44:23.953651 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:23.953591 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-41e952-openshift-default-bf4779887-95k56" event={"ID":"18408c49-ad8e-48b6-97b3-dd10ca1565e5","Type":"ContainerDied","Data":"3c4a3524de43a00cbe75687655124965d7ef97ecb560dccc5ef085674ab10fb5"} Jul 07 22:44:23.953651 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:23.953619 2545 scope.go:117] "RemoveContainer" containerID="f73cbc580da5623177ef47d7a9dc6407d2a9656bea61f736d5042faa4cfd1b09" Jul 07 22:44:23.963121 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:23.963067 2545 scope.go:117] "RemoveContainer" containerID="57cd81887d6c748d26fca3aaa75406bce5ab4b963cdd3680cc22939bd6b3b167" Jul 07 22:44:23.970589 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:23.970568 2545 scope.go:117] "RemoveContainer" containerID="f73cbc580da5623177ef47d7a9dc6407d2a9656bea61f736d5042faa4cfd1b09" Jul 07 22:44:23.970827 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:44:23.970810 2545 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f73cbc580da5623177ef47d7a9dc6407d2a9656bea61f736d5042faa4cfd1b09\": container with ID starting with f73cbc580da5623177ef47d7a9dc6407d2a9656bea61f736d5042faa4cfd1b09 not found: ID does not exist" containerID="f73cbc580da5623177ef47d7a9dc6407d2a9656bea61f736d5042faa4cfd1b09" Jul 07 22:44:23.970866 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:23.970838 2545 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f73cbc580da5623177ef47d7a9dc6407d2a9656bea61f736d5042faa4cfd1b09"} err="failed to get container status \"f73cbc580da5623177ef47d7a9dc6407d2a9656bea61f736d5042faa4cfd1b09\": rpc error: code = NotFound desc = could not find container \"f73cbc580da5623177ef47d7a9dc6407d2a9656bea61f736d5042faa4cfd1b09\": container with ID starting with f73cbc580da5623177ef47d7a9dc6407d2a9656bea61f736d5042faa4cfd1b09 not found: ID does not exist" Jul 07 22:44:23.970866 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:23.970855 2545 scope.go:117] "RemoveContainer" containerID="57cd81887d6c748d26fca3aaa75406bce5ab4b963cdd3680cc22939bd6b3b167" Jul 07 22:44:23.971123 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:44:23.971095 2545 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"57cd81887d6c748d26fca3aaa75406bce5ab4b963cdd3680cc22939bd6b3b167\": container with ID starting with 57cd81887d6c748d26fca3aaa75406bce5ab4b963cdd3680cc22939bd6b3b167 not found: ID does not exist" containerID="57cd81887d6c748d26fca3aaa75406bce5ab4b963cdd3680cc22939bd6b3b167" Jul 07 22:44:23.971166 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:23.971133 2545 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"57cd81887d6c748d26fca3aaa75406bce5ab4b963cdd3680cc22939bd6b3b167"} err="failed to get container status \"57cd81887d6c748d26fca3aaa75406bce5ab4b963cdd3680cc22939bd6b3b167\": rpc error: code = NotFound desc = could not find container \"57cd81887d6c748d26fca3aaa75406bce5ab4b963cdd3680cc22939bd6b3b167\": container with ID starting with 57cd81887d6c748d26fca3aaa75406bce5ab4b963cdd3680cc22939bd6b3b167 not found: ID does not exist" Jul 07 22:44:23.976658 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:23.976636 2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-41e952-openshift-default-bf4779887-95k56"] Jul 07 22:44:23.981207 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:23.981185 2545 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-41e952-openshift-default-bf4779887-95k56"] Jul 07 22:44:24.317290 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:24.317247 2545 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="18408c49-ad8e-48b6-97b3-dd10ca1565e5" path="/var/lib/kubelet/pods/18408c49-ad8e-48b6-97b3-dd10ca1565e5/volumes" Jul 07 22:44:24.605130 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:24.605017 2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-a4bfe6-openshift-default-7f69fb97f-hqxx5"] Jul 07 22:44:24.605287 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:24.605261 2545 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-a4bfe6-openshift-default-7f69fb97f-hqxx5" podUID="b15b945e-9940-4332-8f1a-064a4258b101" containerName="istio-proxy" containerID="cri-o://7f2f4a5df5fe59e4107843bd89e5922310564f7fb3adf7888000de69551a106f" gracePeriod=30 Jul 07 22:44:25.323671 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:25.323644 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6c9b657c9c-jqr7l_303bd640-3c9c-4a1f-8cd6-5d3fb366af14/authorino/0.log" Jul 07 22:44:29.862400 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:29.862373 2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-a4bfe6-openshift-default-7f69fb97f-hqxx5" Jul 07 22:44:29.984520 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:29.984486 2545 generic.go:356] "Generic (PLEG): container finished" podID="b15b945e-9940-4332-8f1a-064a4258b101" containerID="7f2f4a5df5fe59e4107843bd89e5922310564f7fb3adf7888000de69551a106f" exitCode=0 Jul 07 22:44:29.984710 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:29.984530 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-a4bfe6-openshift-default-7f69fb97f-hqxx5" event={"ID":"b15b945e-9940-4332-8f1a-064a4258b101","Type":"ContainerDied","Data":"7f2f4a5df5fe59e4107843bd89e5922310564f7fb3adf7888000de69551a106f"} Jul 07 22:44:29.984710 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:29.984547 2545 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-a4bfe6-openshift-default-7f69fb97f-hqxx5" Jul 07 22:44:29.984710 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:29.984600 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-a4bfe6-openshift-default-7f69fb97f-hqxx5" event={"ID":"b15b945e-9940-4332-8f1a-064a4258b101","Type":"ContainerDied","Data":"9c35f23d703826561abd1f32a309b600f6470f27a44792d7170da9f91020bce6"} Jul 07 22:44:29.984710 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:29.984622 2545 scope.go:117] "RemoveContainer" containerID="7f2f4a5df5fe59e4107843bd89e5922310564f7fb3adf7888000de69551a106f" Jul 07 22:44:29.993136 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:29.993116 2545 scope.go:117] "RemoveContainer" containerID="3255f5daa9322957268313f2f4613ce51636e4c96582468b3fa55511cf798c75" Jul 07 22:44:30.000850 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:30.000833 2545 scope.go:117] "RemoveContainer" containerID="7f2f4a5df5fe59e4107843bd89e5922310564f7fb3adf7888000de69551a106f" Jul 07 22:44:30.001156 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:44:30.001138 2545 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7f2f4a5df5fe59e4107843bd89e5922310564f7fb3adf7888000de69551a106f\": container with ID starting with 7f2f4a5df5fe59e4107843bd89e5922310564f7fb3adf7888000de69551a106f not found: ID does not exist" containerID="7f2f4a5df5fe59e4107843bd89e5922310564f7fb3adf7888000de69551a106f" Jul 07 22:44:30.001216 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:30.001166 2545 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f2f4a5df5fe59e4107843bd89e5922310564f7fb3adf7888000de69551a106f"} err="failed to get container status \"7f2f4a5df5fe59e4107843bd89e5922310564f7fb3adf7888000de69551a106f\": rpc error: code = NotFound desc = could not find container \"7f2f4a5df5fe59e4107843bd89e5922310564f7fb3adf7888000de69551a106f\": container with ID starting with 7f2f4a5df5fe59e4107843bd89e5922310564f7fb3adf7888000de69551a106f not found: ID does not exist" Jul 07 22:44:30.001216 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:30.001183 2545 scope.go:117] "RemoveContainer" containerID="3255f5daa9322957268313f2f4613ce51636e4c96582468b3fa55511cf798c75" Jul 07 22:44:30.001441 ip-10-0-131-55 kubenswrapper[2545]: E0707 22:44:30.001425 2545 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3255f5daa9322957268313f2f4613ce51636e4c96582468b3fa55511cf798c75\": container with ID starting with 3255f5daa9322957268313f2f4613ce51636e4c96582468b3fa55511cf798c75 not found: ID does not exist" containerID="3255f5daa9322957268313f2f4613ce51636e4c96582468b3fa55511cf798c75" Jul 07 22:44:30.001493 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:30.001445 2545 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3255f5daa9322957268313f2f4613ce51636e4c96582468b3fa55511cf798c75"} err="failed to get container status \"3255f5daa9322957268313f2f4613ce51636e4c96582468b3fa55511cf798c75\": rpc error: code = NotFound desc = could not find container \"3255f5daa9322957268313f2f4613ce51636e4c96582468b3fa55511cf798c75\": container with ID starting with 3255f5daa9322957268313f2f4613ce51636e4c96582468b3fa55511cf798c75 not found: ID does not exist" Jul 07 22:44:30.011736 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:30.011716 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b15b945e-9940-4332-8f1a-064a4258b101-istio-token\") pod \"b15b945e-9940-4332-8f1a-064a4258b101\" (UID: \"b15b945e-9940-4332-8f1a-064a4258b101\") " Jul 07 22:44:30.011811 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:30.011786 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b15b945e-9940-4332-8f1a-064a4258b101-istiod-ca-cert\") pod \"b15b945e-9940-4332-8f1a-064a4258b101\" (UID: \"b15b945e-9940-4332-8f1a-064a4258b101\") " Jul 07 22:44:30.011855 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:30.011815 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b15b945e-9940-4332-8f1a-064a4258b101-credential-socket\") pod \"b15b945e-9940-4332-8f1a-064a4258b101\" (UID: \"b15b945e-9940-4332-8f1a-064a4258b101\") " Jul 07 22:44:30.011855 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:30.011840 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b15b945e-9940-4332-8f1a-064a4258b101-istio-podinfo\") pod \"b15b945e-9940-4332-8f1a-064a4258b101\" (UID: \"b15b945e-9940-4332-8f1a-064a4258b101\") " Jul 07 22:44:30.011963 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:30.011898 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b15b945e-9940-4332-8f1a-064a4258b101-workload-socket\") pod \"b15b945e-9940-4332-8f1a-064a4258b101\" (UID: \"b15b945e-9940-4332-8f1a-064a4258b101\") " Jul 07 22:44:30.011963 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:30.011929 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b15b945e-9940-4332-8f1a-064a4258b101-workload-certs\") pod \"b15b945e-9940-4332-8f1a-064a4258b101\" (UID: \"b15b945e-9940-4332-8f1a-064a4258b101\") " Jul 07 22:44:30.012207 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:30.012122 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b15b945e-9940-4332-8f1a-064a4258b101-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b15b945e-9940-4332-8f1a-064a4258b101" (UID: "b15b945e-9940-4332-8f1a-064a4258b101"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 22:44:30.012207 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:30.012177 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b15b945e-9940-4332-8f1a-064a4258b101-istio-envoy\") pod \"b15b945e-9940-4332-8f1a-064a4258b101\" (UID: \"b15b945e-9940-4332-8f1a-064a4258b101\") " Jul 07 22:44:30.012365 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:30.012229 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b15b945e-9940-4332-8f1a-064a4258b101-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b15b945e-9940-4332-8f1a-064a4258b101" (UID: "b15b945e-9940-4332-8f1a-064a4258b101"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 22:44:30.012365 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:30.012245 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b15b945e-9940-4332-8f1a-064a4258b101-istio-data\") pod \"b15b945e-9940-4332-8f1a-064a4258b101\" (UID: \"b15b945e-9940-4332-8f1a-064a4258b101\") " Jul 07 22:44:30.012365 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:30.012242 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b15b945e-9940-4332-8f1a-064a4258b101-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b15b945e-9940-4332-8f1a-064a4258b101" (UID: "b15b945e-9940-4332-8f1a-064a4258b101"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 22:44:30.012365 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:30.012278 2545 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-trm5j\" (UniqueName: \"kubernetes.io/projected/b15b945e-9940-4332-8f1a-064a4258b101-kube-api-access-trm5j\") pod \"b15b945e-9940-4332-8f1a-064a4258b101\" (UID: \"b15b945e-9940-4332-8f1a-064a4258b101\") " Jul 07 22:44:30.012365 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:30.012298 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b15b945e-9940-4332-8f1a-064a4258b101-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b15b945e-9940-4332-8f1a-064a4258b101" (UID: "b15b945e-9940-4332-8f1a-064a4258b101"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 07 22:44:30.012617 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:30.012600 2545 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b15b945e-9940-4332-8f1a-064a4258b101-istiod-ca-cert\") on node \"ip-10-0-131-55.ec2.internal\" DevicePath \"\"" Jul 07 22:44:30.012617 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:30.012615 2545 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b15b945e-9940-4332-8f1a-064a4258b101-credential-socket\") on node \"ip-10-0-131-55.ec2.internal\" DevicePath \"\"" Jul 07 22:44:30.012736 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:30.012624 2545 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b15b945e-9940-4332-8f1a-064a4258b101-workload-socket\") on node \"ip-10-0-131-55.ec2.internal\" DevicePath \"\"" Jul 07 22:44:30.012736 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:30.012633 2545 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b15b945e-9940-4332-8f1a-064a4258b101-workload-certs\") on node \"ip-10-0-131-55.ec2.internal\" DevicePath \"\"" Jul 07 22:44:30.013208 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:30.013184 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b15b945e-9940-4332-8f1a-064a4258b101-istio-data" (OuterVolumeSpecName: "istio-data") pod "b15b945e-9940-4332-8f1a-064a4258b101" (UID: "b15b945e-9940-4332-8f1a-064a4258b101"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 22:44:30.014188 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:30.014165 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b15b945e-9940-4332-8f1a-064a4258b101-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b15b945e-9940-4332-8f1a-064a4258b101" (UID: "b15b945e-9940-4332-8f1a-064a4258b101"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 07 22:44:30.014320 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:30.014304 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b15b945e-9940-4332-8f1a-064a4258b101-istio-token" (OuterVolumeSpecName: "istio-token") pod "b15b945e-9940-4332-8f1a-064a4258b101" (UID: "b15b945e-9940-4332-8f1a-064a4258b101"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 22:44:30.014504 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:30.014482 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b15b945e-9940-4332-8f1a-064a4258b101-kube-api-access-trm5j" (OuterVolumeSpecName: "kube-api-access-trm5j") pod "b15b945e-9940-4332-8f1a-064a4258b101" (UID: "b15b945e-9940-4332-8f1a-064a4258b101"). InnerVolumeSpecName "kube-api-access-trm5j". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 07 22:44:30.014571 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:30.014488 2545 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b15b945e-9940-4332-8f1a-064a4258b101-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b15b945e-9940-4332-8f1a-064a4258b101" (UID: "b15b945e-9940-4332-8f1a-064a4258b101"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 07 22:44:30.113702 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:30.113665 2545 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b15b945e-9940-4332-8f1a-064a4258b101-istio-data\") on node \"ip-10-0-131-55.ec2.internal\" DevicePath \"\"" Jul 07 22:44:30.113702 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:30.113696 2545 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-trm5j\" (UniqueName: \"kubernetes.io/projected/b15b945e-9940-4332-8f1a-064a4258b101-kube-api-access-trm5j\") on node \"ip-10-0-131-55.ec2.internal\" DevicePath \"\"" Jul 07 22:44:30.113702 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:30.113706 2545 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b15b945e-9940-4332-8f1a-064a4258b101-istio-token\") on node \"ip-10-0-131-55.ec2.internal\" DevicePath \"\"" Jul 07 22:44:30.113935 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:30.113714 2545 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b15b945e-9940-4332-8f1a-064a4258b101-istio-podinfo\") on node \"ip-10-0-131-55.ec2.internal\" DevicePath \"\"" Jul 07 22:44:30.113935 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:30.113723 2545 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b15b945e-9940-4332-8f1a-064a4258b101-istio-envoy\") on node \"ip-10-0-131-55.ec2.internal\" DevicePath \"\"" Jul 07 22:44:30.311974 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:30.311943 2545 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-a4bfe6-openshift-default-7f69fb97f-hqxx5"] Jul 07 22:44:30.311974 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:30.311981 2545 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-a4bfe6-openshift-default-7f69fb97f-hqxx5"] Jul 07 22:44:31.211998 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:31.211970 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-597db99f65-jjzbb_559369c5-4d6c-427a-90c7-2c49e47827ac/copy-manifests/0.log" Jul 07 22:44:31.219235 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:31.219215 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-597db99f65-jjzbb_559369c5-4d6c-427a-90c7-2c49e47827ac/manager/0.log" Jul 07 22:44:32.312795 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:32.312761 2545 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b15b945e-9940-4332-8f1a-064a4258b101" path="/var/lib/kubelet/pods/b15b945e-9940-4332-8f1a-064a4258b101/volumes" Jul 07 22:44:32.603915 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:32.603793 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6c9b657c9c-jqr7l_303bd640-3c9c-4a1f-8cd6-5d3fb366af14/authorino/0.log" Jul 07 22:44:32.834566 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:32.834527 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-5z2vw_4687ae14-1e22-4218-9fec-004e72063d8b/manager/0.log" Jul 07 22:44:32.942133 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:32.942038 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-m8nxm_6941be32-c4d0-4973-87ec-11caa5ee0fc9/kuadrant-console-plugin/0.log" Jul 07 22:44:33.445171 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:33.445139 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-sqptd_1e06e247-ff08-43e1-a884-79bb36006378/manager/0.log" Jul 07 22:44:34.024942 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:34.024916 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6b4fccd749-28d8d_980381c5-f1a1-41ea-ba0f-0a582f6c2577/kube-auth-proxy/0.log" Jul 07 22:44:34.641798 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:34.641768 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-67496c4dcc-nnvc2_84447fdf-4f2d-45fd-9e94-acce197ae75b/router/0.log" Jul 07 22:44:35.103074 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:35.103038 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-wtq5s_302c7937-e5b4-49b8-a96f-1600395c20e1/main/0.log" Jul 07 22:44:35.547458 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:35.547420 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-7m5n5_55987ec6-aed9-47dd-8410-9aebcfcfa32d/main/0.log" Jul 07 22:44:39.600348 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:39.600314 2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-kzlgw/must-gather-vnpqx"] Jul 07 22:44:39.600720 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:39.600681 2545 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2fa5830e-a011-4ed7-96ee-1f6b2da5f000" containerName="main" Jul 07 22:44:39.600720 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:39.600694 2545 state_mem.go:107] "Deleted CPUSet assignment" podUID="2fa5830e-a011-4ed7-96ee-1f6b2da5f000" containerName="main" Jul 07 22:44:39.600720 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:39.600709 2545 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="18408c49-ad8e-48b6-97b3-dd10ca1565e5" containerName="istio-proxy" Jul 07 22:44:39.600720 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:39.600714 2545 state_mem.go:107] "Deleted CPUSet assignment" podUID="18408c49-ad8e-48b6-97b3-dd10ca1565e5" containerName="istio-proxy" Jul 07 22:44:39.600846 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:39.600722 2545 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b15b945e-9940-4332-8f1a-064a4258b101" containerName="istio-proxy" Jul 07 22:44:39.600846 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:39.600728 2545 state_mem.go:107] "Deleted CPUSet assignment" podUID="b15b945e-9940-4332-8f1a-064a4258b101" containerName="istio-proxy" Jul 07 22:44:39.600846 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:39.600741 2545 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="18408c49-ad8e-48b6-97b3-dd10ca1565e5" containerName="istio-proxy" Jul 07 22:44:39.600846 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:39.600746 2545 state_mem.go:107] "Deleted CPUSet assignment" podUID="18408c49-ad8e-48b6-97b3-dd10ca1565e5" containerName="istio-proxy" Jul 07 22:44:39.600846 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:39.600752 2545 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b15b945e-9940-4332-8f1a-064a4258b101" containerName="istio-proxy" Jul 07 22:44:39.600846 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:39.600758 2545 state_mem.go:107] "Deleted CPUSet assignment" podUID="b15b945e-9940-4332-8f1a-064a4258b101" containerName="istio-proxy" Jul 07 22:44:39.600846 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:39.600765 2545 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ad20429-54bd-4414-92fb-dc0554db927b" containerName="main" Jul 07 22:44:39.600846 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:39.600770 2545 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ad20429-54bd-4414-92fb-dc0554db927b" containerName="main" Jul 07 22:44:39.600846 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:39.600836 2545 memory_manager.go:368] "RemoveStaleState removing state" podUID="18408c49-ad8e-48b6-97b3-dd10ca1565e5" containerName="istio-proxy" Jul 07 22:44:39.600846 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:39.600843 2545 memory_manager.go:368] "RemoveStaleState removing state" podUID="b15b945e-9940-4332-8f1a-064a4258b101" containerName="istio-proxy" Jul 07 22:44:39.600846 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:39.600850 2545 memory_manager.go:368] "RemoveStaleState removing state" podUID="8ad20429-54bd-4414-92fb-dc0554db927b" containerName="main" Jul 07 22:44:39.601198 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:39.600858 2545 memory_manager.go:368] "RemoveStaleState removing state" podUID="2fa5830e-a011-4ed7-96ee-1f6b2da5f000" containerName="main" Jul 07 22:44:39.601198 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:39.600865 2545 memory_manager.go:368] "RemoveStaleState removing state" podUID="b15b945e-9940-4332-8f1a-064a4258b101" containerName="istio-proxy" Jul 07 22:44:39.601198 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:39.600970 2545 memory_manager.go:368] "RemoveStaleState removing state" podUID="18408c49-ad8e-48b6-97b3-dd10ca1565e5" containerName="istio-proxy" Jul 07 22:44:39.605519 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:39.605501 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-kzlgw/must-gather-vnpqx" Jul 07 22:44:39.608175 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:39.608152 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-kzlgw\"/\"kube-root-ca.crt\"" Jul 07 22:44:39.608317 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:39.608195 2545 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-kzlgw\"/\"openshift-service-ca.crt\"" Jul 07 22:44:39.608985 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:39.608968 2545 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-kzlgw\"/\"default-dockercfg-7bpws\"" Jul 07 22:44:39.613206 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:39.613186 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-kzlgw/must-gather-vnpqx"] Jul 07 22:44:39.703380 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:39.703347 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/2e5babe5-f8ca-495d-a425-b11fcf4ffbdf-must-gather-output\") pod \"must-gather-vnpqx\" (UID: \"2e5babe5-f8ca-495d-a425-b11fcf4ffbdf\") " pod="openshift-must-gather-kzlgw/must-gather-vnpqx" Jul 07 22:44:39.703552 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:39.703398 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dmxss\" (UniqueName: \"kubernetes.io/projected/2e5babe5-f8ca-495d-a425-b11fcf4ffbdf-kube-api-access-dmxss\") pod \"must-gather-vnpqx\" (UID: \"2e5babe5-f8ca-495d-a425-b11fcf4ffbdf\") " pod="openshift-must-gather-kzlgw/must-gather-vnpqx" Jul 07 22:44:39.804610 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:39.804568 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/2e5babe5-f8ca-495d-a425-b11fcf4ffbdf-must-gather-output\") pod \"must-gather-vnpqx\" (UID: \"2e5babe5-f8ca-495d-a425-b11fcf4ffbdf\") " pod="openshift-must-gather-kzlgw/must-gather-vnpqx" Jul 07 22:44:39.804796 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:39.804628 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dmxss\" (UniqueName: \"kubernetes.io/projected/2e5babe5-f8ca-495d-a425-b11fcf4ffbdf-kube-api-access-dmxss\") pod \"must-gather-vnpqx\" (UID: \"2e5babe5-f8ca-495d-a425-b11fcf4ffbdf\") " pod="openshift-must-gather-kzlgw/must-gather-vnpqx" Jul 07 22:44:39.804908 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:39.804887 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/2e5babe5-f8ca-495d-a425-b11fcf4ffbdf-must-gather-output\") pod \"must-gather-vnpqx\" (UID: \"2e5babe5-f8ca-495d-a425-b11fcf4ffbdf\") " pod="openshift-must-gather-kzlgw/must-gather-vnpqx" Jul 07 22:44:39.814479 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:39.814447 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dmxss\" (UniqueName: \"kubernetes.io/projected/2e5babe5-f8ca-495d-a425-b11fcf4ffbdf-kube-api-access-dmxss\") pod \"must-gather-vnpqx\" (UID: \"2e5babe5-f8ca-495d-a425-b11fcf4ffbdf\") " pod="openshift-must-gather-kzlgw/must-gather-vnpqx" Jul 07 22:44:39.917993 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:39.917891 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-kzlgw/must-gather-vnpqx" Jul 07 22:44:40.048681 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:40.048653 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-kzlgw/must-gather-vnpqx"] Jul 07 22:44:40.050488 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:44:40.050461 2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2e5babe5_f8ca_495d_a425_b11fcf4ffbdf.slice/crio-554cb0b7b2e8418521debe7452ea732b07332059d47ec6a786422507f4d70cc6 WatchSource:0}: Error finding container 554cb0b7b2e8418521debe7452ea732b07332059d47ec6a786422507f4d70cc6: Status 404 returned error can't find the container with id 554cb0b7b2e8418521debe7452ea732b07332059d47ec6a786422507f4d70cc6 Jul 07 22:44:40.052327 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:40.052312 2545 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 07 22:44:41.041734 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:41.041694 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-kzlgw/must-gather-vnpqx" event={"ID":"2e5babe5-f8ca-495d-a425-b11fcf4ffbdf","Type":"ContainerStarted","Data":"554cb0b7b2e8418521debe7452ea732b07332059d47ec6a786422507f4d70cc6"} Jul 07 22:44:42.050909 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:42.050396 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-kzlgw/must-gather-vnpqx" event={"ID":"2e5babe5-f8ca-495d-a425-b11fcf4ffbdf","Type":"ContainerStarted","Data":"b243e91d2fae56a334bceada2fa0256b1842b499610f932624c1dfe5087caf89"} Jul 07 22:44:42.050909 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:42.050457 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-kzlgw/must-gather-vnpqx" event={"ID":"2e5babe5-f8ca-495d-a425-b11fcf4ffbdf","Type":"ContainerStarted","Data":"48b26c8dc1035f9ce4d7fb93c34ef33456aa7b6b062c00ce7026811477ea2ea4"} Jul 07 22:44:42.066770 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:42.066697 2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-kzlgw/must-gather-vnpqx" podStartSLOduration=2.107754765 podStartE2EDuration="3.06667719s" totalImagesPullingTime="958.922425ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-07 22:44:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:44:42.065120823 +0000 UTC m=+2464.379343844" watchObservedRunningTime="2026-07-07 22:44:42.06667719 +0000 UTC m=+2464.380900211" Jul 07 22:44:42.524218 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:42.524164 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-nqfk8_01e846e0-a94f-446e-8ec7-fdeb51040759/global-pull-secret-syncer/0.log" Jul 07 22:44:42.681042 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:42.681011 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-ms5bj_35a9b908-7fda-4fb9-80b9-56b119f92de0/konnectivity-agent/0.log" Jul 07 22:44:42.700767 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:42.700705 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-131-55.ec2.internal_164cedd539cf1aa15ead9eedbab978df/haproxy/0.log" Jul 07 22:44:48.052740 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:48.052701 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6c9b657c9c-jqr7l_303bd640-3c9c-4a1f-8cd6-5d3fb366af14/authorino/0.log" Jul 07 22:44:48.104196 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:48.104167 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-5z2vw_4687ae14-1e22-4218-9fec-004e72063d8b/manager/0.log" Jul 07 22:44:48.128305 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:48.128275 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-m8nxm_6941be32-c4d0-4973-87ec-11caa5ee0fc9/kuadrant-console-plugin/0.log" Jul 07 22:44:48.442532 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:48.442380 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-sqptd_1e06e247-ff08-43e1-a884-79bb36006378/manager/0.log" Jul 07 22:44:50.244218 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:50.244188 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-j629n_60b9212f-09a8-4beb-a1f5-c242e7cd135c/kube-state-metrics/0.log" Jul 07 22:44:50.260552 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:50.260515 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-j629n_60b9212f-09a8-4beb-a1f5-c242e7cd135c/kube-rbac-proxy-main/0.log" Jul 07 22:44:50.280067 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:50.280039 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-j629n_60b9212f-09a8-4beb-a1f5-c242e7cd135c/kube-rbac-proxy-self/0.log" Jul 07 22:44:50.309057 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:50.309031 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-7c9bdddb8d-tlwzc_4ded1363-3a10-48da-9736-4c181ae38c2e/metrics-server/0.log" Jul 07 22:44:50.454537 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:50.454492 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-9hl2n_4ae9b323-7862-4003-a018-71b5e4d4705a/node-exporter/0.log" Jul 07 22:44:50.477421 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:50.477390 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-9hl2n_4ae9b323-7862-4003-a018-71b5e4d4705a/kube-rbac-proxy/0.log" Jul 07 22:44:50.498307 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:50.498274 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-9hl2n_4ae9b323-7862-4003-a018-71b5e4d4705a/init-textfile/0.log" Jul 07 22:44:50.608342 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:50.608308 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-nkc2m_1e0d5872-f2ba-4d37-b3ad-80f1d502b936/kube-rbac-proxy-main/0.log" Jul 07 22:44:50.628478 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:50.628450 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-nkc2m_1e0d5872-f2ba-4d37-b3ad-80f1d502b936/kube-rbac-proxy-self/0.log" Jul 07 22:44:50.648828 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:50.648780 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-nkc2m_1e0d5872-f2ba-4d37-b3ad-80f1d502b936/openshift-state-metrics/0.log" Jul 07 22:44:50.688592 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:50.688539 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_339d6ea1-7785-4db8-bf7c-abb7bab436d0/prometheus/0.log" Jul 07 22:44:50.709212 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:50.709130 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_339d6ea1-7785-4db8-bf7c-abb7bab436d0/config-reloader/0.log" Jul 07 22:44:50.729026 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:50.729002 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_339d6ea1-7785-4db8-bf7c-abb7bab436d0/thanos-sidecar/0.log" Jul 07 22:44:50.750640 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:50.750614 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_339d6ea1-7785-4db8-bf7c-abb7bab436d0/kube-rbac-proxy-web/0.log" Jul 07 22:44:50.770889 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:50.770860 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_339d6ea1-7785-4db8-bf7c-abb7bab436d0/kube-rbac-proxy/0.log" Jul 07 22:44:50.791102 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:50.791046 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_339d6ea1-7785-4db8-bf7c-abb7bab436d0/kube-rbac-proxy-thanos/0.log" Jul 07 22:44:50.811706 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:50.811669 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_339d6ea1-7785-4db8-bf7c-abb7bab436d0/init-config-reloader/0.log" Jul 07 22:44:50.841683 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:50.841655 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-tvxfb_7a0aec9a-a95c-4742-9c09-54c6f03ea2bb/prometheus-operator/0.log" Jul 07 22:44:50.860712 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:50.860684 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-tvxfb_7a0aec9a-a95c-4742-9c09-54c6f03ea2bb/kube-rbac-proxy/0.log" Jul 07 22:44:50.890823 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:50.890783 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-d559d66d5-vh2rt_d81a9215-297f-45fa-bc60-1e7993111c96/prometheus-operator-admission-webhook/0.log" Jul 07 22:44:50.915097 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:50.915055 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-78cf548768-7mq45_efbefc08-120c-46e7-a5f4-f5e95cadbb2c/telemeter-client/0.log" Jul 07 22:44:50.935271 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:50.935239 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-78cf548768-7mq45_efbefc08-120c-46e7-a5f4-f5e95cadbb2c/reload/0.log" Jul 07 22:44:50.954605 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:50.954576 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-78cf548768-7mq45_efbefc08-120c-46e7-a5f4-f5e95cadbb2c/kube-rbac-proxy/0.log" Jul 07 22:44:52.507006 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:52.506969 2545 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-kzlgw/perf-node-gather-daemonset-54gzw"] Jul 07 22:44:52.515208 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:52.515179 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-kzlgw/perf-node-gather-daemonset-54gzw" Jul 07 22:44:52.519073 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:52.519042 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-kzlgw/perf-node-gather-daemonset-54gzw"] Jul 07 22:44:52.650855 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:52.650816 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f3029423-6925-49d0-851b-74b622ae9c1a-sys\") pod \"perf-node-gather-daemonset-54gzw\" (UID: \"f3029423-6925-49d0-851b-74b622ae9c1a\") " pod="openshift-must-gather-kzlgw/perf-node-gather-daemonset-54gzw" Jul 07 22:44:52.651031 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:52.650881 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f3029423-6925-49d0-851b-74b622ae9c1a-proc\") pod \"perf-node-gather-daemonset-54gzw\" (UID: \"f3029423-6925-49d0-851b-74b622ae9c1a\") " pod="openshift-must-gather-kzlgw/perf-node-gather-daemonset-54gzw" Jul 07 22:44:52.651031 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:52.651004 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f3029423-6925-49d0-851b-74b622ae9c1a-podres\") pod \"perf-node-gather-daemonset-54gzw\" (UID: \"f3029423-6925-49d0-851b-74b622ae9c1a\") " pod="openshift-must-gather-kzlgw/perf-node-gather-daemonset-54gzw" Jul 07 22:44:52.651193 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:52.651121 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-97n2x\" (UniqueName: \"kubernetes.io/projected/f3029423-6925-49d0-851b-74b622ae9c1a-kube-api-access-97n2x\") pod \"perf-node-gather-daemonset-54gzw\" (UID: \"f3029423-6925-49d0-851b-74b622ae9c1a\") " pod="openshift-must-gather-kzlgw/perf-node-gather-daemonset-54gzw" Jul 07 22:44:52.651193 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:52.651172 2545 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f3029423-6925-49d0-851b-74b622ae9c1a-lib-modules\") pod \"perf-node-gather-daemonset-54gzw\" (UID: \"f3029423-6925-49d0-851b-74b622ae9c1a\") " pod="openshift-must-gather-kzlgw/perf-node-gather-daemonset-54gzw" Jul 07 22:44:52.752501 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:52.752458 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f3029423-6925-49d0-851b-74b622ae9c1a-podres\") pod \"perf-node-gather-daemonset-54gzw\" (UID: \"f3029423-6925-49d0-851b-74b622ae9c1a\") " pod="openshift-must-gather-kzlgw/perf-node-gather-daemonset-54gzw" Jul 07 22:44:52.752809 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:52.752755 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f3029423-6925-49d0-851b-74b622ae9c1a-podres\") pod \"perf-node-gather-daemonset-54gzw\" (UID: \"f3029423-6925-49d0-851b-74b622ae9c1a\") " pod="openshift-must-gather-kzlgw/perf-node-gather-daemonset-54gzw" Jul 07 22:44:52.752903 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:52.752828 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-97n2x\" (UniqueName: \"kubernetes.io/projected/f3029423-6925-49d0-851b-74b622ae9c1a-kube-api-access-97n2x\") pod \"perf-node-gather-daemonset-54gzw\" (UID: \"f3029423-6925-49d0-851b-74b622ae9c1a\") " pod="openshift-must-gather-kzlgw/perf-node-gather-daemonset-54gzw" Jul 07 22:44:52.752903 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:52.752889 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f3029423-6925-49d0-851b-74b622ae9c1a-lib-modules\") pod \"perf-node-gather-daemonset-54gzw\" (UID: \"f3029423-6925-49d0-851b-74b622ae9c1a\") " pod="openshift-must-gather-kzlgw/perf-node-gather-daemonset-54gzw" Jul 07 22:44:52.753014 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:52.752919 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f3029423-6925-49d0-851b-74b622ae9c1a-sys\") pod \"perf-node-gather-daemonset-54gzw\" (UID: \"f3029423-6925-49d0-851b-74b622ae9c1a\") " pod="openshift-must-gather-kzlgw/perf-node-gather-daemonset-54gzw" Jul 07 22:44:52.753014 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:52.752962 2545 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f3029423-6925-49d0-851b-74b622ae9c1a-proc\") pod \"perf-node-gather-daemonset-54gzw\" (UID: \"f3029423-6925-49d0-851b-74b622ae9c1a\") " pod="openshift-must-gather-kzlgw/perf-node-gather-daemonset-54gzw" Jul 07 22:44:52.753136 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:52.753037 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f3029423-6925-49d0-851b-74b622ae9c1a-lib-modules\") pod \"perf-node-gather-daemonset-54gzw\" (UID: \"f3029423-6925-49d0-851b-74b622ae9c1a\") " pod="openshift-must-gather-kzlgw/perf-node-gather-daemonset-54gzw" Jul 07 22:44:52.753136 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:52.753062 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f3029423-6925-49d0-851b-74b622ae9c1a-sys\") pod \"perf-node-gather-daemonset-54gzw\" (UID: \"f3029423-6925-49d0-851b-74b622ae9c1a\") " pod="openshift-must-gather-kzlgw/perf-node-gather-daemonset-54gzw" Jul 07 22:44:52.753260 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:52.753144 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f3029423-6925-49d0-851b-74b622ae9c1a-proc\") pod \"perf-node-gather-daemonset-54gzw\" (UID: \"f3029423-6925-49d0-851b-74b622ae9c1a\") " pod="openshift-must-gather-kzlgw/perf-node-gather-daemonset-54gzw" Jul 07 22:44:52.762281 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:52.762200 2545 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-97n2x\" (UniqueName: \"kubernetes.io/projected/f3029423-6925-49d0-851b-74b622ae9c1a-kube-api-access-97n2x\") pod \"perf-node-gather-daemonset-54gzw\" (UID: \"f3029423-6925-49d0-851b-74b622ae9c1a\") " pod="openshift-must-gather-kzlgw/perf-node-gather-daemonset-54gzw" Jul 07 22:44:52.838145 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:52.838115 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-rhg5r_71f36bfa-4b8a-4572-b522-57c16392aa0b/console-operator/1.log" Jul 07 22:44:52.846828 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:52.846802 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-rhg5r_71f36bfa-4b8a-4572-b522-57c16392aa0b/console-operator/2.log" Jul 07 22:44:52.849647 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:52.849619 2545 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-kzlgw/perf-node-gather-daemonset-54gzw" Jul 07 22:44:53.042004 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:53.041919 2545 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-kzlgw/perf-node-gather-daemonset-54gzw"] Jul 07 22:44:53.047288 ip-10-0-131-55 kubenswrapper[2545]: W0707 22:44:53.047256 2545 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podf3029423_6925_49d0_851b_74b622ae9c1a.slice/crio-dd6d45ead150d1e89fbad41e10207cd2a4409f1b484acf352800fafd203b4f8c WatchSource:0}: Error finding container dd6d45ead150d1e89fbad41e10207cd2a4409f1b484acf352800fafd203b4f8c: Status 404 returned error can't find the container with id dd6d45ead150d1e89fbad41e10207cd2a4409f1b484acf352800fafd203b4f8c Jul 07 22:44:53.139322 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:53.139279 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-kzlgw/perf-node-gather-daemonset-54gzw" event={"ID":"f3029423-6925-49d0-851b-74b622ae9c1a","Type":"ContainerStarted","Data":"dd6d45ead150d1e89fbad41e10207cd2a4409f1b484acf352800fafd203b4f8c"} Jul 07 22:44:53.436648 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:53.436567 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-685bfb5f64-q7rgl_f9c45f5f-a6f4-406c-a8f8-3d474a0dce97/download-server/0.log" Jul 07 22:44:54.030884 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:54.030854 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-5d456fdbcd-82vxn_84813585-7321-4216-bab4-9258b9ecff6a/volume-data-source-validator/0.log" Jul 07 22:44:54.146689 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:54.146646 2545 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-kzlgw/perf-node-gather-daemonset-54gzw" event={"ID":"f3029423-6925-49d0-851b-74b622ae9c1a","Type":"ContainerStarted","Data":"810fa2996ae8f6e789f695fcdc4af913f37a551a8f41063160f2230f86a93992"} Jul 07 22:44:54.146866 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:54.146833 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-kzlgw/perf-node-gather-daemonset-54gzw" Jul 07 22:44:54.161558 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:54.161496 2545 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-kzlgw/perf-node-gather-daemonset-54gzw" podStartSLOduration=2.161475067 podStartE2EDuration="2.161475067s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-07 22:44:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-07 22:44:54.160493549 +0000 UTC m=+2476.474716567" watchObservedRunningTime="2026-07-07 22:44:54.161475067 +0000 UTC m=+2476.475698088" Jul 07 22:44:55.051587 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:55.051557 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-bqv92_c80e0cce-1240-4a8f-873b-abd71e85b59a/dns/0.log" Jul 07 22:44:55.071416 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:55.071387 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-bqv92_c80e0cce-1240-4a8f-873b-abd71e85b59a/kube-rbac-proxy/0.log" Jul 07 22:44:55.155496 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:55.155463 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-9vvqt_7ac39c08-2ce6-4578-8b01-b68587adf750/dns-node-resolver/0.log" Jul 07 22:44:55.729632 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:55.729600 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-cc72r_381be182-aba1-4465-a899-8f217bbeacd2/node-ca/0.log" Jul 07 22:44:56.859687 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:56.859655 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6b4fccd749-28d8d_980381c5-f1a1-41ea-ba0f-0a582f6c2577/kube-auth-proxy/0.log" Jul 07 22:44:57.110238 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:57.110140 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-67496c4dcc-nnvc2_84447fdf-4f2d-45fd-9e94-acce197ae75b/router/0.log" Jul 07 22:44:57.831532 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:57.831498 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-wrlkn_230f7860-610d-430a-987b-6dfaa258d4e7/serve-healthcheck-canary/0.log" Jul 07 22:44:58.376493 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:58.376467 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-9rmdm_add27e45-747d-4e8d-9c57-4ca7f6268343/kube-rbac-proxy/0.log" Jul 07 22:44:58.395872 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:58.395849 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-9rmdm_add27e45-747d-4e8d-9c57-4ca7f6268343/exporter/0.log" Jul 07 22:44:58.415784 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:44:58.415756 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-9rmdm_add27e45-747d-4e8d-9c57-4ca7f6268343/extractor/0.log" Jul 07 22:45:00.163222 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:45:00.163197 2545 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-kzlgw/perf-node-gather-daemonset-54gzw" Jul 07 22:45:01.409320 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:45:01.409227 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-597db99f65-jjzbb_559369c5-4d6c-427a-90c7-2c49e47827ac/manager/0.log" Jul 07 22:45:01.431504 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:45:01.431476 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-597db99f65-jjzbb_559369c5-4d6c-427a-90c7-2c49e47827ac/copy-manifests/0.log" Jul 07 22:45:02.927675 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:45:02.927647 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-654d77bd8f-dnshd_8e1d7f55-e188-4c00-84c2-024a2d313383/manager/0.log" Jul 07 22:45:03.015726 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:45:03.015698 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6b4fccd749-28d8d_980381c5-f1a1-41ea-ba0f-0a582f6c2577/kube-auth-proxy/0.log" Jul 07 22:45:03.222372 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:45:03.222347 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-67496c4dcc-nnvc2_84447fdf-4f2d-45fd-9e94-acce197ae75b/router/0.log" Jul 07 22:45:04.648470 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:45:04.648367 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6b4fccd749-28d8d_980381c5-f1a1-41ea-ba0f-0a582f6c2577/kube-auth-proxy/0.log" Jul 07 22:45:04.857395 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:45:04.857361 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-67496c4dcc-nnvc2_84447fdf-4f2d-45fd-9e94-acce197ae75b/router/0.log" Jul 07 22:45:08.228060 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:45:08.228024 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-846bf668f7-wt5nz_01d309f5-5c78-4908-a057-7dcb433b7f3b/kube-storage-version-migrator-operator/1.log" Jul 07 22:45:08.228960 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:45:08.228937 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-846bf668f7-wt5nz_01d309f5-5c78-4908-a057-7dcb433b7f3b/kube-storage-version-migrator-operator/0.log" Jul 07 22:45:09.480468 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:45:09.480421 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ztkm2_00baab1c-b8e0-4b5e-bd42-647ba72c5aca/kube-multus-additional-cni-plugins/0.log" Jul 07 22:45:09.500653 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:45:09.500623 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ztkm2_00baab1c-b8e0-4b5e-bd42-647ba72c5aca/egress-router-binary-copy/0.log" Jul 07 22:45:09.522893 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:45:09.522860 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ztkm2_00baab1c-b8e0-4b5e-bd42-647ba72c5aca/cni-plugins/0.log" Jul 07 22:45:09.542817 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:45:09.542786 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ztkm2_00baab1c-b8e0-4b5e-bd42-647ba72c5aca/bond-cni-plugin/0.log" Jul 07 22:45:09.563763 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:45:09.563727 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ztkm2_00baab1c-b8e0-4b5e-bd42-647ba72c5aca/routeoverride-cni/0.log" Jul 07 22:45:09.583869 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:45:09.583839 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ztkm2_00baab1c-b8e0-4b5e-bd42-647ba72c5aca/whereabouts-cni-bincopy/0.log" Jul 07 22:45:09.603824 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:45:09.603798 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ztkm2_00baab1c-b8e0-4b5e-bd42-647ba72c5aca/whereabouts-cni/0.log" Jul 07 22:45:09.669750 ip-10-0-131-55 kubenswrapper[2545]: I0707 22:45:09.669724 2545 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-fg88n_f7c107a3-d0f6-479d-b548-c4c59b612631/kube-multus/0.log"