Jul 23 21:49:48.759866 ip-10-0-136-157 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 23 21:49:48.759878 ip-10-0-136-157 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 23 21:49:48.759885 ip-10-0-136-157 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 23 21:49:48.760103 ip-10-0-136-157 systemd[1]: Failed to start Kubernetes Kubelet. Jul 23 21:49:59.009318 ip-10-0-136-157 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 23 21:49:59.009336 ip-10-0-136-157 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot c80dce173c23471492abd392c4b94b97 -- Jul 23 21:51:37.651837 ip-10-0-136-157 systemd[1]: Starting Kubernetes Kubelet... Jul 23 21:51:38.011090 ip-10-0-136-157 kubenswrapper[2578]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 23 21:51:38.011090 ip-10-0-136-157 kubenswrapper[2578]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 23 21:51:38.011090 ip-10-0-136-157 kubenswrapper[2578]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 23 21:51:38.011090 ip-10-0-136-157 kubenswrapper[2578]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 23 21:51:38.011090 ip-10-0-136-157 kubenswrapper[2578]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 23 21:51:38.011936 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.011148 2578 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 23 21:51:38.015843 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.015821 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 23 21:51:38.015843 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.015839 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 23 21:51:38.015843 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.015844 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 23 21:51:38.015843 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.015848 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 23 21:51:38.015843 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.015852 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 23 21:51:38.016133 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.015856 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 23 21:51:38.016133 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.015861 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 23 21:51:38.016133 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.015865 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 23 21:51:38.016133 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.015868 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 23 21:51:38.016133 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.015871 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 23 21:51:38.016133 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.015874 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 23 21:51:38.016133 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.015877 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 23 21:51:38.016133 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.015881 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 23 21:51:38.016133 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.015885 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 23 21:51:38.016133 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.015888 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 23 21:51:38.016133 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.015892 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 23 21:51:38.016133 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.015896 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 23 21:51:38.016133 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.015899 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 23 21:51:38.016133 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.015903 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 23 21:51:38.016133 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.015908 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 23 21:51:38.016133 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.015912 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 23 21:51:38.016133 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.015916 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 23 21:51:38.016133 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.015919 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 23 21:51:38.016133 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.015924 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 23 21:51:38.016133 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.015927 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 23 21:51:38.016951 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.015931 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 23 21:51:38.016951 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.015934 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 23 21:51:38.016951 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.015938 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 23 21:51:38.016951 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.015942 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 23 21:51:38.016951 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.015946 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 23 21:51:38.016951 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.015950 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 23 21:51:38.016951 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.015953 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 23 21:51:38.016951 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.015958 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 23 21:51:38.016951 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.015961 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 23 21:51:38.016951 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.015965 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 23 21:51:38.016951 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.015970 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 23 21:51:38.016951 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.015973 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 23 21:51:38.016951 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.015978 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 23 21:51:38.016951 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.015983 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 23 21:51:38.016951 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.015987 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 23 21:51:38.016951 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.015991 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 23 21:51:38.016951 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.015995 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 23 21:51:38.016951 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.016000 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 23 21:51:38.016951 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.016003 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 23 21:51:38.017674 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.016010 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 23 21:51:38.017674 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.016016 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 23 21:51:38.017674 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.016020 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 23 21:51:38.017674 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.016026 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 23 21:51:38.017674 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.016031 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 23 21:51:38.017674 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.016035 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 23 21:51:38.017674 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.016039 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 23 21:51:38.017674 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.016043 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 23 21:51:38.017674 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.016048 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 23 21:51:38.017674 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.016052 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 23 21:51:38.017674 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.016055 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 23 21:51:38.017674 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.016059 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 23 21:51:38.017674 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.016063 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 23 21:51:38.017674 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.016067 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 23 21:51:38.017674 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.016071 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 23 21:51:38.017674 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.016075 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 23 21:51:38.017674 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.016079 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 23 21:51:38.017674 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.016083 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 23 21:51:38.017674 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.016087 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 23 21:51:38.018362 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.016091 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 23 21:51:38.018362 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.016095 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 23 21:51:38.018362 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.016100 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 23 21:51:38.018362 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.016104 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 23 21:51:38.018362 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.016108 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 23 21:51:38.018362 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.016112 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 23 21:51:38.018362 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.016117 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 23 21:51:38.018362 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.016122 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 23 21:51:38.018362 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.016126 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 23 21:51:38.018362 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.016130 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 23 21:51:38.018362 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.016134 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 23 21:51:38.018362 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.016137 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 23 21:51:38.018362 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.016141 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 23 21:51:38.018362 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.016149 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 23 21:51:38.018362 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.016154 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 23 21:51:38.018362 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.016159 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 23 21:51:38.018362 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.016163 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 23 21:51:38.018362 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.016167 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 23 21:51:38.018362 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.016172 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 23 21:51:38.018362 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.016176 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 23 21:51:38.018930 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.016180 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 23 21:51:38.018930 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.016184 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 23 21:51:38.018930 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.016189 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 23 21:51:38.018930 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.016192 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 23 21:51:38.018930 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.016196 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 23 21:51:38.018930 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.016200 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 23 21:51:38.018930 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.016204 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 23 21:51:38.018930 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.016208 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 23 21:51:38.018930 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.016212 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 23 21:51:38.018930 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.016217 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 23 21:51:38.018930 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.016221 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 23 21:51:38.018930 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.016227 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 23 21:51:38.018930 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.016233 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 23 21:51:38.018930 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.016237 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 23 21:51:38.018930 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.016242 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 23 21:51:38.018930 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.016247 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 23 21:51:38.018930 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.018849 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 23 21:51:38.018930 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.018862 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 23 21:51:38.018930 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.018865 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 23 21:51:38.019403 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.018868 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 23 21:51:38.019403 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.018871 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 23 21:51:38.019403 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.018874 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 23 21:51:38.019403 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.018877 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 23 21:51:38.019403 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.018880 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 23 21:51:38.019403 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.018882 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 23 21:51:38.019403 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.018885 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 23 21:51:38.019403 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.018887 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 23 21:51:38.019403 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.018890 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 23 21:51:38.019403 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.018892 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 23 21:51:38.019403 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.018895 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 23 21:51:38.019403 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.018899 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 23 21:51:38.019403 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.018902 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 23 21:51:38.019403 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.018905 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 23 21:51:38.019403 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.018908 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 23 21:51:38.019403 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.018910 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 23 21:51:38.019403 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.018913 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 23 21:51:38.019403 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.018915 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 23 21:51:38.019403 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.018918 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 23 21:51:38.019403 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.018920 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 23 21:51:38.019897 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.018923 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 23 21:51:38.019897 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.018925 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 23 21:51:38.019897 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.018928 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 23 21:51:38.019897 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.018931 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 23 21:51:38.019897 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.018933 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 23 21:51:38.019897 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.018936 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 23 21:51:38.019897 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.018938 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 23 21:51:38.019897 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.018941 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 23 21:51:38.019897 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.018943 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 23 21:51:38.019897 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.018947 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 23 21:51:38.019897 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.018950 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 23 21:51:38.019897 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.018952 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 23 21:51:38.019897 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.018975 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 23 21:51:38.019897 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.018980 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 23 21:51:38.019897 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.018983 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 23 21:51:38.019897 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.018985 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 23 21:51:38.019897 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.018988 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 23 21:51:38.019897 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.018992 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 23 21:51:38.019897 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.018996 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 23 21:51:38.019897 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.018999 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 23 21:51:38.020363 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019002 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 23 21:51:38.020363 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019004 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 23 21:51:38.020363 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019006 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 23 21:51:38.020363 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019009 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 23 21:51:38.020363 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019012 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 23 21:51:38.020363 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019014 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 23 21:51:38.020363 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019016 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 23 21:51:38.020363 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019019 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 23 21:51:38.020363 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019021 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 23 21:51:38.020363 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019023 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 23 21:51:38.020363 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019026 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 23 21:51:38.020363 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019028 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 23 21:51:38.020363 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019031 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 23 21:51:38.020363 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019033 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 23 21:51:38.020363 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019036 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 23 21:51:38.020363 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019038 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 23 21:51:38.020363 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019041 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 23 21:51:38.020363 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019043 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 23 21:51:38.020363 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019046 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 23 21:51:38.020363 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019048 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 23 21:51:38.020856 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019051 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 23 21:51:38.020856 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019053 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 23 21:51:38.020856 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019056 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 23 21:51:38.020856 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019058 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 23 21:51:38.020856 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019061 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 23 21:51:38.020856 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019064 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 23 21:51:38.020856 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019069 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 23 21:51:38.020856 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019073 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 23 21:51:38.020856 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019076 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 23 21:51:38.020856 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019079 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 23 21:51:38.020856 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019082 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 23 21:51:38.020856 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019085 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 23 21:51:38.020856 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019087 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 23 21:51:38.020856 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019090 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 23 21:51:38.020856 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019092 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 23 21:51:38.020856 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019095 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 23 21:51:38.020856 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019098 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 23 21:51:38.020856 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019100 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 23 21:51:38.020856 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019102 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 23 21:51:38.021296 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019105 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 23 21:51:38.021296 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019109 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 23 21:51:38.021296 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019113 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 23 21:51:38.021296 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019116 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 23 21:51:38.021296 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019118 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 23 21:51:38.021296 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019121 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 23 21:51:38.021296 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019123 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 23 21:51:38.021296 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019126 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 23 21:51:38.021296 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019130 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 23 21:51:38.021296 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019134 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 23 21:51:38.021296 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019137 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 23 21:51:38.021296 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019141 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 23 21:51:38.021296 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019144 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 23 21:51:38.021296 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019146 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 23 21:51:38.021296 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019149 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 23 21:51:38.021296 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019152 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 23 21:51:38.021296 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019155 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 23 21:51:38.021296 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019528 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 23 21:51:38.021296 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019532 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 23 21:51:38.021296 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019550 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 23 21:51:38.021783 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019553 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 23 21:51:38.021783 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019555 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 23 21:51:38.021783 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019559 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 23 21:51:38.021783 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019561 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 23 21:51:38.021783 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019564 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 23 21:51:38.021783 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019566 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 23 21:51:38.021783 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019568 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 23 21:51:38.021783 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019571 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 23 21:51:38.021783 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019575 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 23 21:51:38.021783 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019578 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 23 21:51:38.021783 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019581 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 23 21:51:38.021783 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019583 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 23 21:51:38.021783 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019586 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 23 21:51:38.021783 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019589 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 23 21:51:38.021783 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019592 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 23 21:51:38.021783 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019595 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 23 21:51:38.021783 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019597 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 23 21:51:38.021783 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019600 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 23 21:51:38.021783 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019602 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 23 21:51:38.021783 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019604 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 23 21:51:38.022252 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019607 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 23 21:51:38.022252 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019610 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 23 21:51:38.022252 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019613 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 23 21:51:38.022252 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019615 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 23 21:51:38.022252 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019617 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 23 21:51:38.022252 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019620 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 23 21:51:38.022252 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019622 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 23 21:51:38.022252 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019625 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 23 21:51:38.022252 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019627 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 23 21:51:38.022252 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019630 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 23 21:51:38.022252 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019632 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 23 21:51:38.022252 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019635 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 23 21:51:38.022252 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019637 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 23 21:51:38.022252 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019640 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 23 21:51:38.022252 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019642 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 23 21:51:38.022252 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019645 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 23 21:51:38.022252 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019648 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 23 21:51:38.022252 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019650 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 23 21:51:38.022252 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019652 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 23 21:51:38.022713 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019655 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 23 21:51:38.022713 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019657 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 23 21:51:38.022713 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019659 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 23 21:51:38.022713 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019662 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 23 21:51:38.022713 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019664 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 23 21:51:38.022713 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019666 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 23 21:51:38.022713 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019669 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 23 21:51:38.022713 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019672 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 23 21:51:38.022713 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019674 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 23 21:51:38.022713 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019677 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 23 21:51:38.022713 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019679 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 23 21:51:38.022713 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019682 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 23 21:51:38.022713 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019684 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 23 21:51:38.022713 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019686 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 23 21:51:38.022713 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019689 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 23 21:51:38.022713 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019691 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 23 21:51:38.022713 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019694 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 23 21:51:38.022713 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019696 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 23 21:51:38.022713 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019698 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 23 21:51:38.022713 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019701 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 23 21:51:38.023176 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019703 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 23 21:51:38.023176 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019705 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 23 21:51:38.023176 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019708 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 23 21:51:38.023176 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019710 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 23 21:51:38.023176 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019713 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 23 21:51:38.023176 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019715 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 23 21:51:38.023176 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019718 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 23 21:51:38.023176 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019720 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 23 21:51:38.023176 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019722 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 23 21:51:38.023176 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019726 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 23 21:51:38.023176 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019729 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 23 21:51:38.023176 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019732 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 23 21:51:38.023176 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019734 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 23 21:51:38.023176 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019736 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 23 21:51:38.023176 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019739 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 23 21:51:38.023176 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019741 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 23 21:51:38.023176 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019744 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 23 21:51:38.023176 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019747 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 23 21:51:38.023176 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019749 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 23 21:51:38.023647 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019752 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 23 21:51:38.023647 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019755 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 23 21:51:38.023647 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019758 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 23 21:51:38.023647 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019760 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 23 21:51:38.023647 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019763 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 23 21:51:38.023647 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019765 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 23 21:51:38.023647 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019767 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 23 21:51:38.023647 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019770 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 23 21:51:38.023647 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019772 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 23 21:51:38.023647 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019774 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 23 21:51:38.023647 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019777 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 23 21:51:38.023647 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019779 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 23 21:51:38.023647 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019783 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 23 21:51:38.023647 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019785 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 23 21:51:38.023647 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019788 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 23 21:51:38.023647 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019790 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 23 21:51:38.023647 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019793 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 23 21:51:38.023647 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.019795 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 23 21:51:38.023647 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.019864 2578 flags.go:64] FLAG: --address="0.0.0.0" Jul 23 21:51:38.023647 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.019871 2578 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 23 21:51:38.024141 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.019878 2578 flags.go:64] FLAG: --anonymous-auth="true" Jul 23 21:51:38.024141 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.019882 2578 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 23 21:51:38.024141 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.019886 2578 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 23 21:51:38.024141 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.019889 2578 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 23 21:51:38.024141 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.019894 2578 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 23 21:51:38.024141 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.019898 2578 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 23 21:51:38.024141 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.019901 2578 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 23 21:51:38.024141 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.019904 2578 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 23 21:51:38.024141 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.019908 2578 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 23 21:51:38.024141 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.019911 2578 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 23 21:51:38.024141 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.019914 2578 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 23 21:51:38.024141 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.019917 2578 flags.go:64] FLAG: --cgroup-root="" Jul 23 21:51:38.024141 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.019922 2578 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 23 21:51:38.024141 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.019925 2578 flags.go:64] FLAG: --client-ca-file="" Jul 23 21:51:38.024141 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.019929 2578 flags.go:64] FLAG: --cloud-provider="external" Jul 23 21:51:38.024141 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.019932 2578 flags.go:64] FLAG: --cluster-dns="[]" Jul 23 21:51:38.024141 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.019936 2578 flags.go:64] FLAG: --cluster-domain="" Jul 23 21:51:38.024141 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.019939 2578 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 23 21:51:38.024141 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.019942 2578 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 23 21:51:38.024141 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.019945 2578 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 23 21:51:38.024141 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.019948 2578 flags.go:64] FLAG: --container-log-max-files="5" Jul 23 21:51:38.024141 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.019951 2578 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 23 21:51:38.024141 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.019954 2578 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 23 21:51:38.024829 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.019957 2578 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 23 21:51:38.024829 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.019960 2578 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 23 21:51:38.024829 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.019963 2578 flags.go:64] FLAG: --contention-profiling="false" Jul 23 21:51:38.024829 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.019966 2578 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 23 21:51:38.024829 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.019968 2578 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 23 21:51:38.024829 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.019971 2578 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 23 21:51:38.024829 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.019974 2578 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 23 21:51:38.024829 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.019978 2578 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 23 21:51:38.024829 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.019981 2578 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 23 21:51:38.024829 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.019984 2578 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 23 21:51:38.024829 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.019986 2578 flags.go:64] FLAG: --enable-load-reader="false" Jul 23 21:51:38.024829 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.019989 2578 flags.go:64] FLAG: --enable-server="true" Jul 23 21:51:38.024829 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.019991 2578 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 23 21:51:38.024829 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.019996 2578 flags.go:64] FLAG: --event-burst="100" Jul 23 21:51:38.024829 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.019999 2578 flags.go:64] FLAG: --event-qps="50" Jul 23 21:51:38.024829 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.020002 2578 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 23 21:51:38.024829 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.020005 2578 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 23 21:51:38.024829 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.020007 2578 flags.go:64] FLAG: --eviction-hard="" Jul 23 21:51:38.024829 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.020011 2578 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 23 21:51:38.024829 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.020014 2578 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 23 21:51:38.024829 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.020016 2578 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 23 21:51:38.024829 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.020020 2578 flags.go:64] FLAG: --eviction-soft="" Jul 23 21:51:38.024829 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.020023 2578 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 23 21:51:38.024829 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.020029 2578 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 23 21:51:38.024829 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.020033 2578 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 23 21:51:38.025424 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.020035 2578 flags.go:64] FLAG: --experimental-mounter-path="" Jul 23 21:51:38.025424 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.020038 2578 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 23 21:51:38.025424 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.020041 2578 flags.go:64] FLAG: --fail-swap-on="true" Jul 23 21:51:38.025424 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.020043 2578 flags.go:64] FLAG: --feature-gates="" Jul 23 21:51:38.025424 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.020052 2578 flags.go:64] FLAG: --file-check-frequency="20s" Jul 23 21:51:38.025424 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.020054 2578 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 23 21:51:38.025424 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.020057 2578 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 23 21:51:38.025424 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.020060 2578 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 23 21:51:38.025424 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.020063 2578 flags.go:64] FLAG: --healthz-port="10248" Jul 23 21:51:38.025424 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.020065 2578 flags.go:64] FLAG: --help="false" Jul 23 21:51:38.025424 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.020068 2578 flags.go:64] FLAG: --hostname-override="ip-10-0-136-157.ec2.internal" Jul 23 21:51:38.025424 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.020071 2578 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 23 21:51:38.025424 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.020074 2578 flags.go:64] FLAG: --http-check-frequency="20s" Jul 23 21:51:38.025424 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.020077 2578 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 23 21:51:38.025424 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.020080 2578 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 23 21:51:38.025424 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.020083 2578 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 23 21:51:38.025424 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.020085 2578 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 23 21:51:38.025424 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.020088 2578 flags.go:64] FLAG: --image-service-endpoint="" Jul 23 21:51:38.025424 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.020091 2578 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 23 21:51:38.025424 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.020093 2578 flags.go:64] FLAG: --kube-api-burst="100" Jul 23 21:51:38.025424 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.020096 2578 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 23 21:51:38.025424 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.020098 2578 flags.go:64] FLAG: --kube-api-qps="50" Jul 23 21:51:38.025424 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.020101 2578 flags.go:64] FLAG: --kube-reserved="" Jul 23 21:51:38.025424 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.020103 2578 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 23 21:51:38.026007 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.020106 2578 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 23 21:51:38.026007 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.020109 2578 flags.go:64] FLAG: --kubelet-cgroups="" Jul 23 21:51:38.026007 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.020111 2578 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 23 21:51:38.026007 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.020114 2578 flags.go:64] FLAG: --lock-file="" Jul 23 21:51:38.026007 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.020117 2578 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 23 21:51:38.026007 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.020120 2578 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 23 21:51:38.026007 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.020124 2578 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 23 21:51:38.026007 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.020129 2578 flags.go:64] FLAG: --log-json-split-stream="false" Jul 23 21:51:38.026007 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.020132 2578 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 23 21:51:38.026007 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.020136 2578 flags.go:64] FLAG: --log-text-split-stream="false" Jul 23 21:51:38.026007 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.020138 2578 flags.go:64] FLAG: --logging-format="text" Jul 23 21:51:38.026007 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.020141 2578 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 23 21:51:38.026007 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.020144 2578 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 23 21:51:38.026007 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.020147 2578 flags.go:64] FLAG: --manifest-url="" Jul 23 21:51:38.026007 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.020149 2578 flags.go:64] FLAG: --manifest-url-header="" Jul 23 21:51:38.026007 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.020153 2578 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 23 21:51:38.026007 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.020156 2578 flags.go:64] FLAG: --max-open-files="1000000" Jul 23 21:51:38.026007 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.020159 2578 flags.go:64] FLAG: --max-pods="110" Jul 23 21:51:38.026007 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.020162 2578 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 23 21:51:38.026007 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.020165 2578 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 23 21:51:38.026007 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.020167 2578 flags.go:64] FLAG: --memory-manager-policy="None" Jul 23 21:51:38.026007 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.020170 2578 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 23 21:51:38.026007 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.020173 2578 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 23 21:51:38.026007 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.020175 2578 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 23 21:51:38.026007 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.020178 2578 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 23 21:51:38.026633 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.020185 2578 flags.go:64] FLAG: --node-status-max-images="50" Jul 23 21:51:38.026633 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.020188 2578 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 23 21:51:38.026633 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.020191 2578 flags.go:64] FLAG: --oom-score-adj="-999" Jul 23 21:51:38.026633 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.020194 2578 flags.go:64] FLAG: --pod-cidr="" Jul 23 21:51:38.026633 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.020196 2578 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0" Jul 23 21:51:38.026633 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.020201 2578 flags.go:64] FLAG: --pod-manifest-path="" Jul 23 21:51:38.026633 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.020204 2578 flags.go:64] FLAG: --pod-max-pids="-1" Jul 23 21:51:38.026633 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.020207 2578 flags.go:64] FLAG: --pods-per-core="0" Jul 23 21:51:38.026633 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.020209 2578 flags.go:64] FLAG: --port="10250" Jul 23 21:51:38.026633 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.020212 2578 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 23 21:51:38.026633 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.020215 2578 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-06fb4cb4b811337aa" Jul 23 21:51:38.026633 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.020219 2578 flags.go:64] FLAG: --qos-reserved="" Jul 23 21:51:38.026633 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.020222 2578 flags.go:64] FLAG: --read-only-port="10255" Jul 23 21:51:38.026633 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.020224 2578 flags.go:64] FLAG: --register-node="true" Jul 23 21:51:38.026633 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.020230 2578 flags.go:64] FLAG: --register-with-taints="" Jul 23 21:51:38.026633 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.020234 2578 flags.go:64] FLAG: --registry-burst="10" Jul 23 21:51:38.026633 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.020237 2578 flags.go:64] FLAG: --registry-qps="5" Jul 23 21:51:38.026633 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.020239 2578 flags.go:64] FLAG: --reserved-cpus="" Jul 23 21:51:38.026633 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.020242 2578 flags.go:64] FLAG: --reserved-memory="" Jul 23 21:51:38.026633 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.020246 2578 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 23 21:51:38.026633 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.020249 2578 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 23 21:51:38.026633 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.020252 2578 flags.go:64] FLAG: --rotate-certificates="false" Jul 23 21:51:38.026633 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.020255 2578 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 23 21:51:38.026633 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.020257 2578 flags.go:64] FLAG: --runonce="false" Jul 23 21:51:38.026633 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.020260 2578 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 23 21:51:38.027280 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.020263 2578 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 23 21:51:38.027280 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.020265 2578 flags.go:64] FLAG: --seccomp-default="false" Jul 23 21:51:38.027280 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.020269 2578 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 23 21:51:38.027280 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.020271 2578 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 23 21:51:38.027280 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.020274 2578 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 23 21:51:38.027280 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.020276 2578 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 23 21:51:38.027280 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.020279 2578 flags.go:64] FLAG: --storage-driver-password="root" Jul 23 21:51:38.027280 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.020282 2578 flags.go:64] FLAG: --storage-driver-secure="false" Jul 23 21:51:38.027280 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.020285 2578 flags.go:64] FLAG: --storage-driver-table="stats" Jul 23 21:51:38.027280 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.020287 2578 flags.go:64] FLAG: --storage-driver-user="root" Jul 23 21:51:38.027280 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.020290 2578 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 23 21:51:38.027280 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.020293 2578 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 23 21:51:38.027280 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.020295 2578 flags.go:64] FLAG: --system-cgroups="" Jul 23 21:51:38.027280 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.020298 2578 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 23 21:51:38.027280 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.020303 2578 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 23 21:51:38.027280 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.020305 2578 flags.go:64] FLAG: --tls-cert-file="" Jul 23 21:51:38.027280 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.020308 2578 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 23 21:51:38.027280 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.020311 2578 flags.go:64] FLAG: --tls-min-version="" Jul 23 21:51:38.027280 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.020315 2578 flags.go:64] FLAG: --tls-private-key-file="" Jul 23 21:51:38.027280 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.020317 2578 flags.go:64] FLAG: --topology-manager-policy="none" Jul 23 21:51:38.027280 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.020320 2578 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 23 21:51:38.027280 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.020324 2578 flags.go:64] FLAG: --topology-manager-scope="container" Jul 23 21:51:38.027280 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.020326 2578 flags.go:64] FLAG: --v="2" Jul 23 21:51:38.027280 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.020330 2578 flags.go:64] FLAG: --version="false" Jul 23 21:51:38.027280 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.020334 2578 flags.go:64] FLAG: --vmodule="" Jul 23 21:51:38.027899 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.020338 2578 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 23 21:51:38.027899 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.020341 2578 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 23 21:51:38.027899 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.020466 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 23 21:51:38.027899 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.020469 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 23 21:51:38.027899 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.020472 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 23 21:51:38.027899 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.020475 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 23 21:51:38.027899 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.020478 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 23 21:51:38.027899 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.020481 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 23 21:51:38.027899 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.020484 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 23 21:51:38.027899 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.020486 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 23 21:51:38.027899 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.020489 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 23 21:51:38.027899 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.020492 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 23 21:51:38.027899 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.020495 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 23 21:51:38.027899 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.020498 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 23 21:51:38.027899 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.020500 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 23 21:51:38.027899 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.020504 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 23 21:51:38.027899 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.020507 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 23 21:51:38.027899 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.020510 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 23 21:51:38.027899 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.020513 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 23 21:51:38.027899 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.020515 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 23 21:51:38.028358 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.020517 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 23 21:51:38.028358 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.020519 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 23 21:51:38.028358 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.020522 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 23 21:51:38.028358 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.020524 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 23 21:51:38.028358 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.020527 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 23 21:51:38.028358 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.020531 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 23 21:51:38.028358 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.020533 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 23 21:51:38.028358 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.020551 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 23 21:51:38.028358 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.020554 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 23 21:51:38.028358 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.020557 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 23 21:51:38.028358 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.020559 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 23 21:51:38.028358 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.020562 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 23 21:51:38.028358 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.020565 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 23 21:51:38.028358 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.020567 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 23 21:51:38.028358 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.020570 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 23 21:51:38.028358 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.020573 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 23 21:51:38.028358 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.020575 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 23 21:51:38.028358 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.020577 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 23 21:51:38.028358 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.020580 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 23 21:51:38.028358 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.020582 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 23 21:51:38.028844 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.020585 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 23 21:51:38.028844 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.020587 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 23 21:51:38.028844 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.020590 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 23 21:51:38.028844 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.020592 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 23 21:51:38.028844 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.020595 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 23 21:51:38.028844 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.020597 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 23 21:51:38.028844 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.020599 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 23 21:51:38.028844 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.020602 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 23 21:51:38.028844 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.020606 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 23 21:51:38.028844 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.020611 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 23 21:51:38.028844 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.020614 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 23 21:51:38.028844 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.020617 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 23 21:51:38.028844 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.020619 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 23 21:51:38.028844 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.020622 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 23 21:51:38.028844 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.020624 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 23 21:51:38.028844 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.020627 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 23 21:51:38.028844 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.020629 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 23 21:51:38.028844 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.020632 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 23 21:51:38.028844 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.020635 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 23 21:51:38.029292 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.020637 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 23 21:51:38.029292 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.020640 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 23 21:51:38.029292 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.020642 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 23 21:51:38.029292 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.020645 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 23 21:51:38.029292 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.020647 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 23 21:51:38.029292 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.020650 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 23 21:51:38.029292 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.020653 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 23 21:51:38.029292 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.020655 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 23 21:51:38.029292 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.020657 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 23 21:51:38.029292 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.020660 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 23 21:51:38.029292 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.020662 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 23 21:51:38.029292 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.020664 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 23 21:51:38.029292 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.020667 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 23 21:51:38.029292 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.020669 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 23 21:51:38.029292 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.020671 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 23 21:51:38.029292 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.020674 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 23 21:51:38.029292 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.020676 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 23 21:51:38.029292 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.020678 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 23 21:51:38.029292 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.020681 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 23 21:51:38.029292 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.020683 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 23 21:51:38.029781 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.020685 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 23 21:51:38.029781 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.020688 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 23 21:51:38.029781 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.020690 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 23 21:51:38.029781 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.020692 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 23 21:51:38.029781 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.020695 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 23 21:51:38.029781 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.020697 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 23 21:51:38.029781 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.020700 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 23 21:51:38.029781 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.020702 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 23 21:51:38.029781 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.020704 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 23 21:51:38.029781 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.020707 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 23 21:51:38.029781 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.020711 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 23 21:51:38.029781 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.020713 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 23 21:51:38.029781 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.020715 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 23 21:51:38.029781 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.020718 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 23 21:51:38.029781 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.020720 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 23 21:51:38.029781 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.020723 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 23 21:51:38.029781 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.020725 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 23 21:51:38.029781 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.020728 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 23 21:51:38.029781 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.020730 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 23 21:51:38.030232 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.020733 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 23 21:51:38.030232 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.020736 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 23 21:51:38.030232 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.020738 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 23 21:51:38.030232 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.021289 2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 23 21:51:38.030232 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.028060 2578 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 23 21:51:38.030232 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.028076 2578 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 23 21:51:38.030232 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.028090 2578 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 23 21:51:38.030232 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.028096 2578 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 23 21:51:38.030232 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028152 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 23 21:51:38.030232 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028156 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 23 21:51:38.030232 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028159 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 23 21:51:38.030232 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028163 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 23 21:51:38.030232 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028165 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 23 21:51:38.030232 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028168 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 23 21:51:38.030579 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028171 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 23 21:51:38.030579 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028173 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 23 21:51:38.030579 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028176 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 23 21:51:38.030579 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028178 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 23 21:51:38.030579 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028180 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 23 21:51:38.030579 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028184 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 23 21:51:38.030579 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028188 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 23 21:51:38.030579 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028191 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 23 21:51:38.030579 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028194 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 23 21:51:38.030579 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028197 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 23 21:51:38.030579 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028200 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 23 21:51:38.030579 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028202 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 23 21:51:38.030579 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028205 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 23 21:51:38.030579 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028208 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 23 21:51:38.030579 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028210 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 23 21:51:38.030579 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028212 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 23 21:51:38.030579 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028215 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 23 21:51:38.030579 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028217 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 23 21:51:38.030579 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028219 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 23 21:51:38.030579 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028222 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 23 21:51:38.031083 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028225 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 23 21:51:38.031083 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028228 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 23 21:51:38.031083 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028231 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 23 21:51:38.031083 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028234 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 23 21:51:38.031083 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028237 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 23 21:51:38.031083 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028239 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 23 21:51:38.031083 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028242 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 23 21:51:38.031083 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028245 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 23 21:51:38.031083 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028247 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 23 21:51:38.031083 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028249 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 23 21:51:38.031083 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028252 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 23 21:51:38.031083 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028254 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 23 21:51:38.031083 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028256 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 23 21:51:38.031083 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028258 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 23 21:51:38.031083 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028261 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 23 21:51:38.031083 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028263 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 23 21:51:38.031083 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028265 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 23 21:51:38.031083 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028268 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 23 21:51:38.031083 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028270 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 23 21:51:38.031595 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028272 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 23 21:51:38.031595 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028274 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 23 21:51:38.031595 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028277 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 23 21:51:38.031595 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028279 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 23 21:51:38.031595 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028281 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 23 21:51:38.031595 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028284 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 23 21:51:38.031595 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028286 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 23 21:51:38.031595 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028288 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 23 21:51:38.031595 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028291 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 23 21:51:38.031595 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028293 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 23 21:51:38.031595 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028296 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 23 21:51:38.031595 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028298 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 23 21:51:38.031595 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028301 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 23 21:51:38.031595 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028303 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 23 21:51:38.031595 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028306 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 23 21:51:38.031595 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028308 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 23 21:51:38.031595 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028311 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 23 21:51:38.031595 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028315 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 23 21:51:38.031595 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028317 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 23 21:51:38.031595 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028321 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 23 21:51:38.032068 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028325 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 23 21:51:38.032068 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028327 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 23 21:51:38.032068 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028330 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 23 21:51:38.032068 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028332 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 23 21:51:38.032068 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028334 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 23 21:51:38.032068 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028337 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 23 21:51:38.032068 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028339 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 23 21:51:38.032068 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028341 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 23 21:51:38.032068 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028344 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 23 21:51:38.032068 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028346 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 23 21:51:38.032068 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028348 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 23 21:51:38.032068 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028351 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 23 21:51:38.032068 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028353 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 23 21:51:38.032068 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028356 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 23 21:51:38.032068 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028358 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 23 21:51:38.032068 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028361 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 23 21:51:38.032068 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028364 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 23 21:51:38.032068 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028366 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 23 21:51:38.032068 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028368 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 23 21:51:38.032068 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028371 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 23 21:51:38.032531 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028373 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 23 21:51:38.032531 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028376 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 23 21:51:38.032531 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028378 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 23 21:51:38.032531 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028380 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 23 21:51:38.032531 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028383 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 23 21:51:38.032531 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028385 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 23 21:51:38.032531 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028388 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 23 21:51:38.032531 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028390 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 23 21:51:38.032531 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028393 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 23 21:51:38.032531 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028395 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 23 21:51:38.032531 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028398 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 23 21:51:38.032531 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028401 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 23 21:51:38.032531 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028405 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 23 21:51:38.032531 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028408 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 23 21:51:38.032982 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.028414 2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 23 21:51:38.032982 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028525 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 23 21:51:38.032982 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028529 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 23 21:51:38.032982 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028532 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 23 21:51:38.032982 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028552 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 23 21:51:38.032982 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028556 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 23 21:51:38.032982 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028559 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 23 21:51:38.032982 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028562 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 23 21:51:38.032982 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028564 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 23 21:51:38.032982 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028567 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 23 21:51:38.032982 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028570 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 23 21:51:38.032982 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028572 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 23 21:51:38.032982 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028576 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 23 21:51:38.032982 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028579 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 23 21:51:38.033316 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028582 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 23 21:51:38.033316 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028586 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 23 21:51:38.033316 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028589 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 23 21:51:38.033316 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028592 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 23 21:51:38.033316 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028594 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 23 21:51:38.033316 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028597 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 23 21:51:38.033316 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028600 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 23 21:51:38.033316 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028602 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 23 21:51:38.033316 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028605 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 23 21:51:38.033316 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028607 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 23 21:51:38.033316 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028609 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 23 21:51:38.033316 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028612 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 23 21:51:38.033316 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028614 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 23 21:51:38.033316 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028618 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 23 21:51:38.033316 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028620 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 23 21:51:38.033316 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028623 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 23 21:51:38.033316 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028625 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 23 21:51:38.033316 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028628 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 23 21:51:38.033316 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028630 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 23 21:51:38.033804 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028633 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 23 21:51:38.033804 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028635 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 23 21:51:38.033804 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028638 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 23 21:51:38.033804 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028640 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 23 21:51:38.033804 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028643 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 23 21:51:38.033804 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028645 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 23 21:51:38.033804 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028648 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 23 21:51:38.033804 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028650 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 23 21:51:38.033804 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028652 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 23 21:51:38.033804 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028654 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 23 21:51:38.033804 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028657 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 23 21:51:38.033804 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028659 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 23 21:51:38.033804 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028661 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 23 21:51:38.033804 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028663 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 23 21:51:38.033804 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028666 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 23 21:51:38.033804 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028668 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 23 21:51:38.033804 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028671 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 23 21:51:38.033804 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028673 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 23 21:51:38.033804 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028675 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 23 21:51:38.033804 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028678 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 23 21:51:38.034324 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028680 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 23 21:51:38.034324 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028682 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 23 21:51:38.034324 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028685 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 23 21:51:38.034324 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028687 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 23 21:51:38.034324 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028689 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 23 21:51:38.034324 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028692 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 23 21:51:38.034324 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028694 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 23 21:51:38.034324 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028696 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 23 21:51:38.034324 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028699 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 23 21:51:38.034324 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028701 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 23 21:51:38.034324 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028704 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 23 21:51:38.034324 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028706 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 23 21:51:38.034324 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028708 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 23 21:51:38.034324 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028710 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 23 21:51:38.034324 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028712 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 23 21:51:38.034324 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028715 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 23 21:51:38.034324 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028717 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 23 21:51:38.034324 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028719 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 23 21:51:38.034324 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028721 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 23 21:51:38.034324 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028724 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 23 21:51:38.034838 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028726 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 23 21:51:38.034838 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028728 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 23 21:51:38.034838 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028730 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 23 21:51:38.034838 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028733 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 23 21:51:38.034838 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028735 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 23 21:51:38.034838 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028737 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 23 21:51:38.034838 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028740 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 23 21:51:38.034838 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028742 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 23 21:51:38.034838 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028744 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 23 21:51:38.034838 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028746 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 23 21:51:38.034838 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028748 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 23 21:51:38.034838 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028751 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 23 21:51:38.034838 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028753 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 23 21:51:38.034838 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028756 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 23 21:51:38.034838 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028758 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 23 21:51:38.034838 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028761 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 23 21:51:38.034838 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028765 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 23 21:51:38.034838 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028767 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 23 21:51:38.034838 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028770 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 23 21:51:38.034838 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028772 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 23 21:51:38.035601 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028775 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 23 21:51:38.035601 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028777 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 23 21:51:38.035601 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028780 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 23 21:51:38.035601 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028782 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 23 21:51:38.035601 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028785 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 23 21:51:38.035601 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028787 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 23 21:51:38.035601 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:38.028789 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 23 21:51:38.035601 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.028794 2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 23 21:51:38.035601 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.028906 2578 server.go:962] "Client rotation is on, will bootstrap in background" Jul 23 21:51:38.035601 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.031321 2578 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 23 21:51:38.035601 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.032532 2578 server.go:1019] "Starting client certificate rotation" Jul 23 21:51:38.035601 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.032668 2578 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 23 21:51:38.035601 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.032710 2578 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 23 21:51:38.036073 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.035652 2578 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 23 21:51:38.038272 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.038254 2578 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 23 21:51:38.051939 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.051918 2578 log.go:25] "Validated CRI v1 runtime API" Jul 23 21:51:38.054828 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.054809 2578 log.go:25] "Validated CRI v1 image API" Jul 23 21:51:38.056024 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.056004 2578 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 23 21:51:38.058236 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.058217 2578 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 b6d6daab-3099-4c05-aba1-96ccb504bcf0:/dev/nvme0n1p3 f6a47b9a-7d52-47df-8d26-73121f623caf:/dev/nvme0n1p4] Jul 23 21:51:38.058315 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.058235 2578 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 23 21:51:38.063732 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.063714 2578 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 23 21:51:38.065364 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.065257 2578 manager.go:216] Machine: {Timestamp:2026-07-23 21:51:38.063989798 +0000 UTC m=+0.305897663 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3199277 MemoryCapacity:32812175360 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec24462ac94c73e034040c491fdcde7b SystemUUID:ec24462a-c94c-73e0-3404-0c491fdcde7b BootID:c80dce17-3c23-4714-92ab-d392c4b94b97 Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ea:dc:3f:36:dd Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ea:dc:3f:36:dd Speed:0 Mtu:9001} {Name:ovs-system MacAddress:1a:7d:75:f9:31:eb Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812175360 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 23 21:51:38.065364 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.065362 2578 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 23 21:51:38.065471 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.065447 2578 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 23 21:51:38.068345 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.068322 2578 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 23 21:51:38.068472 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.068347 2578 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-136-157.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 23 21:51:38.069566 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.069556 2578 topology_manager.go:138] "Creating topology manager with none policy" Jul 23 21:51:38.069640 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.069568 2578 container_manager_linux.go:309] "Creating device plugin manager" Jul 23 21:51:38.069640 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.069575 2578 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 23 21:51:38.070231 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.070221 2578 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 23 21:51:38.071597 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.071587 2578 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 23 21:51:38.074339 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.074328 2578 state_mem.go:36] "Initialized new in-memory state store" Jul 23 21:51:38.074455 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.074446 2578 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 23 21:51:38.074490 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.074478 2578 kubelet.go:486] "Attempting to sync node with API server" Jul 23 21:51:38.074523 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.074493 2578 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 23 21:51:38.074523 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.074511 2578 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 23 21:51:38.074605 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.074523 2578 kubelet.go:398] "Adding apiserver pod source" Jul 23 21:51:38.074605 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.074556 2578 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 23 21:51:38.074996 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.074984 2578 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 23 21:51:38.075030 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.075004 2578 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 23 21:51:38.077524 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.077503 2578 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 23 21:51:38.079279 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.079266 2578 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 23 21:51:38.079326 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.079296 2578 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 23 21:51:38.080948 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.080933 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 23 21:51:38.081012 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.080954 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 23 21:51:38.081012 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.080960 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 23 21:51:38.081012 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.080966 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 23 21:51:38.081012 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.080974 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 23 21:51:38.081012 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.080979 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 23 21:51:38.081012 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.080987 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 23 21:51:38.081012 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.080992 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 23 21:51:38.081012 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.080999 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 23 21:51:38.081012 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.081004 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 23 21:51:38.081012 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.081013 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 23 21:51:38.081255 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.081018 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 23 21:51:38.082332 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.082320 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 23 21:51:38.082364 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.082334 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 23 21:51:38.085688 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.085669 2578 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-136-157.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 23 21:51:38.085771 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:51:38.085687 2578 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 23 21:51:38.085771 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:51:38.085686 2578 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-136-157.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 23 21:51:38.085846 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.085838 2578 server.go:1268] "Started kubelet" Jul 23 21:51:38.085952 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.085929 2578 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 23 21:51:38.086025 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.085978 2578 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 23 21:51:38.086119 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.086037 2578 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 23 21:51:38.087106 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.087082 2578 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-xgmcj" Jul 23 21:51:38.087232 ip-10-0-136-157 systemd[1]: Started Kubernetes Kubelet. Jul 23 21:51:38.088023 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.088010 2578 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 23 21:51:38.088877 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.088853 2578 server.go:310] "Adding debug handlers to kubelet server" Jul 23 21:51:38.091856 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:51:38.091832 2578 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 23 21:51:38.094433 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.094414 2578 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 23 21:51:38.094433 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.094427 2578 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 23 21:51:38.094570 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:51:38.094477 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-157.ec2.internal\" not found" Jul 23 21:51:38.095027 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.095009 2578 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-xgmcj" Jul 23 21:51:38.095103 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.095072 2578 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 23 21:51:38.095103 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.095069 2578 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 23 21:51:38.095103 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.095097 2578 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 23 21:51:38.095225 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.095179 2578 factory.go:55] Registering systemd factory Jul 23 21:51:38.095225 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.095195 2578 factory.go:223] Registration of the systemd container factory successfully Jul 23 21:51:38.095309 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.095300 2578 reconstruct.go:97] "Volume reconstruction finished" Jul 23 21:51:38.095309 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.095307 2578 reconciler.go:29] "Reconciler: start to sync state" Jul 23 21:51:38.095490 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.095472 2578 factory.go:153] Registering CRI-O factory Jul 23 21:51:38.095490 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.095490 2578 factory.go:223] Registration of the crio container factory successfully Jul 23 21:51:38.095622 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.095558 2578 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 23 21:51:38.095622 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.095600 2578 factory.go:103] Registering Raw factory Jul 23 21:51:38.095699 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.095678 2578 manager.go:1196] Started watching for new ooms in manager Jul 23 21:51:38.095753 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:51:38.094871 2578 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-157.ec2.internal.18c509b47b120742 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-157.ec2.internal,UID:ip-10-0-136-157.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-136-157.ec2.internal,},FirstTimestamp:2026-07-23 21:51:38.085812034 +0000 UTC m=+0.327719898,LastTimestamp:2026-07-23 21:51:38.085812034 +0000 UTC m=+0.327719898,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-157.ec2.internal,}" Jul 23 21:51:38.096256 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.096241 2578 manager.go:349] Starting recovery of all containers Jul 23 21:51:38.099895 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:51:38.099843 2578 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-136-157.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 23 21:51:38.100085 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:51:38.100059 2578 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 23 21:51:38.105140 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.105109 2578 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 23 21:51:38.109395 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.109273 2578 manager.go:354] Recovery completed Jul 23 21:51:38.114392 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.114377 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 23 21:51:38.116294 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.116277 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-157.ec2.internal" event="NodeHasSufficientMemory" Jul 23 21:51:38.116359 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.116311 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-157.ec2.internal" event="NodeHasNoDiskPressure" Jul 23 21:51:38.116359 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.116325 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-157.ec2.internal" event="NodeHasSufficientPID" Jul 23 21:51:38.116789 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.116773 2578 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 23 21:51:38.116789 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.116784 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 23 21:51:38.116915 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.116785 2578 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 23 21:51:38.116915 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.116819 2578 state_mem.go:36] "Initialized new in-memory state store" Jul 23 21:51:38.117411 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.117399 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-157.ec2.internal" event="NodeHasSufficientMemory" Jul 23 21:51:38.117455 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.117421 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-157.ec2.internal" event="NodeHasNoDiskPressure" Jul 23 21:51:38.117455 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.117431 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-157.ec2.internal" event="NodeHasSufficientPID" Jul 23 21:51:38.118045 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.118033 2578 policy_none.go:49] "None policy: Start" Jul 23 21:51:38.118090 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.118049 2578 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 23 21:51:38.118090 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.118060 2578 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 23 21:51:38.119210 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.119200 2578 policy_none.go:47] "Start" Jul 23 21:51:38.162575 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.162383 2578 manager.go:337] "Starting Device Plugin manager" Jul 23 21:51:38.180077 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:51:38.162581 2578 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 23 21:51:38.180077 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.162596 2578 server.go:87] "Starting device plugin registration server" Jul 23 21:51:38.180077 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.162861 2578 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 23 21:51:38.180077 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.162873 2578 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 23 21:51:38.180077 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.162991 2578 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 23 21:51:38.180077 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.163102 2578 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 23 21:51:38.180077 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.163110 2578 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 23 21:51:38.180077 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:51:38.163689 2578 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 23 21:51:38.180077 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:51:38.163747 2578 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-136-157.ec2.internal\" not found" Jul 23 21:51:38.244883 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.244851 2578 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 23 21:51:38.245013 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.244894 2578 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 23 21:51:38.245013 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.244914 2578 kubelet.go:2448] "Starting kubelet main sync loop" Jul 23 21:51:38.245013 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:51:38.244943 2578 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 23 21:51:38.247500 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.247478 2578 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 23 21:51:38.263723 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.263653 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 23 21:51:38.264574 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.264558 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-157.ec2.internal" event="NodeHasSufficientMemory" Jul 23 21:51:38.264645 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.264589 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-157.ec2.internal" event="NodeHasNoDiskPressure" Jul 23 21:51:38.264645 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.264599 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-157.ec2.internal" event="NodeHasSufficientPID" Jul 23 21:51:38.264645 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.264623 2578 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-136-157.ec2.internal" Jul 23 21:51:38.270766 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.270747 2578 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-136-157.ec2.internal" Jul 23 21:51:38.270850 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:51:38.270772 2578 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-136-157.ec2.internal\": node \"ip-10-0-136-157.ec2.internal\" not found" Jul 23 21:51:38.288736 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:51:38.288708 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-157.ec2.internal\" not found" Jul 23 21:51:38.345398 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.345357 2578 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-157.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-136-157.ec2.internal"] Jul 23 21:51:38.345600 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.345433 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 23 21:51:38.346419 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.346403 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-157.ec2.internal" event="NodeHasSufficientMemory" Jul 23 21:51:38.346506 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.346433 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-157.ec2.internal" event="NodeHasNoDiskPressure" Jul 23 21:51:38.346506 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.346445 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-157.ec2.internal" event="NodeHasSufficientPID" Jul 23 21:51:38.347701 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.347688 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 23 21:51:38.347846 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.347834 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-157.ec2.internal" Jul 23 21:51:38.347892 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.347862 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 23 21:51:38.348592 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.348576 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-157.ec2.internal" event="NodeHasSufficientMemory" Jul 23 21:51:38.348682 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.348601 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-157.ec2.internal" event="NodeHasNoDiskPressure" Jul 23 21:51:38.348682 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.348614 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-157.ec2.internal" event="NodeHasSufficientPID" Jul 23 21:51:38.348682 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.348623 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-157.ec2.internal" event="NodeHasSufficientMemory" Jul 23 21:51:38.348682 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.348639 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-157.ec2.internal" event="NodeHasNoDiskPressure" Jul 23 21:51:38.348682 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.348648 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-157.ec2.internal" event="NodeHasSufficientPID" Jul 23 21:51:38.349749 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.349732 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-157.ec2.internal" Jul 23 21:51:38.349825 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.349758 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 23 21:51:38.350350 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.350338 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-157.ec2.internal" event="NodeHasSufficientMemory" Jul 23 21:51:38.350401 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.350363 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-157.ec2.internal" event="NodeHasNoDiskPressure" Jul 23 21:51:38.350401 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.350373 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-157.ec2.internal" event="NodeHasSufficientPID" Jul 23 21:51:38.376231 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:51:38.376211 2578 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-157.ec2.internal\" not found" node="ip-10-0-136-157.ec2.internal" Jul 23 21:51:38.380635 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:51:38.380621 2578 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-157.ec2.internal\" not found" node="ip-10-0-136-157.ec2.internal" Jul 23 21:51:38.389261 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:51:38.389247 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-157.ec2.internal\" not found" Jul 23 21:51:38.397394 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.397373 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a9bc0c1f6ccfd7b14a1f3c2bc283e4db-config\") pod \"kube-apiserver-proxy-ip-10-0-136-157.ec2.internal\" (UID: \"a9bc0c1f6ccfd7b14a1f3c2bc283e4db\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-157.ec2.internal" Jul 23 21:51:38.397450 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.397399 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/8746433a99157ef819679bf50ca9e8e4-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-157.ec2.internal\" (UID: \"8746433a99157ef819679bf50ca9e8e4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-157.ec2.internal" Jul 23 21:51:38.397450 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.397418 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8746433a99157ef819679bf50ca9e8e4-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-157.ec2.internal\" (UID: \"8746433a99157ef819679bf50ca9e8e4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-157.ec2.internal" Jul 23 21:51:38.490328 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:51:38.490299 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-157.ec2.internal\" not found" Jul 23 21:51:38.497579 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.497557 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a9bc0c1f6ccfd7b14a1f3c2bc283e4db-config\") pod \"kube-apiserver-proxy-ip-10-0-136-157.ec2.internal\" (UID: \"a9bc0c1f6ccfd7b14a1f3c2bc283e4db\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-157.ec2.internal" Jul 23 21:51:38.497636 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.497587 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/8746433a99157ef819679bf50ca9e8e4-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-157.ec2.internal\" (UID: \"8746433a99157ef819679bf50ca9e8e4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-157.ec2.internal" Jul 23 21:51:38.497636 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.497613 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a9bc0c1f6ccfd7b14a1f3c2bc283e4db-config\") pod \"kube-apiserver-proxy-ip-10-0-136-157.ec2.internal\" (UID: \"a9bc0c1f6ccfd7b14a1f3c2bc283e4db\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-157.ec2.internal" Jul 23 21:51:38.497692 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.497641 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8746433a99157ef819679bf50ca9e8e4-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-157.ec2.internal\" (UID: \"8746433a99157ef819679bf50ca9e8e4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-157.ec2.internal" Jul 23 21:51:38.497692 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.497617 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8746433a99157ef819679bf50ca9e8e4-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-157.ec2.internal\" (UID: \"8746433a99157ef819679bf50ca9e8e4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-157.ec2.internal" Jul 23 21:51:38.497692 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.497647 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/8746433a99157ef819679bf50ca9e8e4-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-157.ec2.internal\" (UID: \"8746433a99157ef819679bf50ca9e8e4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-157.ec2.internal" Jul 23 21:51:38.591039 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:51:38.590979 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-157.ec2.internal\" not found" Jul 23 21:51:38.679504 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.679463 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-157.ec2.internal" Jul 23 21:51:38.683669 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:38.683276 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-157.ec2.internal" Jul 23 21:51:38.691487 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:51:38.691465 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-157.ec2.internal\" not found" Jul 23 21:51:38.792065 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:51:38.792011 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-157.ec2.internal\" not found" Jul 23 21:51:38.892590 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:51:38.892490 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-157.ec2.internal\" not found" Jul 23 21:51:38.993088 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:51:38.993030 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-157.ec2.internal\" not found" Jul 23 21:51:39.031921 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:39.031889 2578 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 23 21:51:39.032425 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:39.032036 2578 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 23 21:51:39.093231 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:51:39.093205 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-157.ec2.internal\" not found" Jul 23 21:51:39.095617 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:39.095594 2578 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 23 21:51:39.099314 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:39.099267 2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-22 21:46:38 +0000 UTC" deadline="2027-12-17 00:36:56.084418295 +0000 UTC" Jul 23 21:51:39.099314 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:39.099309 2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12266h45m16.985111773s" Jul 23 21:51:39.178852 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:39.178822 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8746433a99157ef819679bf50ca9e8e4.slice/crio-cc228f7a6bcf9e59d69a80c49e57870b23040f10e1a1c33647805f0c1900b0c6 WatchSource:0}: Error finding container cc228f7a6bcf9e59d69a80c49e57870b23040f10e1a1c33647805f0c1900b0c6: Status 404 returned error can't find the container with id cc228f7a6bcf9e59d69a80c49e57870b23040f10e1a1c33647805f0c1900b0c6 Jul 23 21:51:39.179260 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:39.179237 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9bc0c1f6ccfd7b14a1f3c2bc283e4db.slice/crio-8abfed73499b3ce2079f9a8f95d3b1af1ac28bdeb6a8962233b4d45e6824be3a WatchSource:0}: Error finding container 8abfed73499b3ce2079f9a8f95d3b1af1ac28bdeb6a8962233b4d45e6824be3a: Status 404 returned error can't find the container with id 8abfed73499b3ce2079f9a8f95d3b1af1ac28bdeb6a8962233b4d45e6824be3a Jul 23 21:51:39.183648 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:39.183631 2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 23 21:51:39.193694 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:51:39.193674 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-157.ec2.internal\" not found" Jul 23 21:51:39.248298 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:39.248245 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-157.ec2.internal" event={"ID":"8746433a99157ef819679bf50ca9e8e4","Type":"ContainerStarted","Data":"cc228f7a6bcf9e59d69a80c49e57870b23040f10e1a1c33647805f0c1900b0c6"} Jul 23 21:51:39.249257 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:39.249236 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-157.ec2.internal" event={"ID":"a9bc0c1f6ccfd7b14a1f3c2bc283e4db","Type":"ContainerStarted","Data":"8abfed73499b3ce2079f9a8f95d3b1af1ac28bdeb6a8962233b4d45e6824be3a"} Jul 23 21:51:39.294446 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:51:39.294398 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-157.ec2.internal\" not found" Jul 23 21:51:39.344424 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:39.344396 2578 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 23 21:51:39.395510 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:39.395459 2578 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-157.ec2.internal" Jul 23 21:51:39.407210 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:39.407189 2578 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 23 21:51:39.408173 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:39.408159 2578 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-157.ec2.internal" Jul 23 21:51:39.415418 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:39.415404 2578 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 23 21:51:39.511343 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:39.511272 2578 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 23 21:51:40.075402 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.075362 2578 apiserver.go:52] "Watching apiserver" Jul 23 21:51:40.083179 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.083149 2578 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 23 21:51:40.083614 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.083589 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-operator/iptables-alerter-cmqcr","openshift-ovn-kubernetes/ovnkube-node-7jkjm","kube-system/kube-apiserver-proxy-ip-10-0-136-157.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4pdct","openshift-cluster-node-tuning-operator/tuned-qf8mt","openshift-dns/node-resolver-w6jzd","openshift-multus/multus-additional-cni-plugins-ntjs5","openshift-multus/multus-d5lp9","openshift-multus/network-metrics-daemon-kl5st","openshift-network-diagnostics/network-check-target-4cq9t","kube-system/global-pull-secret-syncer-26shl","kube-system/konnectivity-agent-rpgtk","openshift-image-registry/node-ca-nt77r","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-157.ec2.internal"] Jul 23 21:51:40.085352 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.085323 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-ntjs5" Jul 23 21:51:40.086657 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.086626 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-nt77r" Jul 23 21:51:40.088286 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.087989 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-d5lp9" Jul 23 21:51:40.088405 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.088349 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-6g78c\"" Jul 23 21:51:40.088499 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.088468 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 23 21:51:40.088825 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.088808 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 23 21:51:40.089078 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.089062 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 23 21:51:40.089646 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.089361 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 23 21:51:40.089646 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.089401 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-8dfmj\"" Jul 23 21:51:40.089646 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.089429 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4pdct" Jul 23 21:51:40.089646 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.089442 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 23 21:51:40.089900 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.089885 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 23 21:51:40.089971 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.089895 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 23 21:51:40.090557 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.090416 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-27b97\"" Jul 23 21:51:40.090667 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.090566 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 23 21:51:40.090727 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.090670 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 23 21:51:40.090908 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.090883 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-qf8mt" Jul 23 21:51:40.092448 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.092164 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-w6jzd" Jul 23 21:51:40.093582 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.093564 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kl5st" Jul 23 21:51:40.093699 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:51:40.093655 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kl5st" podUID="7c998f5f-33c9-44c6-abf7-f042f767fc4f" Jul 23 21:51:40.094591 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.094570 2578 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 23 21:51:40.095109 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.095092 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-cmqcr" Jul 23 21:51:40.098601 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.097344 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-7jkjm" Jul 23 21:51:40.099064 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.099041 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4cq9t" Jul 23 21:51:40.099169 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:51:40.099127 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4cq9t" podUID="ef1bd359-5601-4a79-9b52-03d6e6a49bc6" Jul 23 21:51:40.101036 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.101010 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-26shl" Jul 23 21:51:40.101155 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:51:40.101127 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-26shl" podUID="500a30f7-ac15-47da-adf9-d0485f80af93" Jul 23 21:51:40.101266 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.101233 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 23 21:51:40.101325 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.101262 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 23 21:51:40.101396 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.101378 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 23 21:51:40.101510 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.101473 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 23 21:51:40.101582 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.101525 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 23 21:51:40.101582 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.101555 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 23 21:51:40.101717 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.101637 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 23 21:51:40.101717 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.101642 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-8zfks\"" Jul 23 21:51:40.101717 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.101667 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 23 21:51:40.101850 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.101741 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 23 21:51:40.101850 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.101818 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 23 21:51:40.101951 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.101901 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 23 21:51:40.102334 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.102312 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 23 21:51:40.102435 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.102389 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 23 21:51:40.102435 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.102317 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-4nl8p\"" Jul 23 21:51:40.102435 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.102433 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 23 21:51:40.102739 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.102707 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-c8tnz\"" Jul 23 21:51:40.102739 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.102714 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-ds2rt\"" Jul 23 21:51:40.102849 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.102759 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-gf6qx\"" Jul 23 21:51:40.102900 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.102869 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 23 21:51:40.103043 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.103024 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 23 21:51:40.103794 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.103778 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-rpgtk" Jul 23 21:51:40.106660 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.106644 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-mg6mh\"" Jul 23 21:51:40.106660 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.106660 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 23 21:51:40.106839 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.106696 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 23 21:51:40.107035 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.107015 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-crj5k\" (UniqueName: \"kubernetes.io/projected/49757977-27cf-4177-b295-c830e4a71b03-kube-api-access-crj5k\") pod \"node-ca-nt77r\" (UID: \"49757977-27cf-4177-b295-c830e4a71b03\") " pod="openshift-image-registry/node-ca-nt77r" Jul 23 21:51:40.107130 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.107044 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/7342a9f6-72b1-4e3c-8d50-0b0326454a86-run-ovn\") pod \"ovnkube-node-7jkjm\" (UID: \"7342a9f6-72b1-4e3c-8d50-0b0326454a86\") " pod="openshift-ovn-kubernetes/ovnkube-node-7jkjm" Jul 23 21:51:40.107130 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.107061 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7342a9f6-72b1-4e3c-8d50-0b0326454a86-host-cni-bin\") pod \"ovnkube-node-7jkjm\" (UID: \"7342a9f6-72b1-4e3c-8d50-0b0326454a86\") " pod="openshift-ovn-kubernetes/ovnkube-node-7jkjm" Jul 23 21:51:40.107130 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.107077 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4e42ce8b-422d-4281-b12e-6371c9744f8a-hosts-file\") pod \"node-resolver-w6jzd\" (UID: \"4e42ce8b-422d-4281-b12e-6371c9744f8a\") " pod="openshift-dns/node-resolver-w6jzd" Jul 23 21:51:40.107130 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.107100 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6d542711-813a-4501-a243-b799ddd73340-kubelet-dir\") pod \"aws-ebs-csi-driver-node-4pdct\" (UID: \"6d542711-813a-4501-a243-b799ddd73340\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4pdct" Jul 23 21:51:40.107330 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.107134 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/93c54b15-009c-43ca-93a6-5a03909fc703-iptables-alerter-script\") pod \"iptables-alerter-cmqcr\" (UID: \"93c54b15-009c-43ca-93a6-5a03909fc703\") " pod="openshift-network-operator/iptables-alerter-cmqcr" Jul 23 21:51:40.107330 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.107163 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/7342a9f6-72b1-4e3c-8d50-0b0326454a86-systemd-units\") pod \"ovnkube-node-7jkjm\" (UID: \"7342a9f6-72b1-4e3c-8d50-0b0326454a86\") " pod="openshift-ovn-kubernetes/ovnkube-node-7jkjm" Jul 23 21:51:40.107330 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.107186 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7342a9f6-72b1-4e3c-8d50-0b0326454a86-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7jkjm\" (UID: \"7342a9f6-72b1-4e3c-8d50-0b0326454a86\") " pod="openshift-ovn-kubernetes/ovnkube-node-7jkjm" Jul 23 21:51:40.107330 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.107211 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/7342a9f6-72b1-4e3c-8d50-0b0326454a86-ovnkube-script-lib\") pod \"ovnkube-node-7jkjm\" (UID: \"7342a9f6-72b1-4e3c-8d50-0b0326454a86\") " pod="openshift-ovn-kubernetes/ovnkube-node-7jkjm" Jul 23 21:51:40.107330 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.107256 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/49757977-27cf-4177-b295-c830e4a71b03-serviceca\") pod \"node-ca-nt77r\" (UID: \"49757977-27cf-4177-b295-c830e4a71b03\") " pod="openshift-image-registry/node-ca-nt77r" Jul 23 21:51:40.107330 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.107291 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6d542711-813a-4501-a243-b799ddd73340-socket-dir\") pod \"aws-ebs-csi-driver-node-4pdct\" (UID: \"6d542711-813a-4501-a243-b799ddd73340\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4pdct" Jul 23 21:51:40.107523 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.107319 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9d06a30f-ed03-457e-8318-7d8870ec4eb4-etc-kubernetes\") pod \"tuned-qf8mt\" (UID: \"9d06a30f-ed03-457e-8318-7d8870ec4eb4\") " pod="openshift-cluster-node-tuning-operator/tuned-qf8mt" Jul 23 21:51:40.107523 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.107361 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1b8773cc-8521-4192-a3ab-0a397ba6eb62-os-release\") pod \"multus-d5lp9\" (UID: \"1b8773cc-8521-4192-a3ab-0a397ba6eb62\") " pod="openshift-multus/multus-d5lp9" Jul 23 21:51:40.107523 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.107384 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1b8773cc-8521-4192-a3ab-0a397ba6eb62-etc-kubernetes\") pod \"multus-d5lp9\" (UID: \"1b8773cc-8521-4192-a3ab-0a397ba6eb62\") " pod="openshift-multus/multus-d5lp9" Jul 23 21:51:40.107523 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.107405 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9d06a30f-ed03-457e-8318-7d8870ec4eb4-run\") pod \"tuned-qf8mt\" (UID: \"9d06a30f-ed03-457e-8318-7d8870ec4eb4\") " pod="openshift-cluster-node-tuning-operator/tuned-qf8mt" Jul 23 21:51:40.107523 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.107426 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/7342a9f6-72b1-4e3c-8d50-0b0326454a86-run-systemd\") pod \"ovnkube-node-7jkjm\" (UID: \"7342a9f6-72b1-4e3c-8d50-0b0326454a86\") " pod="openshift-ovn-kubernetes/ovnkube-node-7jkjm" Jul 23 21:51:40.107523 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.107483 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/7342a9f6-72b1-4e3c-8d50-0b0326454a86-host-cni-netd\") pod \"ovnkube-node-7jkjm\" (UID: \"7342a9f6-72b1-4e3c-8d50-0b0326454a86\") " pod="openshift-ovn-kubernetes/ovnkube-node-7jkjm" Jul 23 21:51:40.107523 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.107502 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/7342a9f6-72b1-4e3c-8d50-0b0326454a86-ovnkube-config\") pod \"ovnkube-node-7jkjm\" (UID: \"7342a9f6-72b1-4e3c-8d50-0b0326454a86\") " pod="openshift-ovn-kubernetes/ovnkube-node-7jkjm" Jul 23 21:51:40.107787 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.107527 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/7342a9f6-72b1-4e3c-8d50-0b0326454a86-ovn-node-metrics-cert\") pod \"ovnkube-node-7jkjm\" (UID: \"7342a9f6-72b1-4e3c-8d50-0b0326454a86\") " pod="openshift-ovn-kubernetes/ovnkube-node-7jkjm" Jul 23 21:51:40.107787 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.107551 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c998f5f-33c9-44c6-abf7-f042f767fc4f-metrics-certs\") pod \"network-metrics-daemon-kl5st\" (UID: \"7c998f5f-33c9-44c6-abf7-f042f767fc4f\") " pod="openshift-multus/network-metrics-daemon-kl5st" Jul 23 21:51:40.107787 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.107609 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6d542711-813a-4501-a243-b799ddd73340-registration-dir\") pod \"aws-ebs-csi-driver-node-4pdct\" (UID: \"6d542711-813a-4501-a243-b799ddd73340\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4pdct" Jul 23 21:51:40.107787 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.107647 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6d542711-813a-4501-a243-b799ddd73340-device-dir\") pod \"aws-ebs-csi-driver-node-4pdct\" (UID: \"6d542711-813a-4501-a243-b799ddd73340\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4pdct" Jul 23 21:51:40.107787 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.107670 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9d06a30f-ed03-457e-8318-7d8870ec4eb4-etc-tuned\") pod \"tuned-qf8mt\" (UID: \"9d06a30f-ed03-457e-8318-7d8870ec4eb4\") " pod="openshift-cluster-node-tuning-operator/tuned-qf8mt" Jul 23 21:51:40.107787 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.107693 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qhnlm\" (UniqueName: \"kubernetes.io/projected/1b8773cc-8521-4192-a3ab-0a397ba6eb62-kube-api-access-qhnlm\") pod \"multus-d5lp9\" (UID: \"1b8773cc-8521-4192-a3ab-0a397ba6eb62\") " pod="openshift-multus/multus-d5lp9" Jul 23 21:51:40.107787 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.107717 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7342a9f6-72b1-4e3c-8d50-0b0326454a86-var-lib-openvswitch\") pod \"ovnkube-node-7jkjm\" (UID: \"7342a9f6-72b1-4e3c-8d50-0b0326454a86\") " pod="openshift-ovn-kubernetes/ovnkube-node-7jkjm" Jul 23 21:51:40.107787 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.107739 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1b8773cc-8521-4192-a3ab-0a397ba6eb62-system-cni-dir\") pod \"multus-d5lp9\" (UID: \"1b8773cc-8521-4192-a3ab-0a397ba6eb62\") " pod="openshift-multus/multus-d5lp9" Jul 23 21:51:40.107787 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.107775 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6d542711-813a-4501-a243-b799ddd73340-etc-selinux\") pod \"aws-ebs-csi-driver-node-4pdct\" (UID: \"6d542711-813a-4501-a243-b799ddd73340\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4pdct" Jul 23 21:51:40.108213 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.107814 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9d06a30f-ed03-457e-8318-7d8870ec4eb4-lib-modules\") pod \"tuned-qf8mt\" (UID: \"9d06a30f-ed03-457e-8318-7d8870ec4eb4\") " pod="openshift-cluster-node-tuning-operator/tuned-qf8mt" Jul 23 21:51:40.108213 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.107856 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/93c54b15-009c-43ca-93a6-5a03909fc703-host-slash\") pod \"iptables-alerter-cmqcr\" (UID: \"93c54b15-009c-43ca-93a6-5a03909fc703\") " pod="openshift-network-operator/iptables-alerter-cmqcr" Jul 23 21:51:40.108213 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.107905 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/7342a9f6-72b1-4e3c-8d50-0b0326454a86-log-socket\") pod \"ovnkube-node-7jkjm\" (UID: \"7342a9f6-72b1-4e3c-8d50-0b0326454a86\") " pod="openshift-ovn-kubernetes/ovnkube-node-7jkjm" Jul 23 21:51:40.108213 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.107946 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/49757977-27cf-4177-b295-c830e4a71b03-host\") pod \"node-ca-nt77r\" (UID: \"49757977-27cf-4177-b295-c830e4a71b03\") " pod="openshift-image-registry/node-ca-nt77r" Jul 23 21:51:40.108213 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.107975 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7342a9f6-72b1-4e3c-8d50-0b0326454a86-host-run-ovn-kubernetes\") pod \"ovnkube-node-7jkjm\" (UID: \"7342a9f6-72b1-4e3c-8d50-0b0326454a86\") " pod="openshift-ovn-kubernetes/ovnkube-node-7jkjm" Jul 23 21:51:40.108213 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.108008 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1e27638c-41d9-441b-8cea-dd134bf51985-system-cni-dir\") pod \"multus-additional-cni-plugins-ntjs5\" (UID: \"1e27638c-41d9-441b-8cea-dd134bf51985\") " pod="openshift-multus/multus-additional-cni-plugins-ntjs5" Jul 23 21:51:40.108213 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.108033 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1e27638c-41d9-441b-8cea-dd134bf51985-os-release\") pod \"multus-additional-cni-plugins-ntjs5\" (UID: \"1e27638c-41d9-441b-8cea-dd134bf51985\") " pod="openshift-multus/multus-additional-cni-plugins-ntjs5" Jul 23 21:51:40.108213 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.108056 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9d06a30f-ed03-457e-8318-7d8870ec4eb4-tmp\") pod \"tuned-qf8mt\" (UID: \"9d06a30f-ed03-457e-8318-7d8870ec4eb4\") " pod="openshift-cluster-node-tuning-operator/tuned-qf8mt" Jul 23 21:51:40.108213 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.108079 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7342a9f6-72b1-4e3c-8d50-0b0326454a86-host-run-netns\") pod \"ovnkube-node-7jkjm\" (UID: \"7342a9f6-72b1-4e3c-8d50-0b0326454a86\") " pod="openshift-ovn-kubernetes/ovnkube-node-7jkjm" Jul 23 21:51:40.108213 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.108103 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7342a9f6-72b1-4e3c-8d50-0b0326454a86-etc-openvswitch\") pod \"ovnkube-node-7jkjm\" (UID: \"7342a9f6-72b1-4e3c-8d50-0b0326454a86\") " pod="openshift-ovn-kubernetes/ovnkube-node-7jkjm" Jul 23 21:51:40.108213 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.108126 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1b8773cc-8521-4192-a3ab-0a397ba6eb62-cni-binary-copy\") pod \"multus-d5lp9\" (UID: \"1b8773cc-8521-4192-a3ab-0a397ba6eb62\") " pod="openshift-multus/multus-d5lp9" Jul 23 21:51:40.108213 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.108151 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1b8773cc-8521-4192-a3ab-0a397ba6eb62-hostroot\") pod \"multus-d5lp9\" (UID: \"1b8773cc-8521-4192-a3ab-0a397ba6eb62\") " pod="openshift-multus/multus-d5lp9" Jul 23 21:51:40.108213 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.108176 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1b8773cc-8521-4192-a3ab-0a397ba6eb62-multus-daemon-config\") pod \"multus-d5lp9\" (UID: \"1b8773cc-8521-4192-a3ab-0a397ba6eb62\") " pod="openshift-multus/multus-d5lp9" Jul 23 21:51:40.108213 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.108201 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1e27638c-41d9-441b-8cea-dd134bf51985-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ntjs5\" (UID: \"1e27638c-41d9-441b-8cea-dd134bf51985\") " pod="openshift-multus/multus-additional-cni-plugins-ntjs5" Jul 23 21:51:40.108907 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.108222 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1b8773cc-8521-4192-a3ab-0a397ba6eb62-host-run-netns\") pod \"multus-d5lp9\" (UID: \"1b8773cc-8521-4192-a3ab-0a397ba6eb62\") " pod="openshift-multus/multus-d5lp9" Jul 23 21:51:40.108907 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.108247 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1b8773cc-8521-4192-a3ab-0a397ba6eb62-host-run-multus-certs\") pod \"multus-d5lp9\" (UID: \"1b8773cc-8521-4192-a3ab-0a397ba6eb62\") " pod="openshift-multus/multus-d5lp9" Jul 23 21:51:40.108907 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.108274 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9d06a30f-ed03-457e-8318-7d8870ec4eb4-etc-sysconfig\") pod \"tuned-qf8mt\" (UID: \"9d06a30f-ed03-457e-8318-7d8870ec4eb4\") " pod="openshift-cluster-node-tuning-operator/tuned-qf8mt" Jul 23 21:51:40.108907 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.108297 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9d06a30f-ed03-457e-8318-7d8870ec4eb4-etc-systemd\") pod \"tuned-qf8mt\" (UID: \"9d06a30f-ed03-457e-8318-7d8870ec4eb4\") " pod="openshift-cluster-node-tuning-operator/tuned-qf8mt" Jul 23 21:51:40.108907 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.108340 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gwcfk\" (UniqueName: \"kubernetes.io/projected/9d06a30f-ed03-457e-8318-7d8870ec4eb4-kube-api-access-gwcfk\") pod \"tuned-qf8mt\" (UID: \"9d06a30f-ed03-457e-8318-7d8870ec4eb4\") " pod="openshift-cluster-node-tuning-operator/tuned-qf8mt" Jul 23 21:51:40.108907 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.108389 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e42ce8b-422d-4281-b12e-6371c9744f8a-tmp-dir\") pod \"node-resolver-w6jzd\" (UID: \"4e42ce8b-422d-4281-b12e-6371c9744f8a\") " pod="openshift-dns/node-resolver-w6jzd" Jul 23 21:51:40.108907 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.108413 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1b8773cc-8521-4192-a3ab-0a397ba6eb62-host-var-lib-kubelet\") pod \"multus-d5lp9\" (UID: \"1b8773cc-8521-4192-a3ab-0a397ba6eb62\") " pod="openshift-multus/multus-d5lp9" Jul 23 21:51:40.108907 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.108430 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9bnw9\" (UniqueName: \"kubernetes.io/projected/7c998f5f-33c9-44c6-abf7-f042f767fc4f-kube-api-access-9bnw9\") pod \"network-metrics-daemon-kl5st\" (UID: \"7c998f5f-33c9-44c6-abf7-f042f767fc4f\") " pod="openshift-multus/network-metrics-daemon-kl5st" Jul 23 21:51:40.108907 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.108453 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1e27638c-41d9-441b-8cea-dd134bf51985-cni-binary-copy\") pod \"multus-additional-cni-plugins-ntjs5\" (UID: \"1e27638c-41d9-441b-8cea-dd134bf51985\") " pod="openshift-multus/multus-additional-cni-plugins-ntjs5" Jul 23 21:51:40.108907 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.108476 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6d542711-813a-4501-a243-b799ddd73340-sys-fs\") pod \"aws-ebs-csi-driver-node-4pdct\" (UID: \"6d542711-813a-4501-a243-b799ddd73340\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4pdct" Jul 23 21:51:40.108907 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.108501 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f9skc\" (UniqueName: \"kubernetes.io/projected/93c54b15-009c-43ca-93a6-5a03909fc703-kube-api-access-f9skc\") pod \"iptables-alerter-cmqcr\" (UID: \"93c54b15-009c-43ca-93a6-5a03909fc703\") " pod="openshift-network-operator/iptables-alerter-cmqcr" Jul 23 21:51:40.108907 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.108532 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7342a9f6-72b1-4e3c-8d50-0b0326454a86-run-openvswitch\") pod \"ovnkube-node-7jkjm\" (UID: \"7342a9f6-72b1-4e3c-8d50-0b0326454a86\") " pod="openshift-ovn-kubernetes/ovnkube-node-7jkjm" Jul 23 21:51:40.108907 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.108570 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1b8773cc-8521-4192-a3ab-0a397ba6eb62-multus-cni-dir\") pod \"multus-d5lp9\" (UID: \"1b8773cc-8521-4192-a3ab-0a397ba6eb62\") " pod="openshift-multus/multus-d5lp9" Jul 23 21:51:40.108907 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.108596 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1b8773cc-8521-4192-a3ab-0a397ba6eb62-multus-socket-dir-parent\") pod \"multus-d5lp9\" (UID: \"1b8773cc-8521-4192-a3ab-0a397ba6eb62\") " pod="openshift-multus/multus-d5lp9" Jul 23 21:51:40.108907 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.108617 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1e27638c-41d9-441b-8cea-dd134bf51985-cnibin\") pod \"multus-additional-cni-plugins-ntjs5\" (UID: \"1e27638c-41d9-441b-8cea-dd134bf51985\") " pod="openshift-multus/multus-additional-cni-plugins-ntjs5" Jul 23 21:51:40.108907 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.108638 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/1e27638c-41d9-441b-8cea-dd134bf51985-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ntjs5\" (UID: \"1e27638c-41d9-441b-8cea-dd134bf51985\") " pod="openshift-multus/multus-additional-cni-plugins-ntjs5" Jul 23 21:51:40.109681 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.108679 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9d06a30f-ed03-457e-8318-7d8870ec4eb4-etc-sysctl-d\") pod \"tuned-qf8mt\" (UID: \"9d06a30f-ed03-457e-8318-7d8870ec4eb4\") " pod="openshift-cluster-node-tuning-operator/tuned-qf8mt" Jul 23 21:51:40.109681 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.108702 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9d06a30f-ed03-457e-8318-7d8870ec4eb4-var-lib-kubelet\") pod \"tuned-qf8mt\" (UID: \"9d06a30f-ed03-457e-8318-7d8870ec4eb4\") " pod="openshift-cluster-node-tuning-operator/tuned-qf8mt" Jul 23 21:51:40.109681 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.108746 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zqqkd\" (UniqueName: \"kubernetes.io/projected/7342a9f6-72b1-4e3c-8d50-0b0326454a86-kube-api-access-zqqkd\") pod \"ovnkube-node-7jkjm\" (UID: \"7342a9f6-72b1-4e3c-8d50-0b0326454a86\") " pod="openshift-ovn-kubernetes/ovnkube-node-7jkjm" Jul 23 21:51:40.109681 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.108769 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1b8773cc-8521-4192-a3ab-0a397ba6eb62-cnibin\") pod \"multus-d5lp9\" (UID: \"1b8773cc-8521-4192-a3ab-0a397ba6eb62\") " pod="openshift-multus/multus-d5lp9" Jul 23 21:51:40.109681 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.108791 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1b8773cc-8521-4192-a3ab-0a397ba6eb62-host-var-lib-cni-multus\") pod \"multus-d5lp9\" (UID: \"1b8773cc-8521-4192-a3ab-0a397ba6eb62\") " pod="openshift-multus/multus-d5lp9" Jul 23 21:51:40.109681 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.108816 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1b8773cc-8521-4192-a3ab-0a397ba6eb62-multus-conf-dir\") pod \"multus-d5lp9\" (UID: \"1b8773cc-8521-4192-a3ab-0a397ba6eb62\") " pod="openshift-multus/multus-d5lp9" Jul 23 21:51:40.109681 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.108831 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/7342a9f6-72b1-4e3c-8d50-0b0326454a86-env-overrides\") pod \"ovnkube-node-7jkjm\" (UID: \"7342a9f6-72b1-4e3c-8d50-0b0326454a86\") " pod="openshift-ovn-kubernetes/ovnkube-node-7jkjm" Jul 23 21:51:40.109681 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.108853 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/1e27638c-41d9-441b-8cea-dd134bf51985-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ntjs5\" (UID: \"1e27638c-41d9-441b-8cea-dd134bf51985\") " pod="openshift-multus/multus-additional-cni-plugins-ntjs5" Jul 23 21:51:40.109681 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.108890 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8xsxf\" (UniqueName: \"kubernetes.io/projected/6d542711-813a-4501-a243-b799ddd73340-kube-api-access-8xsxf\") pod \"aws-ebs-csi-driver-node-4pdct\" (UID: \"6d542711-813a-4501-a243-b799ddd73340\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4pdct" Jul 23 21:51:40.109681 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.108906 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9d06a30f-ed03-457e-8318-7d8870ec4eb4-etc-sysctl-conf\") pod \"tuned-qf8mt\" (UID: \"9d06a30f-ed03-457e-8318-7d8870ec4eb4\") " pod="openshift-cluster-node-tuning-operator/tuned-qf8mt" Jul 23 21:51:40.109681 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.108920 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/7342a9f6-72b1-4e3c-8d50-0b0326454a86-host-kubelet\") pod \"ovnkube-node-7jkjm\" (UID: \"7342a9f6-72b1-4e3c-8d50-0b0326454a86\") " pod="openshift-ovn-kubernetes/ovnkube-node-7jkjm" Jul 23 21:51:40.109681 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.108935 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/7342a9f6-72b1-4e3c-8d50-0b0326454a86-node-log\") pod \"ovnkube-node-7jkjm\" (UID: \"7342a9f6-72b1-4e3c-8d50-0b0326454a86\") " pod="openshift-ovn-kubernetes/ovnkube-node-7jkjm" Jul 23 21:51:40.109681 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.108953 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1b8773cc-8521-4192-a3ab-0a397ba6eb62-host-var-lib-cni-bin\") pod \"multus-d5lp9\" (UID: \"1b8773cc-8521-4192-a3ab-0a397ba6eb62\") " pod="openshift-multus/multus-d5lp9" Jul 23 21:51:40.109681 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.108971 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9d06a30f-ed03-457e-8318-7d8870ec4eb4-etc-modprobe-d\") pod \"tuned-qf8mt\" (UID: \"9d06a30f-ed03-457e-8318-7d8870ec4eb4\") " pod="openshift-cluster-node-tuning-operator/tuned-qf8mt" Jul 23 21:51:40.109681 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.108985 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9d06a30f-ed03-457e-8318-7d8870ec4eb4-sys\") pod \"tuned-qf8mt\" (UID: \"9d06a30f-ed03-457e-8318-7d8870ec4eb4\") " pod="openshift-cluster-node-tuning-operator/tuned-qf8mt" Jul 23 21:51:40.109681 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.109007 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9d06a30f-ed03-457e-8318-7d8870ec4eb4-host\") pod \"tuned-qf8mt\" (UID: \"9d06a30f-ed03-457e-8318-7d8870ec4eb4\") " pod="openshift-cluster-node-tuning-operator/tuned-qf8mt" Jul 23 21:51:40.110361 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.109028 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7342a9f6-72b1-4e3c-8d50-0b0326454a86-host-slash\") pod \"ovnkube-node-7jkjm\" (UID: \"7342a9f6-72b1-4e3c-8d50-0b0326454a86\") " pod="openshift-ovn-kubernetes/ovnkube-node-7jkjm" Jul 23 21:51:40.110361 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.109046 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1b8773cc-8521-4192-a3ab-0a397ba6eb62-host-run-k8s-cni-cncf-io\") pod \"multus-d5lp9\" (UID: \"1b8773cc-8521-4192-a3ab-0a397ba6eb62\") " pod="openshift-multus/multus-d5lp9" Jul 23 21:51:40.110361 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.109080 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-89d24\" (UniqueName: \"kubernetes.io/projected/4e42ce8b-422d-4281-b12e-6371c9744f8a-kube-api-access-89d24\") pod \"node-resolver-w6jzd\" (UID: \"4e42ce8b-422d-4281-b12e-6371c9744f8a\") " pod="openshift-dns/node-resolver-w6jzd" Jul 23 21:51:40.110361 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.109101 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cggcm\" (UniqueName: \"kubernetes.io/projected/1e27638c-41d9-441b-8cea-dd134bf51985-kube-api-access-cggcm\") pod \"multus-additional-cni-plugins-ntjs5\" (UID: \"1e27638c-41d9-441b-8cea-dd134bf51985\") " pod="openshift-multus/multus-additional-cni-plugins-ntjs5" Jul 23 21:51:40.116173 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.116147 2578 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 23 21:51:40.152903 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.152877 2578 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-qk5g2" Jul 23 21:51:40.163715 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.163690 2578 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-qk5g2" Jul 23 21:51:40.196398 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.196367 2578 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 23 21:51:40.209351 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.209324 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9d06a30f-ed03-457e-8318-7d8870ec4eb4-etc-tuned\") pod \"tuned-qf8mt\" (UID: \"9d06a30f-ed03-457e-8318-7d8870ec4eb4\") " pod="openshift-cluster-node-tuning-operator/tuned-qf8mt" Jul 23 21:51:40.209499 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.209361 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qhnlm\" (UniqueName: \"kubernetes.io/projected/1b8773cc-8521-4192-a3ab-0a397ba6eb62-kube-api-access-qhnlm\") pod \"multus-d5lp9\" (UID: \"1b8773cc-8521-4192-a3ab-0a397ba6eb62\") " pod="openshift-multus/multus-d5lp9" Jul 23 21:51:40.209499 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.209384 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7342a9f6-72b1-4e3c-8d50-0b0326454a86-var-lib-openvswitch\") pod \"ovnkube-node-7jkjm\" (UID: \"7342a9f6-72b1-4e3c-8d50-0b0326454a86\") " pod="openshift-ovn-kubernetes/ovnkube-node-7jkjm" Jul 23 21:51:40.209499 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.209414 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1b8773cc-8521-4192-a3ab-0a397ba6eb62-system-cni-dir\") pod \"multus-d5lp9\" (UID: \"1b8773cc-8521-4192-a3ab-0a397ba6eb62\") " pod="openshift-multus/multus-d5lp9" Jul 23 21:51:40.209499 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.209434 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6d542711-813a-4501-a243-b799ddd73340-etc-selinux\") pod \"aws-ebs-csi-driver-node-4pdct\" (UID: \"6d542711-813a-4501-a243-b799ddd73340\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4pdct" Jul 23 21:51:40.209499 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.209450 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9d06a30f-ed03-457e-8318-7d8870ec4eb4-lib-modules\") pod \"tuned-qf8mt\" (UID: \"9d06a30f-ed03-457e-8318-7d8870ec4eb4\") " pod="openshift-cluster-node-tuning-operator/tuned-qf8mt" Jul 23 21:51:40.209499 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.209479 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/93c54b15-009c-43ca-93a6-5a03909fc703-host-slash\") pod \"iptables-alerter-cmqcr\" (UID: \"93c54b15-009c-43ca-93a6-5a03909fc703\") " pod="openshift-network-operator/iptables-alerter-cmqcr" Jul 23 21:51:40.209781 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.209509 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/7342a9f6-72b1-4e3c-8d50-0b0326454a86-log-socket\") pod \"ovnkube-node-7jkjm\" (UID: \"7342a9f6-72b1-4e3c-8d50-0b0326454a86\") " pod="openshift-ovn-kubernetes/ovnkube-node-7jkjm" Jul 23 21:51:40.209781 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.209536 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/49757977-27cf-4177-b295-c830e4a71b03-host\") pod \"node-ca-nt77r\" (UID: \"49757977-27cf-4177-b295-c830e4a71b03\") " pod="openshift-image-registry/node-ca-nt77r" Jul 23 21:51:40.209781 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.209561 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7342a9f6-72b1-4e3c-8d50-0b0326454a86-host-run-ovn-kubernetes\") pod \"ovnkube-node-7jkjm\" (UID: \"7342a9f6-72b1-4e3c-8d50-0b0326454a86\") " pod="openshift-ovn-kubernetes/ovnkube-node-7jkjm" Jul 23 21:51:40.209781 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.209606 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/500a30f7-ac15-47da-adf9-d0485f80af93-dbus\") pod \"global-pull-secret-syncer-26shl\" (UID: \"500a30f7-ac15-47da-adf9-d0485f80af93\") " pod="kube-system/global-pull-secret-syncer-26shl" Jul 23 21:51:40.209781 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.209625 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1e27638c-41d9-441b-8cea-dd134bf51985-system-cni-dir\") pod \"multus-additional-cni-plugins-ntjs5\" (UID: \"1e27638c-41d9-441b-8cea-dd134bf51985\") " pod="openshift-multus/multus-additional-cni-plugins-ntjs5" Jul 23 21:51:40.209781 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.209642 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1e27638c-41d9-441b-8cea-dd134bf51985-os-release\") pod \"multus-additional-cni-plugins-ntjs5\" (UID: \"1e27638c-41d9-441b-8cea-dd134bf51985\") " pod="openshift-multus/multus-additional-cni-plugins-ntjs5" Jul 23 21:51:40.209781 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.209664 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9d06a30f-ed03-457e-8318-7d8870ec4eb4-tmp\") pod \"tuned-qf8mt\" (UID: \"9d06a30f-ed03-457e-8318-7d8870ec4eb4\") " pod="openshift-cluster-node-tuning-operator/tuned-qf8mt" Jul 23 21:51:40.209781 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.209687 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7342a9f6-72b1-4e3c-8d50-0b0326454a86-host-run-netns\") pod \"ovnkube-node-7jkjm\" (UID: \"7342a9f6-72b1-4e3c-8d50-0b0326454a86\") " pod="openshift-ovn-kubernetes/ovnkube-node-7jkjm" Jul 23 21:51:40.209781 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.209705 2578 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 23 21:51:40.209781 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.209743 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7342a9f6-72b1-4e3c-8d50-0b0326454a86-var-lib-openvswitch\") pod \"ovnkube-node-7jkjm\" (UID: \"7342a9f6-72b1-4e3c-8d50-0b0326454a86\") " pod="openshift-ovn-kubernetes/ovnkube-node-7jkjm" Jul 23 21:51:40.209781 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.209760 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7342a9f6-72b1-4e3c-8d50-0b0326454a86-etc-openvswitch\") pod \"ovnkube-node-7jkjm\" (UID: \"7342a9f6-72b1-4e3c-8d50-0b0326454a86\") " pod="openshift-ovn-kubernetes/ovnkube-node-7jkjm" Jul 23 21:51:40.209781 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.209745 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/49757977-27cf-4177-b295-c830e4a71b03-host\") pod \"node-ca-nt77r\" (UID: \"49757977-27cf-4177-b295-c830e4a71b03\") " pod="openshift-image-registry/node-ca-nt77r" Jul 23 21:51:40.210257 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.209787 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7342a9f6-72b1-4e3c-8d50-0b0326454a86-host-run-ovn-kubernetes\") pod \"ovnkube-node-7jkjm\" (UID: \"7342a9f6-72b1-4e3c-8d50-0b0326454a86\") " pod="openshift-ovn-kubernetes/ovnkube-node-7jkjm" Jul 23 21:51:40.210257 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.209815 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1e27638c-41d9-441b-8cea-dd134bf51985-system-cni-dir\") pod \"multus-additional-cni-plugins-ntjs5\" (UID: \"1e27638c-41d9-441b-8cea-dd134bf51985\") " pod="openshift-multus/multus-additional-cni-plugins-ntjs5" Jul 23 21:51:40.210257 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.209819 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1b8773cc-8521-4192-a3ab-0a397ba6eb62-system-cni-dir\") pod \"multus-d5lp9\" (UID: \"1b8773cc-8521-4192-a3ab-0a397ba6eb62\") " pod="openshift-multus/multus-d5lp9" Jul 23 21:51:40.210257 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.209714 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7342a9f6-72b1-4e3c-8d50-0b0326454a86-etc-openvswitch\") pod \"ovnkube-node-7jkjm\" (UID: \"7342a9f6-72b1-4e3c-8d50-0b0326454a86\") " pod="openshift-ovn-kubernetes/ovnkube-node-7jkjm" Jul 23 21:51:40.210257 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.209873 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7342a9f6-72b1-4e3c-8d50-0b0326454a86-host-run-netns\") pod \"ovnkube-node-7jkjm\" (UID: \"7342a9f6-72b1-4e3c-8d50-0b0326454a86\") " pod="openshift-ovn-kubernetes/ovnkube-node-7jkjm" Jul 23 21:51:40.210257 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.209886 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1b8773cc-8521-4192-a3ab-0a397ba6eb62-cni-binary-copy\") pod \"multus-d5lp9\" (UID: \"1b8773cc-8521-4192-a3ab-0a397ba6eb62\") " pod="openshift-multus/multus-d5lp9" Jul 23 21:51:40.210257 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.209910 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1b8773cc-8521-4192-a3ab-0a397ba6eb62-hostroot\") pod \"multus-d5lp9\" (UID: \"1b8773cc-8521-4192-a3ab-0a397ba6eb62\") " pod="openshift-multus/multus-d5lp9" Jul 23 21:51:40.210257 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.209935 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/7342a9f6-72b1-4e3c-8d50-0b0326454a86-log-socket\") pod \"ovnkube-node-7jkjm\" (UID: \"7342a9f6-72b1-4e3c-8d50-0b0326454a86\") " pod="openshift-ovn-kubernetes/ovnkube-node-7jkjm" Jul 23 21:51:40.210257 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.209936 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1b8773cc-8521-4192-a3ab-0a397ba6eb62-multus-daemon-config\") pod \"multus-d5lp9\" (UID: \"1b8773cc-8521-4192-a3ab-0a397ba6eb62\") " pod="openshift-multus/multus-d5lp9" Jul 23 21:51:40.210257 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.209966 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1e27638c-41d9-441b-8cea-dd134bf51985-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ntjs5\" (UID: \"1e27638c-41d9-441b-8cea-dd134bf51985\") " pod="openshift-multus/multus-additional-cni-plugins-ntjs5" Jul 23 21:51:40.210257 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.209989 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1b8773cc-8521-4192-a3ab-0a397ba6eb62-host-run-netns\") pod \"multus-d5lp9\" (UID: \"1b8773cc-8521-4192-a3ab-0a397ba6eb62\") " pod="openshift-multus/multus-d5lp9" Jul 23 21:51:40.210257 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.210006 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1e27638c-41d9-441b-8cea-dd134bf51985-os-release\") pod \"multus-additional-cni-plugins-ntjs5\" (UID: \"1e27638c-41d9-441b-8cea-dd134bf51985\") " pod="openshift-multus/multus-additional-cni-plugins-ntjs5" Jul 23 21:51:40.210257 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.210013 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1b8773cc-8521-4192-a3ab-0a397ba6eb62-host-run-multus-certs\") pod \"multus-d5lp9\" (UID: \"1b8773cc-8521-4192-a3ab-0a397ba6eb62\") " pod="openshift-multus/multus-d5lp9" Jul 23 21:51:40.210257 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.209910 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6d542711-813a-4501-a243-b799ddd73340-etc-selinux\") pod \"aws-ebs-csi-driver-node-4pdct\" (UID: \"6d542711-813a-4501-a243-b799ddd73340\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4pdct" Jul 23 21:51:40.210257 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.210042 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/500a30f7-ac15-47da-adf9-d0485f80af93-kubelet-config\") pod \"global-pull-secret-syncer-26shl\" (UID: \"500a30f7-ac15-47da-adf9-d0485f80af93\") " pod="kube-system/global-pull-secret-syncer-26shl" Jul 23 21:51:40.210257 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.210069 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9d06a30f-ed03-457e-8318-7d8870ec4eb4-etc-sysconfig\") pod \"tuned-qf8mt\" (UID: \"9d06a30f-ed03-457e-8318-7d8870ec4eb4\") " pod="openshift-cluster-node-tuning-operator/tuned-qf8mt" Jul 23 21:51:40.210257 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.210082 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1b8773cc-8521-4192-a3ab-0a397ba6eb62-hostroot\") pod \"multus-d5lp9\" (UID: \"1b8773cc-8521-4192-a3ab-0a397ba6eb62\") " pod="openshift-multus/multus-d5lp9" Jul 23 21:51:40.210257 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.210092 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9d06a30f-ed03-457e-8318-7d8870ec4eb4-etc-systemd\") pod \"tuned-qf8mt\" (UID: \"9d06a30f-ed03-457e-8318-7d8870ec4eb4\") " pod="openshift-cluster-node-tuning-operator/tuned-qf8mt" Jul 23 21:51:40.211008 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.210008 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9d06a30f-ed03-457e-8318-7d8870ec4eb4-lib-modules\") pod \"tuned-qf8mt\" (UID: \"9d06a30f-ed03-457e-8318-7d8870ec4eb4\") " pod="openshift-cluster-node-tuning-operator/tuned-qf8mt" Jul 23 21:51:40.211008 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.210044 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/93c54b15-009c-43ca-93a6-5a03909fc703-host-slash\") pod \"iptables-alerter-cmqcr\" (UID: \"93c54b15-009c-43ca-93a6-5a03909fc703\") " pod="openshift-network-operator/iptables-alerter-cmqcr" Jul 23 21:51:40.211008 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.210154 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1b8773cc-8521-4192-a3ab-0a397ba6eb62-host-run-multus-certs\") pod \"multus-d5lp9\" (UID: \"1b8773cc-8521-4192-a3ab-0a397ba6eb62\") " pod="openshift-multus/multus-d5lp9" Jul 23 21:51:40.211008 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.210117 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gwcfk\" (UniqueName: \"kubernetes.io/projected/9d06a30f-ed03-457e-8318-7d8870ec4eb4-kube-api-access-gwcfk\") pod \"tuned-qf8mt\" (UID: \"9d06a30f-ed03-457e-8318-7d8870ec4eb4\") " pod="openshift-cluster-node-tuning-operator/tuned-qf8mt" Jul 23 21:51:40.211008 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.210208 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e42ce8b-422d-4281-b12e-6371c9744f8a-tmp-dir\") pod \"node-resolver-w6jzd\" (UID: \"4e42ce8b-422d-4281-b12e-6371c9744f8a\") " pod="openshift-dns/node-resolver-w6jzd" Jul 23 21:51:40.211008 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.210237 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1b8773cc-8521-4192-a3ab-0a397ba6eb62-host-var-lib-kubelet\") pod \"multus-d5lp9\" (UID: \"1b8773cc-8521-4192-a3ab-0a397ba6eb62\") " pod="openshift-multus/multus-d5lp9" Jul 23 21:51:40.211008 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.210260 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1e27638c-41d9-441b-8cea-dd134bf51985-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ntjs5\" (UID: \"1e27638c-41d9-441b-8cea-dd134bf51985\") " pod="openshift-multus/multus-additional-cni-plugins-ntjs5" Jul 23 21:51:40.211008 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.210269 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9bnw9\" (UniqueName: \"kubernetes.io/projected/7c998f5f-33c9-44c6-abf7-f042f767fc4f-kube-api-access-9bnw9\") pod \"network-metrics-daemon-kl5st\" (UID: \"7c998f5f-33c9-44c6-abf7-f042f767fc4f\") " pod="openshift-multus/network-metrics-daemon-kl5st" Jul 23 21:51:40.211008 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.210309 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1e27638c-41d9-441b-8cea-dd134bf51985-cni-binary-copy\") pod \"multus-additional-cni-plugins-ntjs5\" (UID: \"1e27638c-41d9-441b-8cea-dd134bf51985\") " pod="openshift-multus/multus-additional-cni-plugins-ntjs5" Jul 23 21:51:40.211008 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.210336 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6d542711-813a-4501-a243-b799ddd73340-sys-fs\") pod \"aws-ebs-csi-driver-node-4pdct\" (UID: \"6d542711-813a-4501-a243-b799ddd73340\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4pdct" Jul 23 21:51:40.211008 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.210361 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f9skc\" (UniqueName: \"kubernetes.io/projected/93c54b15-009c-43ca-93a6-5a03909fc703-kube-api-access-f9skc\") pod \"iptables-alerter-cmqcr\" (UID: \"93c54b15-009c-43ca-93a6-5a03909fc703\") " pod="openshift-network-operator/iptables-alerter-cmqcr" Jul 23 21:51:40.211008 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.210386 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7342a9f6-72b1-4e3c-8d50-0b0326454a86-run-openvswitch\") pod \"ovnkube-node-7jkjm\" (UID: \"7342a9f6-72b1-4e3c-8d50-0b0326454a86\") " pod="openshift-ovn-kubernetes/ovnkube-node-7jkjm" Jul 23 21:51:40.211008 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.210415 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/9d06a30f-ed03-457e-8318-7d8870ec4eb4-etc-systemd\") pod \"tuned-qf8mt\" (UID: \"9d06a30f-ed03-457e-8318-7d8870ec4eb4\") " pod="openshift-cluster-node-tuning-operator/tuned-qf8mt" Jul 23 21:51:40.211008 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.210476 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6d542711-813a-4501-a243-b799ddd73340-sys-fs\") pod \"aws-ebs-csi-driver-node-4pdct\" (UID: \"6d542711-813a-4501-a243-b799ddd73340\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4pdct" Jul 23 21:51:40.211008 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.210491 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1b8773cc-8521-4192-a3ab-0a397ba6eb62-host-run-netns\") pod \"multus-d5lp9\" (UID: \"1b8773cc-8521-4192-a3ab-0a397ba6eb62\") " pod="openshift-multus/multus-d5lp9" Jul 23 21:51:40.211008 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.210522 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/9d06a30f-ed03-457e-8318-7d8870ec4eb4-etc-sysconfig\") pod \"tuned-qf8mt\" (UID: \"9d06a30f-ed03-457e-8318-7d8870ec4eb4\") " pod="openshift-cluster-node-tuning-operator/tuned-qf8mt" Jul 23 21:51:40.211008 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.210650 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1b8773cc-8521-4192-a3ab-0a397ba6eb62-cni-binary-copy\") pod \"multus-d5lp9\" (UID: \"1b8773cc-8521-4192-a3ab-0a397ba6eb62\") " pod="openshift-multus/multus-d5lp9" Jul 23 21:51:40.211800 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.210765 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1b8773cc-8521-4192-a3ab-0a397ba6eb62-host-var-lib-kubelet\") pod \"multus-d5lp9\" (UID: \"1b8773cc-8521-4192-a3ab-0a397ba6eb62\") " pod="openshift-multus/multus-d5lp9" Jul 23 21:51:40.211800 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.210776 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1b8773cc-8521-4192-a3ab-0a397ba6eb62-multus-daemon-config\") pod \"multus-d5lp9\" (UID: \"1b8773cc-8521-4192-a3ab-0a397ba6eb62\") " pod="openshift-multus/multus-d5lp9" Jul 23 21:51:40.211800 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.210830 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e42ce8b-422d-4281-b12e-6371c9744f8a-tmp-dir\") pod \"node-resolver-w6jzd\" (UID: \"4e42ce8b-422d-4281-b12e-6371c9744f8a\") " pod="openshift-dns/node-resolver-w6jzd" Jul 23 21:51:40.211800 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.210845 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/7342a9f6-72b1-4e3c-8d50-0b0326454a86-run-openvswitch\") pod \"ovnkube-node-7jkjm\" (UID: \"7342a9f6-72b1-4e3c-8d50-0b0326454a86\") " pod="openshift-ovn-kubernetes/ovnkube-node-7jkjm" Jul 23 21:51:40.211800 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.210409 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1b8773cc-8521-4192-a3ab-0a397ba6eb62-multus-cni-dir\") pod \"multus-d5lp9\" (UID: \"1b8773cc-8521-4192-a3ab-0a397ba6eb62\") " pod="openshift-multus/multus-d5lp9" Jul 23 21:51:40.211800 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.210944 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1b8773cc-8521-4192-a3ab-0a397ba6eb62-multus-socket-dir-parent\") pod \"multus-d5lp9\" (UID: \"1b8773cc-8521-4192-a3ab-0a397ba6eb62\") " pod="openshift-multus/multus-d5lp9" Jul 23 21:51:40.211800 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.210970 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-st5zb\" (UniqueName: \"kubernetes.io/projected/ef1bd359-5601-4a79-9b52-03d6e6a49bc6-kube-api-access-st5zb\") pod \"network-check-target-4cq9t\" (UID: \"ef1bd359-5601-4a79-9b52-03d6e6a49bc6\") " pod="openshift-network-diagnostics/network-check-target-4cq9t" Jul 23 21:51:40.211800 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.210989 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1e27638c-41d9-441b-8cea-dd134bf51985-cnibin\") pod \"multus-additional-cni-plugins-ntjs5\" (UID: \"1e27638c-41d9-441b-8cea-dd134bf51985\") " pod="openshift-multus/multus-additional-cni-plugins-ntjs5" Jul 23 21:51:40.211800 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.211005 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/1e27638c-41d9-441b-8cea-dd134bf51985-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ntjs5\" (UID: \"1e27638c-41d9-441b-8cea-dd134bf51985\") " pod="openshift-multus/multus-additional-cni-plugins-ntjs5" Jul 23 21:51:40.211800 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.211020 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9d06a30f-ed03-457e-8318-7d8870ec4eb4-etc-sysctl-d\") pod \"tuned-qf8mt\" (UID: \"9d06a30f-ed03-457e-8318-7d8870ec4eb4\") " pod="openshift-cluster-node-tuning-operator/tuned-qf8mt" Jul 23 21:51:40.211800 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.211035 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9d06a30f-ed03-457e-8318-7d8870ec4eb4-var-lib-kubelet\") pod \"tuned-qf8mt\" (UID: \"9d06a30f-ed03-457e-8318-7d8870ec4eb4\") " pod="openshift-cluster-node-tuning-operator/tuned-qf8mt" Jul 23 21:51:40.211800 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.211057 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zqqkd\" (UniqueName: \"kubernetes.io/projected/7342a9f6-72b1-4e3c-8d50-0b0326454a86-kube-api-access-zqqkd\") pod \"ovnkube-node-7jkjm\" (UID: \"7342a9f6-72b1-4e3c-8d50-0b0326454a86\") " pod="openshift-ovn-kubernetes/ovnkube-node-7jkjm" Jul 23 21:51:40.211800 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.211070 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1e27638c-41d9-441b-8cea-dd134bf51985-cnibin\") pod \"multus-additional-cni-plugins-ntjs5\" (UID: \"1e27638c-41d9-441b-8cea-dd134bf51985\") " pod="openshift-multus/multus-additional-cni-plugins-ntjs5" Jul 23 21:51:40.211800 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.211087 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1b8773cc-8521-4192-a3ab-0a397ba6eb62-multus-socket-dir-parent\") pod \"multus-d5lp9\" (UID: \"1b8773cc-8521-4192-a3ab-0a397ba6eb62\") " pod="openshift-multus/multus-d5lp9" Jul 23 21:51:40.211800 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.211103 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1e27638c-41d9-441b-8cea-dd134bf51985-cni-binary-copy\") pod \"multus-additional-cni-plugins-ntjs5\" (UID: \"1e27638c-41d9-441b-8cea-dd134bf51985\") " pod="openshift-multus/multus-additional-cni-plugins-ntjs5" Jul 23 21:51:40.211800 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.211109 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1b8773cc-8521-4192-a3ab-0a397ba6eb62-cnibin\") pod \"multus-d5lp9\" (UID: \"1b8773cc-8521-4192-a3ab-0a397ba6eb62\") " pod="openshift-multus/multus-d5lp9" Jul 23 21:51:40.211800 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.211107 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1b8773cc-8521-4192-a3ab-0a397ba6eb62-multus-cni-dir\") pod \"multus-d5lp9\" (UID: \"1b8773cc-8521-4192-a3ab-0a397ba6eb62\") " pod="openshift-multus/multus-d5lp9" Jul 23 21:51:40.212732 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.211147 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9d06a30f-ed03-457e-8318-7d8870ec4eb4-var-lib-kubelet\") pod \"tuned-qf8mt\" (UID: \"9d06a30f-ed03-457e-8318-7d8870ec4eb4\") " pod="openshift-cluster-node-tuning-operator/tuned-qf8mt" Jul 23 21:51:40.212732 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.211233 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1b8773cc-8521-4192-a3ab-0a397ba6eb62-cnibin\") pod \"multus-d5lp9\" (UID: \"1b8773cc-8521-4192-a3ab-0a397ba6eb62\") " pod="openshift-multus/multus-d5lp9" Jul 23 21:51:40.212732 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.211259 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/9d06a30f-ed03-457e-8318-7d8870ec4eb4-etc-sysctl-d\") pod \"tuned-qf8mt\" (UID: \"9d06a30f-ed03-457e-8318-7d8870ec4eb4\") " pod="openshift-cluster-node-tuning-operator/tuned-qf8mt" Jul 23 21:51:40.212732 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.211285 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1b8773cc-8521-4192-a3ab-0a397ba6eb62-host-var-lib-cni-multus\") pod \"multus-d5lp9\" (UID: \"1b8773cc-8521-4192-a3ab-0a397ba6eb62\") " pod="openshift-multus/multus-d5lp9" Jul 23 21:51:40.212732 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.211311 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1b8773cc-8521-4192-a3ab-0a397ba6eb62-multus-conf-dir\") pod \"multus-d5lp9\" (UID: \"1b8773cc-8521-4192-a3ab-0a397ba6eb62\") " pod="openshift-multus/multus-d5lp9" Jul 23 21:51:40.212732 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.211329 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1b8773cc-8521-4192-a3ab-0a397ba6eb62-host-var-lib-cni-multus\") pod \"multus-d5lp9\" (UID: \"1b8773cc-8521-4192-a3ab-0a397ba6eb62\") " pod="openshift-multus/multus-d5lp9" Jul 23 21:51:40.212732 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.211336 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/7342a9f6-72b1-4e3c-8d50-0b0326454a86-env-overrides\") pod \"ovnkube-node-7jkjm\" (UID: \"7342a9f6-72b1-4e3c-8d50-0b0326454a86\") " pod="openshift-ovn-kubernetes/ovnkube-node-7jkjm" Jul 23 21:51:40.212732 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.211359 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/1e27638c-41d9-441b-8cea-dd134bf51985-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ntjs5\" (UID: \"1e27638c-41d9-441b-8cea-dd134bf51985\") " pod="openshift-multus/multus-additional-cni-plugins-ntjs5" Jul 23 21:51:40.212732 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.211364 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1b8773cc-8521-4192-a3ab-0a397ba6eb62-multus-conf-dir\") pod \"multus-d5lp9\" (UID: \"1b8773cc-8521-4192-a3ab-0a397ba6eb62\") " pod="openshift-multus/multus-d5lp9" Jul 23 21:51:40.212732 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.211381 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8xsxf\" (UniqueName: \"kubernetes.io/projected/6d542711-813a-4501-a243-b799ddd73340-kube-api-access-8xsxf\") pod \"aws-ebs-csi-driver-node-4pdct\" (UID: \"6d542711-813a-4501-a243-b799ddd73340\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4pdct" Jul 23 21:51:40.212732 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.211398 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9d06a30f-ed03-457e-8318-7d8870ec4eb4-etc-sysctl-conf\") pod \"tuned-qf8mt\" (UID: \"9d06a30f-ed03-457e-8318-7d8870ec4eb4\") " pod="openshift-cluster-node-tuning-operator/tuned-qf8mt" Jul 23 21:51:40.212732 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.211412 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/7342a9f6-72b1-4e3c-8d50-0b0326454a86-host-kubelet\") pod \"ovnkube-node-7jkjm\" (UID: \"7342a9f6-72b1-4e3c-8d50-0b0326454a86\") " pod="openshift-ovn-kubernetes/ovnkube-node-7jkjm" Jul 23 21:51:40.212732 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.211427 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/7342a9f6-72b1-4e3c-8d50-0b0326454a86-node-log\") pod \"ovnkube-node-7jkjm\" (UID: \"7342a9f6-72b1-4e3c-8d50-0b0326454a86\") " pod="openshift-ovn-kubernetes/ovnkube-node-7jkjm" Jul 23 21:51:40.212732 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.211473 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1b8773cc-8521-4192-a3ab-0a397ba6eb62-host-var-lib-cni-bin\") pod \"multus-d5lp9\" (UID: \"1b8773cc-8521-4192-a3ab-0a397ba6eb62\") " pod="openshift-multus/multus-d5lp9" Jul 23 21:51:40.212732 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.211495 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9d06a30f-ed03-457e-8318-7d8870ec4eb4-etc-modprobe-d\") pod \"tuned-qf8mt\" (UID: \"9d06a30f-ed03-457e-8318-7d8870ec4eb4\") " pod="openshift-cluster-node-tuning-operator/tuned-qf8mt" Jul 23 21:51:40.212732 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.211515 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9d06a30f-ed03-457e-8318-7d8870ec4eb4-sys\") pod \"tuned-qf8mt\" (UID: \"9d06a30f-ed03-457e-8318-7d8870ec4eb4\") " pod="openshift-cluster-node-tuning-operator/tuned-qf8mt" Jul 23 21:51:40.212732 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.211539 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9d06a30f-ed03-457e-8318-7d8870ec4eb4-host\") pod \"tuned-qf8mt\" (UID: \"9d06a30f-ed03-457e-8318-7d8870ec4eb4\") " pod="openshift-cluster-node-tuning-operator/tuned-qf8mt" Jul 23 21:51:40.212732 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.211562 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7342a9f6-72b1-4e3c-8d50-0b0326454a86-host-slash\") pod \"ovnkube-node-7jkjm\" (UID: \"7342a9f6-72b1-4e3c-8d50-0b0326454a86\") " pod="openshift-ovn-kubernetes/ovnkube-node-7jkjm" Jul 23 21:51:40.213498 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.211563 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/9d06a30f-ed03-457e-8318-7d8870ec4eb4-etc-sysctl-conf\") pod \"tuned-qf8mt\" (UID: \"9d06a30f-ed03-457e-8318-7d8870ec4eb4\") " pod="openshift-cluster-node-tuning-operator/tuned-qf8mt" Jul 23 21:51:40.213498 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.211585 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1b8773cc-8521-4192-a3ab-0a397ba6eb62-host-run-k8s-cni-cncf-io\") pod \"multus-d5lp9\" (UID: \"1b8773cc-8521-4192-a3ab-0a397ba6eb62\") " pod="openshift-multus/multus-d5lp9" Jul 23 21:51:40.213498 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.211611 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-89d24\" (UniqueName: \"kubernetes.io/projected/4e42ce8b-422d-4281-b12e-6371c9744f8a-kube-api-access-89d24\") pod \"node-resolver-w6jzd\" (UID: \"4e42ce8b-422d-4281-b12e-6371c9744f8a\") " pod="openshift-dns/node-resolver-w6jzd" Jul 23 21:51:40.213498 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.211656 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d76a823d-b6d8-42d4-9060-32fe24a6ab73-konnectivity-ca\") pod \"konnectivity-agent-rpgtk\" (UID: \"d76a823d-b6d8-42d4-9060-32fe24a6ab73\") " pod="kube-system/konnectivity-agent-rpgtk" Jul 23 21:51:40.213498 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.211681 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cggcm\" (UniqueName: \"kubernetes.io/projected/1e27638c-41d9-441b-8cea-dd134bf51985-kube-api-access-cggcm\") pod \"multus-additional-cni-plugins-ntjs5\" (UID: \"1e27638c-41d9-441b-8cea-dd134bf51985\") " pod="openshift-multus/multus-additional-cni-plugins-ntjs5" Jul 23 21:51:40.213498 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.211706 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-crj5k\" (UniqueName: \"kubernetes.io/projected/49757977-27cf-4177-b295-c830e4a71b03-kube-api-access-crj5k\") pod \"node-ca-nt77r\" (UID: \"49757977-27cf-4177-b295-c830e4a71b03\") " pod="openshift-image-registry/node-ca-nt77r" Jul 23 21:51:40.213498 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.211729 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/7342a9f6-72b1-4e3c-8d50-0b0326454a86-run-ovn\") pod \"ovnkube-node-7jkjm\" (UID: \"7342a9f6-72b1-4e3c-8d50-0b0326454a86\") " pod="openshift-ovn-kubernetes/ovnkube-node-7jkjm" Jul 23 21:51:40.213498 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.211753 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7342a9f6-72b1-4e3c-8d50-0b0326454a86-host-cni-bin\") pod \"ovnkube-node-7jkjm\" (UID: \"7342a9f6-72b1-4e3c-8d50-0b0326454a86\") " pod="openshift-ovn-kubernetes/ovnkube-node-7jkjm" Jul 23 21:51:40.213498 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.211774 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4e42ce8b-422d-4281-b12e-6371c9744f8a-hosts-file\") pod \"node-resolver-w6jzd\" (UID: \"4e42ce8b-422d-4281-b12e-6371c9744f8a\") " pod="openshift-dns/node-resolver-w6jzd" Jul 23 21:51:40.213498 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.211793 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6d542711-813a-4501-a243-b799ddd73340-kubelet-dir\") pod \"aws-ebs-csi-driver-node-4pdct\" (UID: \"6d542711-813a-4501-a243-b799ddd73340\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4pdct" Jul 23 21:51:40.213498 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.211811 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/93c54b15-009c-43ca-93a6-5a03909fc703-iptables-alerter-script\") pod \"iptables-alerter-cmqcr\" (UID: \"93c54b15-009c-43ca-93a6-5a03909fc703\") " pod="openshift-network-operator/iptables-alerter-cmqcr" Jul 23 21:51:40.213498 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.211831 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/7342a9f6-72b1-4e3c-8d50-0b0326454a86-systemd-units\") pod \"ovnkube-node-7jkjm\" (UID: \"7342a9f6-72b1-4e3c-8d50-0b0326454a86\") " pod="openshift-ovn-kubernetes/ovnkube-node-7jkjm" Jul 23 21:51:40.213498 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.211845 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/1e27638c-41d9-441b-8cea-dd134bf51985-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ntjs5\" (UID: \"1e27638c-41d9-441b-8cea-dd134bf51985\") " pod="openshift-multus/multus-additional-cni-plugins-ntjs5" Jul 23 21:51:40.213498 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.211854 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7342a9f6-72b1-4e3c-8d50-0b0326454a86-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7jkjm\" (UID: \"7342a9f6-72b1-4e3c-8d50-0b0326454a86\") " pod="openshift-ovn-kubernetes/ovnkube-node-7jkjm" Jul 23 21:51:40.213498 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.211844 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/1e27638c-41d9-441b-8cea-dd134bf51985-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ntjs5\" (UID: \"1e27638c-41d9-441b-8cea-dd134bf51985\") " pod="openshift-multus/multus-additional-cni-plugins-ntjs5" Jul 23 21:51:40.213498 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.211874 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/7342a9f6-72b1-4e3c-8d50-0b0326454a86-ovnkube-script-lib\") pod \"ovnkube-node-7jkjm\" (UID: \"7342a9f6-72b1-4e3c-8d50-0b0326454a86\") " pod="openshift-ovn-kubernetes/ovnkube-node-7jkjm" Jul 23 21:51:40.213498 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.211880 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/7342a9f6-72b1-4e3c-8d50-0b0326454a86-env-overrides\") pod \"ovnkube-node-7jkjm\" (UID: \"7342a9f6-72b1-4e3c-8d50-0b0326454a86\") " pod="openshift-ovn-kubernetes/ovnkube-node-7jkjm" Jul 23 21:51:40.216043 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.211897 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/500a30f7-ac15-47da-adf9-d0485f80af93-original-pull-secret\") pod \"global-pull-secret-syncer-26shl\" (UID: \"500a30f7-ac15-47da-adf9-d0485f80af93\") " pod="kube-system/global-pull-secret-syncer-26shl" Jul 23 21:51:40.216043 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.211939 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/9d06a30f-ed03-457e-8318-7d8870ec4eb4-etc-modprobe-d\") pod \"tuned-qf8mt\" (UID: \"9d06a30f-ed03-457e-8318-7d8870ec4eb4\") " pod="openshift-cluster-node-tuning-operator/tuned-qf8mt" Jul 23 21:51:40.216043 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.211939 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/49757977-27cf-4177-b295-c830e4a71b03-serviceca\") pod \"node-ca-nt77r\" (UID: \"49757977-27cf-4177-b295-c830e4a71b03\") " pod="openshift-image-registry/node-ca-nt77r" Jul 23 21:51:40.216043 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.212000 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9d06a30f-ed03-457e-8318-7d8870ec4eb4-host\") pod \"tuned-qf8mt\" (UID: \"9d06a30f-ed03-457e-8318-7d8870ec4eb4\") " pod="openshift-cluster-node-tuning-operator/tuned-qf8mt" Jul 23 21:51:40.216043 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.212001 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6d542711-813a-4501-a243-b799ddd73340-socket-dir\") pod \"aws-ebs-csi-driver-node-4pdct\" (UID: \"6d542711-813a-4501-a243-b799ddd73340\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4pdct" Jul 23 21:51:40.216043 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.212009 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1b8773cc-8521-4192-a3ab-0a397ba6eb62-host-run-k8s-cni-cncf-io\") pod \"multus-d5lp9\" (UID: \"1b8773cc-8521-4192-a3ab-0a397ba6eb62\") " pod="openshift-multus/multus-d5lp9" Jul 23 21:51:40.216043 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.212063 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1b8773cc-8521-4192-a3ab-0a397ba6eb62-host-var-lib-cni-bin\") pod \"multus-d5lp9\" (UID: \"1b8773cc-8521-4192-a3ab-0a397ba6eb62\") " pod="openshift-multus/multus-d5lp9" Jul 23 21:51:40.216043 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.212030 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9d06a30f-ed03-457e-8318-7d8870ec4eb4-etc-kubernetes\") pod \"tuned-qf8mt\" (UID: \"9d06a30f-ed03-457e-8318-7d8870ec4eb4\") " pod="openshift-cluster-node-tuning-operator/tuned-qf8mt" Jul 23 21:51:40.216043 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.212088 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7342a9f6-72b1-4e3c-8d50-0b0326454a86-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7jkjm\" (UID: \"7342a9f6-72b1-4e3c-8d50-0b0326454a86\") " pod="openshift-ovn-kubernetes/ovnkube-node-7jkjm" Jul 23 21:51:40.216043 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.212106 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1b8773cc-8521-4192-a3ab-0a397ba6eb62-os-release\") pod \"multus-d5lp9\" (UID: \"1b8773cc-8521-4192-a3ab-0a397ba6eb62\") " pod="openshift-multus/multus-d5lp9" Jul 23 21:51:40.216043 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.212135 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1b8773cc-8521-4192-a3ab-0a397ba6eb62-etc-kubernetes\") pod \"multus-d5lp9\" (UID: \"1b8773cc-8521-4192-a3ab-0a397ba6eb62\") " pod="openshift-multus/multus-d5lp9" Jul 23 21:51:40.216043 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.212118 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9d06a30f-ed03-457e-8318-7d8870ec4eb4-etc-kubernetes\") pod \"tuned-qf8mt\" (UID: \"9d06a30f-ed03-457e-8318-7d8870ec4eb4\") " pod="openshift-cluster-node-tuning-operator/tuned-qf8mt" Jul 23 21:51:40.216043 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.212194 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9d06a30f-ed03-457e-8318-7d8870ec4eb4-sys\") pod \"tuned-qf8mt\" (UID: \"9d06a30f-ed03-457e-8318-7d8870ec4eb4\") " pod="openshift-cluster-node-tuning-operator/tuned-qf8mt" Jul 23 21:51:40.216043 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.212209 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7342a9f6-72b1-4e3c-8d50-0b0326454a86-host-cni-bin\") pod \"ovnkube-node-7jkjm\" (UID: \"7342a9f6-72b1-4e3c-8d50-0b0326454a86\") " pod="openshift-ovn-kubernetes/ovnkube-node-7jkjm" Jul 23 21:51:40.216043 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.212215 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/7342a9f6-72b1-4e3c-8d50-0b0326454a86-host-kubelet\") pod \"ovnkube-node-7jkjm\" (UID: \"7342a9f6-72b1-4e3c-8d50-0b0326454a86\") " pod="openshift-ovn-kubernetes/ovnkube-node-7jkjm" Jul 23 21:51:40.216043 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.212232 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/7342a9f6-72b1-4e3c-8d50-0b0326454a86-run-ovn\") pod \"ovnkube-node-7jkjm\" (UID: \"7342a9f6-72b1-4e3c-8d50-0b0326454a86\") " pod="openshift-ovn-kubernetes/ovnkube-node-7jkjm" Jul 23 21:51:40.216043 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.212245 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/7342a9f6-72b1-4e3c-8d50-0b0326454a86-node-log\") pod \"ovnkube-node-7jkjm\" (UID: \"7342a9f6-72b1-4e3c-8d50-0b0326454a86\") " pod="openshift-ovn-kubernetes/ovnkube-node-7jkjm" Jul 23 21:51:40.216043 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.212265 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/7342a9f6-72b1-4e3c-8d50-0b0326454a86-systemd-units\") pod \"ovnkube-node-7jkjm\" (UID: \"7342a9f6-72b1-4e3c-8d50-0b0326454a86\") " pod="openshift-ovn-kubernetes/ovnkube-node-7jkjm" Jul 23 21:51:40.216762 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.212279 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9d06a30f-ed03-457e-8318-7d8870ec4eb4-run\") pod \"tuned-qf8mt\" (UID: \"9d06a30f-ed03-457e-8318-7d8870ec4eb4\") " pod="openshift-cluster-node-tuning-operator/tuned-qf8mt" Jul 23 21:51:40.216762 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.212316 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6d542711-813a-4501-a243-b799ddd73340-socket-dir\") pod \"aws-ebs-csi-driver-node-4pdct\" (UID: \"6d542711-813a-4501-a243-b799ddd73340\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4pdct" Jul 23 21:51:40.216762 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.212317 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1b8773cc-8521-4192-a3ab-0a397ba6eb62-os-release\") pod \"multus-d5lp9\" (UID: \"1b8773cc-8521-4192-a3ab-0a397ba6eb62\") " pod="openshift-multus/multus-d5lp9" Jul 23 21:51:40.216762 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.212216 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/4e42ce8b-422d-4281-b12e-6371c9744f8a-hosts-file\") pod \"node-resolver-w6jzd\" (UID: \"4e42ce8b-422d-4281-b12e-6371c9744f8a\") " pod="openshift-dns/node-resolver-w6jzd" Jul 23 21:51:40.216762 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.212372 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1b8773cc-8521-4192-a3ab-0a397ba6eb62-etc-kubernetes\") pod \"multus-d5lp9\" (UID: \"1b8773cc-8521-4192-a3ab-0a397ba6eb62\") " pod="openshift-multus/multus-d5lp9" Jul 23 21:51:40.216762 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.212394 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/7342a9f6-72b1-4e3c-8d50-0b0326454a86-run-systemd\") pod \"ovnkube-node-7jkjm\" (UID: \"7342a9f6-72b1-4e3c-8d50-0b0326454a86\") " pod="openshift-ovn-kubernetes/ovnkube-node-7jkjm" Jul 23 21:51:40.216762 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.212420 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/7342a9f6-72b1-4e3c-8d50-0b0326454a86-host-cni-netd\") pod \"ovnkube-node-7jkjm\" (UID: \"7342a9f6-72b1-4e3c-8d50-0b0326454a86\") " pod="openshift-ovn-kubernetes/ovnkube-node-7jkjm" Jul 23 21:51:40.216762 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.212430 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7342a9f6-72b1-4e3c-8d50-0b0326454a86-host-slash\") pod \"ovnkube-node-7jkjm\" (UID: \"7342a9f6-72b1-4e3c-8d50-0b0326454a86\") " pod="openshift-ovn-kubernetes/ovnkube-node-7jkjm" Jul 23 21:51:40.216762 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.212447 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/7342a9f6-72b1-4e3c-8d50-0b0326454a86-ovnkube-config\") pod \"ovnkube-node-7jkjm\" (UID: \"7342a9f6-72b1-4e3c-8d50-0b0326454a86\") " pod="openshift-ovn-kubernetes/ovnkube-node-7jkjm" Jul 23 21:51:40.216762 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.212492 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/9d06a30f-ed03-457e-8318-7d8870ec4eb4-run\") pod \"tuned-qf8mt\" (UID: \"9d06a30f-ed03-457e-8318-7d8870ec4eb4\") " pod="openshift-cluster-node-tuning-operator/tuned-qf8mt" Jul 23 21:51:40.216762 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.212501 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/93c54b15-009c-43ca-93a6-5a03909fc703-iptables-alerter-script\") pod \"iptables-alerter-cmqcr\" (UID: \"93c54b15-009c-43ca-93a6-5a03909fc703\") " pod="openshift-network-operator/iptables-alerter-cmqcr" Jul 23 21:51:40.216762 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.212516 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/7342a9f6-72b1-4e3c-8d50-0b0326454a86-ovn-node-metrics-cert\") pod \"ovnkube-node-7jkjm\" (UID: \"7342a9f6-72b1-4e3c-8d50-0b0326454a86\") " pod="openshift-ovn-kubernetes/ovnkube-node-7jkjm" Jul 23 21:51:40.216762 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.212557 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6d542711-813a-4501-a243-b799ddd73340-kubelet-dir\") pod \"aws-ebs-csi-driver-node-4pdct\" (UID: \"6d542711-813a-4501-a243-b799ddd73340\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4pdct" Jul 23 21:51:40.216762 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.212543 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c998f5f-33c9-44c6-abf7-f042f767fc4f-metrics-certs\") pod \"network-metrics-daemon-kl5st\" (UID: \"7c998f5f-33c9-44c6-abf7-f042f767fc4f\") " pod="openshift-multus/network-metrics-daemon-kl5st" Jul 23 21:51:40.216762 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.212596 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d76a823d-b6d8-42d4-9060-32fe24a6ab73-agent-certs\") pod \"konnectivity-agent-rpgtk\" (UID: \"d76a823d-b6d8-42d4-9060-32fe24a6ab73\") " pod="kube-system/konnectivity-agent-rpgtk" Jul 23 21:51:40.216762 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.212632 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6d542711-813a-4501-a243-b799ddd73340-registration-dir\") pod \"aws-ebs-csi-driver-node-4pdct\" (UID: \"6d542711-813a-4501-a243-b799ddd73340\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4pdct" Jul 23 21:51:40.216762 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:51:40.212646 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 21:51:40.216762 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.212655 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6d542711-813a-4501-a243-b799ddd73340-device-dir\") pod \"aws-ebs-csi-driver-node-4pdct\" (UID: \"6d542711-813a-4501-a243-b799ddd73340\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4pdct" Jul 23 21:51:40.217588 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.212681 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/7342a9f6-72b1-4e3c-8d50-0b0326454a86-run-systemd\") pod \"ovnkube-node-7jkjm\" (UID: \"7342a9f6-72b1-4e3c-8d50-0b0326454a86\") " pod="openshift-ovn-kubernetes/ovnkube-node-7jkjm" Jul 23 21:51:40.217588 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.212697 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/49757977-27cf-4177-b295-c830e4a71b03-serviceca\") pod \"node-ca-nt77r\" (UID: \"49757977-27cf-4177-b295-c830e4a71b03\") " pod="openshift-image-registry/node-ca-nt77r" Jul 23 21:51:40.217588 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:51:40.212713 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c998f5f-33c9-44c6-abf7-f042f767fc4f-metrics-certs podName:7c998f5f-33c9-44c6-abf7-f042f767fc4f nodeName:}" failed. No retries permitted until 2026-07-23 21:51:40.712696262 +0000 UTC m=+2.954604117 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7c998f5f-33c9-44c6-abf7-f042f767fc4f-metrics-certs") pod "network-metrics-daemon-kl5st" (UID: "7c998f5f-33c9-44c6-abf7-f042f767fc4f") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 21:51:40.217588 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.212784 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6d542711-813a-4501-a243-b799ddd73340-device-dir\") pod \"aws-ebs-csi-driver-node-4pdct\" (UID: \"6d542711-813a-4501-a243-b799ddd73340\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4pdct" Jul 23 21:51:40.217588 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.212830 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/7342a9f6-72b1-4e3c-8d50-0b0326454a86-host-cni-netd\") pod \"ovnkube-node-7jkjm\" (UID: \"7342a9f6-72b1-4e3c-8d50-0b0326454a86\") " pod="openshift-ovn-kubernetes/ovnkube-node-7jkjm" Jul 23 21:51:40.217588 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.212855 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/7342a9f6-72b1-4e3c-8d50-0b0326454a86-ovnkube-script-lib\") pod \"ovnkube-node-7jkjm\" (UID: \"7342a9f6-72b1-4e3c-8d50-0b0326454a86\") " pod="openshift-ovn-kubernetes/ovnkube-node-7jkjm" Jul 23 21:51:40.217588 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.212858 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6d542711-813a-4501-a243-b799ddd73340-registration-dir\") pod \"aws-ebs-csi-driver-node-4pdct\" (UID: \"6d542711-813a-4501-a243-b799ddd73340\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4pdct" Jul 23 21:51:40.217588 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.212933 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/7342a9f6-72b1-4e3c-8d50-0b0326454a86-ovnkube-config\") pod \"ovnkube-node-7jkjm\" (UID: \"7342a9f6-72b1-4e3c-8d50-0b0326454a86\") " pod="openshift-ovn-kubernetes/ovnkube-node-7jkjm" Jul 23 21:51:40.217588 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.214519 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9d06a30f-ed03-457e-8318-7d8870ec4eb4-tmp\") pod \"tuned-qf8mt\" (UID: \"9d06a30f-ed03-457e-8318-7d8870ec4eb4\") " pod="openshift-cluster-node-tuning-operator/tuned-qf8mt" Jul 23 21:51:40.217588 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.215412 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/7342a9f6-72b1-4e3c-8d50-0b0326454a86-ovn-node-metrics-cert\") pod \"ovnkube-node-7jkjm\" (UID: \"7342a9f6-72b1-4e3c-8d50-0b0326454a86\") " pod="openshift-ovn-kubernetes/ovnkube-node-7jkjm" Jul 23 21:51:40.217588 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.215498 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/9d06a30f-ed03-457e-8318-7d8870ec4eb4-etc-tuned\") pod \"tuned-qf8mt\" (UID: \"9d06a30f-ed03-457e-8318-7d8870ec4eb4\") " pod="openshift-cluster-node-tuning-operator/tuned-qf8mt" Jul 23 21:51:40.220279 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.220061 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9bnw9\" (UniqueName: \"kubernetes.io/projected/7c998f5f-33c9-44c6-abf7-f042f767fc4f-kube-api-access-9bnw9\") pod \"network-metrics-daemon-kl5st\" (UID: \"7c998f5f-33c9-44c6-abf7-f042f767fc4f\") " pod="openshift-multus/network-metrics-daemon-kl5st" Jul 23 21:51:40.220279 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.220480 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qhnlm\" (UniqueName: \"kubernetes.io/projected/1b8773cc-8521-4192-a3ab-0a397ba6eb62-kube-api-access-qhnlm\") pod \"multus-d5lp9\" (UID: \"1b8773cc-8521-4192-a3ab-0a397ba6eb62\") " pod="openshift-multus/multus-d5lp9" Jul 23 21:51:40.220279 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.220640 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gwcfk\" (UniqueName: \"kubernetes.io/projected/9d06a30f-ed03-457e-8318-7d8870ec4eb4-kube-api-access-gwcfk\") pod \"tuned-qf8mt\" (UID: \"9d06a30f-ed03-457e-8318-7d8870ec4eb4\") " pod="openshift-cluster-node-tuning-operator/tuned-qf8mt" Jul 23 21:51:40.221461 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.221370 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f9skc\" (UniqueName: \"kubernetes.io/projected/93c54b15-009c-43ca-93a6-5a03909fc703-kube-api-access-f9skc\") pod \"iptables-alerter-cmqcr\" (UID: \"93c54b15-009c-43ca-93a6-5a03909fc703\") " pod="openshift-network-operator/iptables-alerter-cmqcr" Jul 23 21:51:40.224166 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.224123 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-89d24\" (UniqueName: \"kubernetes.io/projected/4e42ce8b-422d-4281-b12e-6371c9744f8a-kube-api-access-89d24\") pod \"node-resolver-w6jzd\" (UID: \"4e42ce8b-422d-4281-b12e-6371c9744f8a\") " pod="openshift-dns/node-resolver-w6jzd" Jul 23 21:51:40.226506 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.226460 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zqqkd\" (UniqueName: \"kubernetes.io/projected/7342a9f6-72b1-4e3c-8d50-0b0326454a86-kube-api-access-zqqkd\") pod \"ovnkube-node-7jkjm\" (UID: \"7342a9f6-72b1-4e3c-8d50-0b0326454a86\") " pod="openshift-ovn-kubernetes/ovnkube-node-7jkjm" Jul 23 21:51:40.226826 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.226773 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8xsxf\" (UniqueName: \"kubernetes.io/projected/6d542711-813a-4501-a243-b799ddd73340-kube-api-access-8xsxf\") pod \"aws-ebs-csi-driver-node-4pdct\" (UID: \"6d542711-813a-4501-a243-b799ddd73340\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4pdct" Jul 23 21:51:40.227176 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.227128 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-crj5k\" (UniqueName: \"kubernetes.io/projected/49757977-27cf-4177-b295-c830e4a71b03-kube-api-access-crj5k\") pod \"node-ca-nt77r\" (UID: \"49757977-27cf-4177-b295-c830e4a71b03\") " pod="openshift-image-registry/node-ca-nt77r" Jul 23 21:51:40.228462 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.228436 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cggcm\" (UniqueName: \"kubernetes.io/projected/1e27638c-41d9-441b-8cea-dd134bf51985-kube-api-access-cggcm\") pod \"multus-additional-cni-plugins-ntjs5\" (UID: \"1e27638c-41d9-441b-8cea-dd134bf51985\") " pod="openshift-multus/multus-additional-cni-plugins-ntjs5" Jul 23 21:51:40.313059 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.313023 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/500a30f7-ac15-47da-adf9-d0485f80af93-kubelet-config\") pod \"global-pull-secret-syncer-26shl\" (UID: \"500a30f7-ac15-47da-adf9-d0485f80af93\") " pod="kube-system/global-pull-secret-syncer-26shl" Jul 23 21:51:40.313265 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.313074 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-st5zb\" (UniqueName: \"kubernetes.io/projected/ef1bd359-5601-4a79-9b52-03d6e6a49bc6-kube-api-access-st5zb\") pod \"network-check-target-4cq9t\" (UID: \"ef1bd359-5601-4a79-9b52-03d6e6a49bc6\") " pod="openshift-network-diagnostics/network-check-target-4cq9t" Jul 23 21:51:40.313265 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.313117 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d76a823d-b6d8-42d4-9060-32fe24a6ab73-konnectivity-ca\") pod \"konnectivity-agent-rpgtk\" (UID: \"d76a823d-b6d8-42d4-9060-32fe24a6ab73\") " pod="kube-system/konnectivity-agent-rpgtk" Jul 23 21:51:40.313265 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.313147 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/500a30f7-ac15-47da-adf9-d0485f80af93-original-pull-secret\") pod \"global-pull-secret-syncer-26shl\" (UID: \"500a30f7-ac15-47da-adf9-d0485f80af93\") " pod="kube-system/global-pull-secret-syncer-26shl" Jul 23 21:51:40.313265 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.313166 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/500a30f7-ac15-47da-adf9-d0485f80af93-kubelet-config\") pod \"global-pull-secret-syncer-26shl\" (UID: \"500a30f7-ac15-47da-adf9-d0485f80af93\") " pod="kube-system/global-pull-secret-syncer-26shl" Jul 23 21:51:40.313265 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.313194 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d76a823d-b6d8-42d4-9060-32fe24a6ab73-agent-certs\") pod \"konnectivity-agent-rpgtk\" (UID: \"d76a823d-b6d8-42d4-9060-32fe24a6ab73\") " pod="kube-system/konnectivity-agent-rpgtk" Jul 23 21:51:40.313265 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.313231 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/500a30f7-ac15-47da-adf9-d0485f80af93-dbus\") pod \"global-pull-secret-syncer-26shl\" (UID: \"500a30f7-ac15-47da-adf9-d0485f80af93\") " pod="kube-system/global-pull-secret-syncer-26shl" Jul 23 21:51:40.313535 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:51:40.313312 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 23 21:51:40.313535 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:51:40.313380 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/500a30f7-ac15-47da-adf9-d0485f80af93-original-pull-secret podName:500a30f7-ac15-47da-adf9-d0485f80af93 nodeName:}" failed. No retries permitted until 2026-07-23 21:51:40.813359992 +0000 UTC m=+3.055267845 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/500a30f7-ac15-47da-adf9-d0485f80af93-original-pull-secret") pod "global-pull-secret-syncer-26shl" (UID: "500a30f7-ac15-47da-adf9-d0485f80af93") : object "kube-system"/"original-pull-secret" not registered Jul 23 21:51:40.313535 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.313403 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/500a30f7-ac15-47da-adf9-d0485f80af93-dbus\") pod \"global-pull-secret-syncer-26shl\" (UID: \"500a30f7-ac15-47da-adf9-d0485f80af93\") " pod="kube-system/global-pull-secret-syncer-26shl" Jul 23 21:51:40.313818 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.313797 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d76a823d-b6d8-42d4-9060-32fe24a6ab73-konnectivity-ca\") pod \"konnectivity-agent-rpgtk\" (UID: \"d76a823d-b6d8-42d4-9060-32fe24a6ab73\") " pod="kube-system/konnectivity-agent-rpgtk" Jul 23 21:51:40.316198 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.316173 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d76a823d-b6d8-42d4-9060-32fe24a6ab73-agent-certs\") pod \"konnectivity-agent-rpgtk\" (UID: \"d76a823d-b6d8-42d4-9060-32fe24a6ab73\") " pod="kube-system/konnectivity-agent-rpgtk" Jul 23 21:51:40.322478 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:51:40.322454 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 23 21:51:40.322591 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:51:40.322485 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 23 21:51:40.322591 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:51:40.322502 2578 projected.go:196] Error preparing data for projected volume kube-api-access-st5zb for pod openshift-network-diagnostics/network-check-target-4cq9t: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 21:51:40.322682 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:51:40.322594 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ef1bd359-5601-4a79-9b52-03d6e6a49bc6-kube-api-access-st5zb podName:ef1bd359-5601-4a79-9b52-03d6e6a49bc6 nodeName:}" failed. No retries permitted until 2026-07-23 21:51:40.822561645 +0000 UTC m=+3.064469517 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-st5zb" (UniqueName: "kubernetes.io/projected/ef1bd359-5601-4a79-9b52-03d6e6a49bc6-kube-api-access-st5zb") pod "network-check-target-4cq9t" (UID: "ef1bd359-5601-4a79-9b52-03d6e6a49bc6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 21:51:40.396889 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.396810 2578 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 23 21:51:40.397901 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.397879 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-ntjs5" Jul 23 21:51:40.407183 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.407163 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-nt77r" Jul 23 21:51:40.415449 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.415428 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-d5lp9" Jul 23 21:51:40.421450 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.421426 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4pdct" Jul 23 21:51:40.429162 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.429144 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-qf8mt" Jul 23 21:51:40.435093 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.435072 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-w6jzd" Jul 23 21:51:40.442231 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.442212 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-cmqcr" Jul 23 21:51:40.450175 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.450155 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-7jkjm" Jul 23 21:51:40.456117 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.456095 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-rpgtk" Jul 23 21:51:40.716327 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.716250 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c998f5f-33c9-44c6-abf7-f042f767fc4f-metrics-certs\") pod \"network-metrics-daemon-kl5st\" (UID: \"7c998f5f-33c9-44c6-abf7-f042f767fc4f\") " pod="openshift-multus/network-metrics-daemon-kl5st" Jul 23 21:51:40.716488 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:51:40.716387 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 21:51:40.716488 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:51:40.716457 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c998f5f-33c9-44c6-abf7-f042f767fc4f-metrics-certs podName:7c998f5f-33c9-44c6-abf7-f042f767fc4f nodeName:}" failed. No retries permitted until 2026-07-23 21:51:41.716436073 +0000 UTC m=+3.958343929 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7c998f5f-33c9-44c6-abf7-f042f767fc4f-metrics-certs") pod "network-metrics-daemon-kl5st" (UID: "7c998f5f-33c9-44c6-abf7-f042f767fc4f") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 21:51:40.816805 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.816765 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/500a30f7-ac15-47da-adf9-d0485f80af93-original-pull-secret\") pod \"global-pull-secret-syncer-26shl\" (UID: \"500a30f7-ac15-47da-adf9-d0485f80af93\") " pod="kube-system/global-pull-secret-syncer-26shl" Jul 23 21:51:40.817002 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:51:40.816884 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 23 21:51:40.817002 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:51:40.816965 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/500a30f7-ac15-47da-adf9-d0485f80af93-original-pull-secret podName:500a30f7-ac15-47da-adf9-d0485f80af93 nodeName:}" failed. No retries permitted until 2026-07-23 21:51:41.816935578 +0000 UTC m=+4.058843434 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/500a30f7-ac15-47da-adf9-d0485f80af93-original-pull-secret") pod "global-pull-secret-syncer-26shl" (UID: "500a30f7-ac15-47da-adf9-d0485f80af93") : object "kube-system"/"original-pull-secret" not registered Jul 23 21:51:40.917511 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:40.917470 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-st5zb\" (UniqueName: \"kubernetes.io/projected/ef1bd359-5601-4a79-9b52-03d6e6a49bc6-kube-api-access-st5zb\") pod \"network-check-target-4cq9t\" (UID: \"ef1bd359-5601-4a79-9b52-03d6e6a49bc6\") " pod="openshift-network-diagnostics/network-check-target-4cq9t" Jul 23 21:51:40.917700 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:51:40.917643 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 23 21:51:40.917700 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:51:40.917661 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 23 21:51:40.917700 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:51:40.917675 2578 projected.go:196] Error preparing data for projected volume kube-api-access-st5zb for pod openshift-network-diagnostics/network-check-target-4cq9t: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 21:51:40.917844 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:51:40.917749 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ef1bd359-5601-4a79-9b52-03d6e6a49bc6-kube-api-access-st5zb podName:ef1bd359-5601-4a79-9b52-03d6e6a49bc6 nodeName:}" failed. No retries permitted until 2026-07-23 21:51:41.917728172 +0000 UTC m=+4.159636029 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-st5zb" (UniqueName: "kubernetes.io/projected/ef1bd359-5601-4a79-9b52-03d6e6a49bc6-kube-api-access-st5zb") pod "network-check-target-4cq9t" (UID: "ef1bd359-5601-4a79-9b52-03d6e6a49bc6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 21:51:41.063655 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:41.063603 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e42ce8b_422d_4281_b12e_6371c9744f8a.slice/crio-cb4bb6c2d55721271d0d350da0411b381bc14c205d31293d8e6fe8a916ce4e8a WatchSource:0}: Error finding container cb4bb6c2d55721271d0d350da0411b381bc14c205d31293d8e6fe8a916ce4e8a: Status 404 returned error can't find the container with id cb4bb6c2d55721271d0d350da0411b381bc14c205d31293d8e6fe8a916ce4e8a Jul 23 21:51:41.064347 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:41.064318 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e27638c_41d9_441b_8cea_dd134bf51985.slice/crio-61b48ec3441af05a03a1700d6ce7dac575f812349a20e5c76846d05a470caf67 WatchSource:0}: Error finding container 61b48ec3441af05a03a1700d6ce7dac575f812349a20e5c76846d05a470caf67: Status 404 returned error can't find the container with id 61b48ec3441af05a03a1700d6ce7dac575f812349a20e5c76846d05a470caf67 Jul 23 21:51:41.066250 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:41.066229 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49757977_27cf_4177_b295_c830e4a71b03.slice/crio-a3b9d9ca978cd699d2400ff9de141b2c6257f00cc2f9d610cd1082ab8fcd6804 WatchSource:0}: Error finding container a3b9d9ca978cd699d2400ff9de141b2c6257f00cc2f9d610cd1082ab8fcd6804: Status 404 returned error can't find the container with id a3b9d9ca978cd699d2400ff9de141b2c6257f00cc2f9d610cd1082ab8fcd6804 Jul 23 21:51:41.067412 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:41.067386 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d06a30f_ed03_457e_8318_7d8870ec4eb4.slice/crio-0e87f82db486018381e0eebf057bc36d7d2eb05528e1cec27bdc98e4d5e2dc48 WatchSource:0}: Error finding container 0e87f82db486018381e0eebf057bc36d7d2eb05528e1cec27bdc98e4d5e2dc48: Status 404 returned error can't find the container with id 0e87f82db486018381e0eebf057bc36d7d2eb05528e1cec27bdc98e4d5e2dc48 Jul 23 21:51:41.068300 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:41.068199 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7342a9f6_72b1_4e3c_8d50_0b0326454a86.slice/crio-cbea9238011cf2cb1b71399be8b353c624d4c92428bfbfe6282a76fb19c6db3b WatchSource:0}: Error finding container cbea9238011cf2cb1b71399be8b353c624d4c92428bfbfe6282a76fb19c6db3b: Status 404 returned error can't find the container with id cbea9238011cf2cb1b71399be8b353c624d4c92428bfbfe6282a76fb19c6db3b Jul 23 21:51:41.070339 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:41.070312 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd76a823d_b6d8_42d4_9060_32fe24a6ab73.slice/crio-d56f362f9a008e6f70adc1b2eb974937ad8f06f08a808958097f2fbb47a11a15 WatchSource:0}: Error finding container d56f362f9a008e6f70adc1b2eb974937ad8f06f08a808958097f2fbb47a11a15: Status 404 returned error can't find the container with id d56f362f9a008e6f70adc1b2eb974937ad8f06f08a808958097f2fbb47a11a15 Jul 23 21:51:41.071649 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:41.071617 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b8773cc_8521_4192_a3ab_0a397ba6eb62.slice/crio-8e648b7dc6c07866d9415777f9a8eed0ce286d1aedc6fc9e0d57dc8c87ebc00c WatchSource:0}: Error finding container 8e648b7dc6c07866d9415777f9a8eed0ce286d1aedc6fc9e0d57dc8c87ebc00c: Status 404 returned error can't find the container with id 8e648b7dc6c07866d9415777f9a8eed0ce286d1aedc6fc9e0d57dc8c87ebc00c Jul 23 21:51:41.075211 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:41.074959 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93c54b15_009c_43ca_93a6_5a03909fc703.slice/crio-ece3b97415629ab3b70c01504b9e0223aed78910c7546742587b4f3fee1a085b WatchSource:0}: Error finding container ece3b97415629ab3b70c01504b9e0223aed78910c7546742587b4f3fee1a085b: Status 404 returned error can't find the container with id ece3b97415629ab3b70c01504b9e0223aed78910c7546742587b4f3fee1a085b Jul 23 21:51:41.076067 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:51:41.076043 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d542711_813a_4501_a243_b799ddd73340.slice/crio-102ef9e832d25e0f818c4b42a342d15ef76558ff4cb0f819bcf72defb0e55c2f WatchSource:0}: Error finding container 102ef9e832d25e0f818c4b42a342d15ef76558ff4cb0f819bcf72defb0e55c2f: Status 404 returned error can't find the container with id 102ef9e832d25e0f818c4b42a342d15ef76558ff4cb0f819bcf72defb0e55c2f Jul 23 21:51:41.167034 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:41.166822 2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-22 21:46:40 +0000 UTC" deadline="2027-12-21 03:56:10.171041969 +0000 UTC" Jul 23 21:51:41.167034 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:41.167036 2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12366h4m29.004011044s" Jul 23 21:51:41.245941 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:41.245911 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-26shl" Jul 23 21:51:41.246084 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:51:41.246024 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-26shl" podUID="500a30f7-ac15-47da-adf9-d0485f80af93" Jul 23 21:51:41.246154 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:41.246082 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4cq9t" Jul 23 21:51:41.246209 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:51:41.246168 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4cq9t" podUID="ef1bd359-5601-4a79-9b52-03d6e6a49bc6" Jul 23 21:51:41.252861 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:41.252829 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7jkjm" event={"ID":"7342a9f6-72b1-4e3c-8d50-0b0326454a86","Type":"ContainerStarted","Data":"cbea9238011cf2cb1b71399be8b353c624d4c92428bfbfe6282a76fb19c6db3b"} Jul 23 21:51:41.253915 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:41.253893 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-rpgtk" event={"ID":"d76a823d-b6d8-42d4-9060-32fe24a6ab73","Type":"ContainerStarted","Data":"d56f362f9a008e6f70adc1b2eb974937ad8f06f08a808958097f2fbb47a11a15"} Jul 23 21:51:41.254878 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:41.254859 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ntjs5" event={"ID":"1e27638c-41d9-441b-8cea-dd134bf51985","Type":"ContainerStarted","Data":"61b48ec3441af05a03a1700d6ce7dac575f812349a20e5c76846d05a470caf67"} Jul 23 21:51:41.255836 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:41.255816 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-d5lp9" event={"ID":"1b8773cc-8521-4192-a3ab-0a397ba6eb62","Type":"ContainerStarted","Data":"8e648b7dc6c07866d9415777f9a8eed0ce286d1aedc6fc9e0d57dc8c87ebc00c"} Jul 23 21:51:41.256913 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:41.256896 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-qf8mt" event={"ID":"9d06a30f-ed03-457e-8318-7d8870ec4eb4","Type":"ContainerStarted","Data":"0e87f82db486018381e0eebf057bc36d7d2eb05528e1cec27bdc98e4d5e2dc48"} Jul 23 21:51:41.258047 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:41.258028 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-nt77r" event={"ID":"49757977-27cf-4177-b295-c830e4a71b03","Type":"ContainerStarted","Data":"a3b9d9ca978cd699d2400ff9de141b2c6257f00cc2f9d610cd1082ab8fcd6804"} Jul 23 21:51:41.259128 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:41.259104 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-w6jzd" event={"ID":"4e42ce8b-422d-4281-b12e-6371c9744f8a","Type":"ContainerStarted","Data":"cb4bb6c2d55721271d0d350da0411b381bc14c205d31293d8e6fe8a916ce4e8a"} Jul 23 21:51:41.260943 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:41.260923 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-157.ec2.internal" event={"ID":"a9bc0c1f6ccfd7b14a1f3c2bc283e4db","Type":"ContainerStarted","Data":"3ce797a95767f5c324d612256c174dfd96e3969b52072c0cd452e5d3e78dab5c"} Jul 23 21:51:41.262095 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:41.262071 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-cmqcr" event={"ID":"93c54b15-009c-43ca-93a6-5a03909fc703","Type":"ContainerStarted","Data":"ece3b97415629ab3b70c01504b9e0223aed78910c7546742587b4f3fee1a085b"} Jul 23 21:51:41.263243 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:41.263222 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4pdct" event={"ID":"6d542711-813a-4501-a243-b799ddd73340","Type":"ContainerStarted","Data":"102ef9e832d25e0f818c4b42a342d15ef76558ff4cb0f819bcf72defb0e55c2f"} Jul 23 21:51:41.282631 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:41.282556 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-157.ec2.internal" podStartSLOduration=2.282543107 podStartE2EDuration="2.282543107s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 21:51:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:51:41.282129657 +0000 UTC m=+3.524037535" watchObservedRunningTime="2026-07-23 21:51:41.282543107 +0000 UTC m=+3.524450984" Jul 23 21:51:41.724315 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:41.724229 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c998f5f-33c9-44c6-abf7-f042f767fc4f-metrics-certs\") pod \"network-metrics-daemon-kl5st\" (UID: \"7c998f5f-33c9-44c6-abf7-f042f767fc4f\") " pod="openshift-multus/network-metrics-daemon-kl5st" Jul 23 21:51:41.724499 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:51:41.724420 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 21:51:41.724562 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:51:41.724500 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c998f5f-33c9-44c6-abf7-f042f767fc4f-metrics-certs podName:7c998f5f-33c9-44c6-abf7-f042f767fc4f nodeName:}" failed. No retries permitted until 2026-07-23 21:51:43.724477631 +0000 UTC m=+5.966385491 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7c998f5f-33c9-44c6-abf7-f042f767fc4f-metrics-certs") pod "network-metrics-daemon-kl5st" (UID: "7c998f5f-33c9-44c6-abf7-f042f767fc4f") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 21:51:41.826310 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:41.825552 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/500a30f7-ac15-47da-adf9-d0485f80af93-original-pull-secret\") pod \"global-pull-secret-syncer-26shl\" (UID: \"500a30f7-ac15-47da-adf9-d0485f80af93\") " pod="kube-system/global-pull-secret-syncer-26shl" Jul 23 21:51:41.826310 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:51:41.825785 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 23 21:51:41.826310 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:51:41.825869 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/500a30f7-ac15-47da-adf9-d0485f80af93-original-pull-secret podName:500a30f7-ac15-47da-adf9-d0485f80af93 nodeName:}" failed. No retries permitted until 2026-07-23 21:51:43.825848951 +0000 UTC m=+6.067756808 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/500a30f7-ac15-47da-adf9-d0485f80af93-original-pull-secret") pod "global-pull-secret-syncer-26shl" (UID: "500a30f7-ac15-47da-adf9-d0485f80af93") : object "kube-system"/"original-pull-secret" not registered Jul 23 21:51:41.926779 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:41.926059 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-st5zb\" (UniqueName: \"kubernetes.io/projected/ef1bd359-5601-4a79-9b52-03d6e6a49bc6-kube-api-access-st5zb\") pod \"network-check-target-4cq9t\" (UID: \"ef1bd359-5601-4a79-9b52-03d6e6a49bc6\") " pod="openshift-network-diagnostics/network-check-target-4cq9t" Jul 23 21:51:41.926779 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:51:41.926234 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 23 21:51:41.926779 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:51:41.926253 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 23 21:51:41.926779 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:51:41.926276 2578 projected.go:196] Error preparing data for projected volume kube-api-access-st5zb for pod openshift-network-diagnostics/network-check-target-4cq9t: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 21:51:41.926779 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:51:41.926342 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ef1bd359-5601-4a79-9b52-03d6e6a49bc6-kube-api-access-st5zb podName:ef1bd359-5601-4a79-9b52-03d6e6a49bc6 nodeName:}" failed. No retries permitted until 2026-07-23 21:51:43.926320413 +0000 UTC m=+6.168228291 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-st5zb" (UniqueName: "kubernetes.io/projected/ef1bd359-5601-4a79-9b52-03d6e6a49bc6-kube-api-access-st5zb") pod "network-check-target-4cq9t" (UID: "ef1bd359-5601-4a79-9b52-03d6e6a49bc6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 21:51:42.167631 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:42.167505 2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-22 21:46:40 +0000 UTC" deadline="2028-04-20 23:25:08.913658848 +0000 UTC" Jul 23 21:51:42.167631 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:42.167568 2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15289h33m26.746111702s" Jul 23 21:51:42.248557 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:42.247992 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kl5st" Jul 23 21:51:42.248557 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:51:42.248152 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kl5st" podUID="7c998f5f-33c9-44c6-abf7-f042f767fc4f" Jul 23 21:51:42.275222 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:42.275164 2578 generic.go:356] "Generic (PLEG): container finished" podID="8746433a99157ef819679bf50ca9e8e4" containerID="f14cd73f1081ec8ffa3214dfe8f8f0d19a3b37930c5671c6f8d411bccc2d2804" exitCode=0 Jul 23 21:51:42.276246 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:42.276151 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-157.ec2.internal" event={"ID":"8746433a99157ef819679bf50ca9e8e4","Type":"ContainerDied","Data":"f14cd73f1081ec8ffa3214dfe8f8f0d19a3b37930c5671c6f8d411bccc2d2804"} Jul 23 21:51:43.245134 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:43.245100 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4cq9t" Jul 23 21:51:43.245670 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:51:43.245243 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4cq9t" podUID="ef1bd359-5601-4a79-9b52-03d6e6a49bc6" Jul 23 21:51:43.245670 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:43.245660 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-26shl" Jul 23 21:51:43.245788 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:51:43.245769 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-26shl" podUID="500a30f7-ac15-47da-adf9-d0485f80af93" Jul 23 21:51:43.308491 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:43.307729 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-157.ec2.internal" event={"ID":"8746433a99157ef819679bf50ca9e8e4","Type":"ContainerStarted","Data":"5e0bb06342987ed758277b1d165e69e7f3db0b033bfcf7719d932f134545e98b"} Jul 23 21:51:43.744881 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:43.744773 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c998f5f-33c9-44c6-abf7-f042f767fc4f-metrics-certs\") pod \"network-metrics-daemon-kl5st\" (UID: \"7c998f5f-33c9-44c6-abf7-f042f767fc4f\") " pod="openshift-multus/network-metrics-daemon-kl5st" Jul 23 21:51:43.745055 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:51:43.744947 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 21:51:43.745055 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:51:43.745027 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c998f5f-33c9-44c6-abf7-f042f767fc4f-metrics-certs podName:7c998f5f-33c9-44c6-abf7-f042f767fc4f nodeName:}" failed. No retries permitted until 2026-07-23 21:51:47.745007124 +0000 UTC m=+9.986914978 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7c998f5f-33c9-44c6-abf7-f042f767fc4f-metrics-certs") pod "network-metrics-daemon-kl5st" (UID: "7c998f5f-33c9-44c6-abf7-f042f767fc4f") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 21:51:43.846768 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:43.846085 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/500a30f7-ac15-47da-adf9-d0485f80af93-original-pull-secret\") pod \"global-pull-secret-syncer-26shl\" (UID: \"500a30f7-ac15-47da-adf9-d0485f80af93\") " pod="kube-system/global-pull-secret-syncer-26shl" Jul 23 21:51:43.846768 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:51:43.846292 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 23 21:51:43.846768 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:51:43.846367 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/500a30f7-ac15-47da-adf9-d0485f80af93-original-pull-secret podName:500a30f7-ac15-47da-adf9-d0485f80af93 nodeName:}" failed. No retries permitted until 2026-07-23 21:51:47.846347192 +0000 UTC m=+10.088255049 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/500a30f7-ac15-47da-adf9-d0485f80af93-original-pull-secret") pod "global-pull-secret-syncer-26shl" (UID: "500a30f7-ac15-47da-adf9-d0485f80af93") : object "kube-system"/"original-pull-secret" not registered Jul 23 21:51:43.947639 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:43.947599 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-st5zb\" (UniqueName: \"kubernetes.io/projected/ef1bd359-5601-4a79-9b52-03d6e6a49bc6-kube-api-access-st5zb\") pod \"network-check-target-4cq9t\" (UID: \"ef1bd359-5601-4a79-9b52-03d6e6a49bc6\") " pod="openshift-network-diagnostics/network-check-target-4cq9t" Jul 23 21:51:43.947829 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:51:43.947807 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 23 21:51:43.947888 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:51:43.947829 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 23 21:51:43.947888 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:51:43.947846 2578 projected.go:196] Error preparing data for projected volume kube-api-access-st5zb for pod openshift-network-diagnostics/network-check-target-4cq9t: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 21:51:43.948003 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:51:43.947912 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ef1bd359-5601-4a79-9b52-03d6e6a49bc6-kube-api-access-st5zb podName:ef1bd359-5601-4a79-9b52-03d6e6a49bc6 nodeName:}" failed. No retries permitted until 2026-07-23 21:51:47.947888784 +0000 UTC m=+10.189796658 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-st5zb" (UniqueName: "kubernetes.io/projected/ef1bd359-5601-4a79-9b52-03d6e6a49bc6-kube-api-access-st5zb") pod "network-check-target-4cq9t" (UID: "ef1bd359-5601-4a79-9b52-03d6e6a49bc6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 21:51:44.248861 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:44.248313 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kl5st" Jul 23 21:51:44.248861 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:51:44.248462 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kl5st" podUID="7c998f5f-33c9-44c6-abf7-f042f767fc4f" Jul 23 21:51:45.245821 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:45.245790 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-26shl" Jul 23 21:51:45.246019 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:51:45.245916 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-26shl" podUID="500a30f7-ac15-47da-adf9-d0485f80af93" Jul 23 21:51:45.246325 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:45.246304 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4cq9t" Jul 23 21:51:45.246444 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:51:45.246401 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4cq9t" podUID="ef1bd359-5601-4a79-9b52-03d6e6a49bc6" Jul 23 21:51:46.247287 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:46.246741 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kl5st" Jul 23 21:51:46.247287 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:51:46.246903 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kl5st" podUID="7c998f5f-33c9-44c6-abf7-f042f767fc4f" Jul 23 21:51:47.245278 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:47.245200 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-26shl" Jul 23 21:51:47.245430 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:51:47.245304 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-26shl" podUID="500a30f7-ac15-47da-adf9-d0485f80af93" Jul 23 21:51:47.245647 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:47.245623 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4cq9t" Jul 23 21:51:47.245740 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:51:47.245724 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4cq9t" podUID="ef1bd359-5601-4a79-9b52-03d6e6a49bc6" Jul 23 21:51:47.779646 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:47.779596 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c998f5f-33c9-44c6-abf7-f042f767fc4f-metrics-certs\") pod \"network-metrics-daemon-kl5st\" (UID: \"7c998f5f-33c9-44c6-abf7-f042f767fc4f\") " pod="openshift-multus/network-metrics-daemon-kl5st" Jul 23 21:51:47.780109 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:51:47.779743 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 21:51:47.780109 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:51:47.779805 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c998f5f-33c9-44c6-abf7-f042f767fc4f-metrics-certs podName:7c998f5f-33c9-44c6-abf7-f042f767fc4f nodeName:}" failed. No retries permitted until 2026-07-23 21:51:55.779784261 +0000 UTC m=+18.021692114 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7c998f5f-33c9-44c6-abf7-f042f767fc4f-metrics-certs") pod "network-metrics-daemon-kl5st" (UID: "7c998f5f-33c9-44c6-abf7-f042f767fc4f") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 21:51:47.880918 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:47.880876 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/500a30f7-ac15-47da-adf9-d0485f80af93-original-pull-secret\") pod \"global-pull-secret-syncer-26shl\" (UID: \"500a30f7-ac15-47da-adf9-d0485f80af93\") " pod="kube-system/global-pull-secret-syncer-26shl" Jul 23 21:51:47.881089 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:51:47.881016 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 23 21:51:47.881089 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:51:47.881083 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/500a30f7-ac15-47da-adf9-d0485f80af93-original-pull-secret podName:500a30f7-ac15-47da-adf9-d0485f80af93 nodeName:}" failed. No retries permitted until 2026-07-23 21:51:55.881063571 +0000 UTC m=+18.122971424 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/500a30f7-ac15-47da-adf9-d0485f80af93-original-pull-secret") pod "global-pull-secret-syncer-26shl" (UID: "500a30f7-ac15-47da-adf9-d0485f80af93") : object "kube-system"/"original-pull-secret" not registered Jul 23 21:51:47.982113 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:47.982074 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-st5zb\" (UniqueName: \"kubernetes.io/projected/ef1bd359-5601-4a79-9b52-03d6e6a49bc6-kube-api-access-st5zb\") pod \"network-check-target-4cq9t\" (UID: \"ef1bd359-5601-4a79-9b52-03d6e6a49bc6\") " pod="openshift-network-diagnostics/network-check-target-4cq9t" Jul 23 21:51:47.982309 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:51:47.982277 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 23 21:51:47.982309 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:51:47.982305 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 23 21:51:47.982421 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:51:47.982319 2578 projected.go:196] Error preparing data for projected volume kube-api-access-st5zb for pod openshift-network-diagnostics/network-check-target-4cq9t: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 21:51:47.982421 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:51:47.982389 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ef1bd359-5601-4a79-9b52-03d6e6a49bc6-kube-api-access-st5zb podName:ef1bd359-5601-4a79-9b52-03d6e6a49bc6 nodeName:}" failed. No retries permitted until 2026-07-23 21:51:55.982367503 +0000 UTC m=+18.224275378 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-st5zb" (UniqueName: "kubernetes.io/projected/ef1bd359-5601-4a79-9b52-03d6e6a49bc6-kube-api-access-st5zb") pod "network-check-target-4cq9t" (UID: "ef1bd359-5601-4a79-9b52-03d6e6a49bc6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 21:51:48.247216 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:48.247123 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kl5st" Jul 23 21:51:48.247393 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:51:48.247267 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kl5st" podUID="7c998f5f-33c9-44c6-abf7-f042f767fc4f" Jul 23 21:51:49.245702 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:49.245669 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-26shl" Jul 23 21:51:49.246183 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:51:49.245787 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-26shl" podUID="500a30f7-ac15-47da-adf9-d0485f80af93" Jul 23 21:51:49.246251 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:49.246189 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4cq9t" Jul 23 21:51:49.246306 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:51:49.246279 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4cq9t" podUID="ef1bd359-5601-4a79-9b52-03d6e6a49bc6" Jul 23 21:51:50.246029 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:50.245528 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kl5st" Jul 23 21:51:50.246029 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:51:50.245689 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kl5st" podUID="7c998f5f-33c9-44c6-abf7-f042f767fc4f" Jul 23 21:51:51.246204 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:51.246030 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-26shl" Jul 23 21:51:51.246204 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:51.246069 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4cq9t" Jul 23 21:51:51.246204 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:51:51.246159 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-26shl" podUID="500a30f7-ac15-47da-adf9-d0485f80af93" Jul 23 21:51:51.246701 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:51:51.246263 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4cq9t" podUID="ef1bd359-5601-4a79-9b52-03d6e6a49bc6" Jul 23 21:51:52.245900 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:52.245868 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kl5st" Jul 23 21:51:52.246099 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:51:52.246014 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kl5st" podUID="7c998f5f-33c9-44c6-abf7-f042f767fc4f" Jul 23 21:51:53.245326 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:53.245287 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-26shl" Jul 23 21:51:53.245326 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:53.245316 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4cq9t" Jul 23 21:51:53.245854 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:51:53.245430 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-26shl" podUID="500a30f7-ac15-47da-adf9-d0485f80af93" Jul 23 21:51:53.245854 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:51:53.245611 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4cq9t" podUID="ef1bd359-5601-4a79-9b52-03d6e6a49bc6" Jul 23 21:51:54.245706 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:54.245666 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kl5st" Jul 23 21:51:54.246189 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:51:54.245813 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kl5st" podUID="7c998f5f-33c9-44c6-abf7-f042f767fc4f" Jul 23 21:51:55.245948 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:55.245913 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-26shl" Jul 23 21:51:55.246325 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:55.245959 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4cq9t" Jul 23 21:51:55.246325 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:51:55.246049 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-26shl" podUID="500a30f7-ac15-47da-adf9-d0485f80af93" Jul 23 21:51:55.246325 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:51:55.246177 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4cq9t" podUID="ef1bd359-5601-4a79-9b52-03d6e6a49bc6" Jul 23 21:51:55.848615 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:55.848586 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c998f5f-33c9-44c6-abf7-f042f767fc4f-metrics-certs\") pod \"network-metrics-daemon-kl5st\" (UID: \"7c998f5f-33c9-44c6-abf7-f042f767fc4f\") " pod="openshift-multus/network-metrics-daemon-kl5st" Jul 23 21:51:55.848766 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:51:55.848688 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 21:51:55.848766 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:51:55.848737 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c998f5f-33c9-44c6-abf7-f042f767fc4f-metrics-certs podName:7c998f5f-33c9-44c6-abf7-f042f767fc4f nodeName:}" failed. No retries permitted until 2026-07-23 21:52:11.848723021 +0000 UTC m=+34.090630873 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7c998f5f-33c9-44c6-abf7-f042f767fc4f-metrics-certs") pod "network-metrics-daemon-kl5st" (UID: "7c998f5f-33c9-44c6-abf7-f042f767fc4f") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 21:51:55.949274 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:55.949245 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/500a30f7-ac15-47da-adf9-d0485f80af93-original-pull-secret\") pod \"global-pull-secret-syncer-26shl\" (UID: \"500a30f7-ac15-47da-adf9-d0485f80af93\") " pod="kube-system/global-pull-secret-syncer-26shl" Jul 23 21:51:55.949458 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:51:55.949436 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 23 21:51:55.949522 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:51:55.949516 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/500a30f7-ac15-47da-adf9-d0485f80af93-original-pull-secret podName:500a30f7-ac15-47da-adf9-d0485f80af93 nodeName:}" failed. No retries permitted until 2026-07-23 21:52:11.949495263 +0000 UTC m=+34.191403130 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/500a30f7-ac15-47da-adf9-d0485f80af93-original-pull-secret") pod "global-pull-secret-syncer-26shl" (UID: "500a30f7-ac15-47da-adf9-d0485f80af93") : object "kube-system"/"original-pull-secret" not registered Jul 23 21:51:56.050003 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:56.049973 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-st5zb\" (UniqueName: \"kubernetes.io/projected/ef1bd359-5601-4a79-9b52-03d6e6a49bc6-kube-api-access-st5zb\") pod \"network-check-target-4cq9t\" (UID: \"ef1bd359-5601-4a79-9b52-03d6e6a49bc6\") " pod="openshift-network-diagnostics/network-check-target-4cq9t" Jul 23 21:51:56.050171 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:51:56.050112 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 23 21:51:56.050171 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:51:56.050135 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 23 21:51:56.050171 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:51:56.050145 2578 projected.go:196] Error preparing data for projected volume kube-api-access-st5zb for pod openshift-network-diagnostics/network-check-target-4cq9t: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 21:51:56.050320 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:51:56.050205 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ef1bd359-5601-4a79-9b52-03d6e6a49bc6-kube-api-access-st5zb podName:ef1bd359-5601-4a79-9b52-03d6e6a49bc6 nodeName:}" failed. No retries permitted until 2026-07-23 21:52:12.05018765 +0000 UTC m=+34.292095521 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-st5zb" (UniqueName: "kubernetes.io/projected/ef1bd359-5601-4a79-9b52-03d6e6a49bc6-kube-api-access-st5zb") pod "network-check-target-4cq9t" (UID: "ef1bd359-5601-4a79-9b52-03d6e6a49bc6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 21:51:56.245629 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:56.245517 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kl5st" Jul 23 21:51:56.245833 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:51:56.245700 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kl5st" podUID="7c998f5f-33c9-44c6-abf7-f042f767fc4f" Jul 23 21:51:57.246096 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:57.246058 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-26shl" Jul 23 21:51:57.246096 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:57.246095 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4cq9t" Jul 23 21:51:57.246589 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:51:57.246175 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-26shl" podUID="500a30f7-ac15-47da-adf9-d0485f80af93" Jul 23 21:51:57.246589 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:51:57.246322 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4cq9t" podUID="ef1bd359-5601-4a79-9b52-03d6e6a49bc6" Jul 23 21:51:58.246447 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:58.246412 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kl5st" Jul 23 21:51:58.246902 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:51:58.246520 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kl5st" podUID="7c998f5f-33c9-44c6-abf7-f042f767fc4f" Jul 23 21:51:59.246088 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:59.246053 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4cq9t" Jul 23 21:51:59.246282 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:51:59.246053 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-26shl" Jul 23 21:51:59.246282 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:51:59.246185 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4cq9t" podUID="ef1bd359-5601-4a79-9b52-03d6e6a49bc6" Jul 23 21:51:59.246390 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:51:59.246282 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-26shl" podUID="500a30f7-ac15-47da-adf9-d0485f80af93" Jul 23 21:52:00.246239 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:00.246058 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kl5st" Jul 23 21:52:00.246824 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:52:00.246334 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kl5st" podUID="7c998f5f-33c9-44c6-abf7-f042f767fc4f" Jul 23 21:52:00.344262 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:00.344232 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4pdct" event={"ID":"6d542711-813a-4501-a243-b799ddd73340","Type":"ContainerStarted","Data":"499ccd1caef38363f9f1a418e3a4fafdcc3cbfbf7cfe27417884bdf501b838ef"} Jul 23 21:52:00.345838 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:00.345819 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7jkjm_7342a9f6-72b1-4e3c-8d50-0b0326454a86/ovn-acl-logging/0.log" Jul 23 21:52:00.346206 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:00.346186 2578 generic.go:356] "Generic (PLEG): container finished" podID="7342a9f6-72b1-4e3c-8d50-0b0326454a86" containerID="6d236190bedd736d8df45ac078370757e1e7bc0ddb109dba6a2cb79fa43ff4a0" exitCode=1 Jul 23 21:52:00.346338 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:00.346256 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7jkjm" event={"ID":"7342a9f6-72b1-4e3c-8d50-0b0326454a86","Type":"ContainerDied","Data":"6d236190bedd736d8df45ac078370757e1e7bc0ddb109dba6a2cb79fa43ff4a0"} Jul 23 21:52:00.346396 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:00.346340 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7jkjm" event={"ID":"7342a9f6-72b1-4e3c-8d50-0b0326454a86","Type":"ContainerStarted","Data":"0f92246c27905ef056a22e7fb809c598817d3909c61a7a6598603a4a082c86dc"} Jul 23 21:52:00.347904 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:00.347826 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-rpgtk" event={"ID":"d76a823d-b6d8-42d4-9060-32fe24a6ab73","Type":"ContainerStarted","Data":"9f345282fcd684f9c41e92d289af199d28fe8c0613334f324fad18f4c36967fb"} Jul 23 21:52:00.350448 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:00.350424 2578 generic.go:356] "Generic (PLEG): container finished" podID="1e27638c-41d9-441b-8cea-dd134bf51985" containerID="c758f5191df121ecadb4b0559a24dedb31f0c78ef60dcb6b27c62113f7604cde" exitCode=0 Jul 23 21:52:00.350572 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:00.350524 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ntjs5" event={"ID":"1e27638c-41d9-441b-8cea-dd134bf51985","Type":"ContainerDied","Data":"c758f5191df121ecadb4b0559a24dedb31f0c78ef60dcb6b27c62113f7604cde"} Jul 23 21:52:00.353354 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:00.353272 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-d5lp9" event={"ID":"1b8773cc-8521-4192-a3ab-0a397ba6eb62","Type":"ContainerStarted","Data":"1abbd10034ec2fefa278fbe0bdda4651cae7a6ef76c0206aa1d73cc272785eaa"} Jul 23 21:52:00.355441 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:00.355412 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-qf8mt" event={"ID":"9d06a30f-ed03-457e-8318-7d8870ec4eb4","Type":"ContainerStarted","Data":"11a3afefc21055cfc5fcda1d5300a10a24c1601e3a58ccfd0bb2252258b8385e"} Jul 23 21:52:00.357335 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:00.357032 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-nt77r" event={"ID":"49757977-27cf-4177-b295-c830e4a71b03","Type":"ContainerStarted","Data":"9d2840ae6415b0d0cdabeefd37305d1c4212796f9529ed20ac21b31ac689849b"} Jul 23 21:52:00.358501 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:00.358480 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-w6jzd" event={"ID":"4e42ce8b-422d-4281-b12e-6371c9744f8a","Type":"ContainerStarted","Data":"67644291c395dd1d31e8d469e2e86299d777eb463e7cca80e5ca444eb9aace87"} Jul 23 21:52:00.362890 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:00.362855 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-157.ec2.internal" podStartSLOduration=21.362844356 podStartE2EDuration="21.362844356s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 21:51:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:51:43.326292252 +0000 UTC m=+5.568200130" watchObservedRunningTime="2026-07-23 21:52:00.362844356 +0000 UTC m=+22.604752230" Jul 23 21:52:00.363725 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:00.363686 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-rpgtk" podStartSLOduration=11.78109415 podStartE2EDuration="22.363673993s" totalImagesPullingTime="10.582579843s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:51:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:52:00.362967044 +0000 UTC m=+22.604874899" watchObservedRunningTime="2026-07-23 21:52:00.363673993 +0000 UTC m=+22.605581867" Jul 23 21:52:00.399048 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:00.398872 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-nt77r" podStartSLOduration=3.6090386150000002 podStartE2EDuration="22.398858785s" totalImagesPullingTime="18.78982017s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 21:51:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:52:00.398703701 +0000 UTC m=+22.640611577" watchObservedRunningTime="2026-07-23 21:52:00.398858785 +0000 UTC m=+22.640766658" Jul 23 21:52:00.415629 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:00.415588 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-qf8mt" podStartSLOduration=3.6270864229999997 podStartE2EDuration="22.415573551s" totalImagesPullingTime="18.788487128s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 21:51:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:52:00.415247049 +0000 UTC m=+22.657154923" watchObservedRunningTime="2026-07-23 21:52:00.415573551 +0000 UTC m=+22.657481457" Jul 23 21:52:00.454175 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:00.454081 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-w6jzd" podStartSLOduration=3.680586364 podStartE2EDuration="22.454059714s" totalImagesPullingTime="18.77347335s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 21:51:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:52:00.430344014 +0000 UTC m=+22.672251888" watchObservedRunningTime="2026-07-23 21:52:00.454059714 +0000 UTC m=+22.695967590" Jul 23 21:52:00.454318 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:00.454289 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-d5lp9" podStartSLOduration=3.624987263 podStartE2EDuration="22.45428369s" totalImagesPullingTime="18.829296427s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 21:51:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:52:00.453863125 +0000 UTC m=+22.695771000" watchObservedRunningTime="2026-07-23 21:52:00.45428369 +0000 UTC m=+22.696191566" Jul 23 21:52:01.246003 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:01.245970 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-26shl" Jul 23 21:52:01.246140 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:52:01.246099 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-26shl" podUID="500a30f7-ac15-47da-adf9-d0485f80af93" Jul 23 21:52:01.246140 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:01.246131 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4cq9t" Jul 23 21:52:01.246224 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:52:01.246199 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4cq9t" podUID="ef1bd359-5601-4a79-9b52-03d6e6a49bc6" Jul 23 21:52:01.257314 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:01.257288 2578 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 23 21:52:01.364099 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:01.364072 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7jkjm_7342a9f6-72b1-4e3c-8d50-0b0326454a86/ovn-acl-logging/0.log" Jul 23 21:52:01.364590 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:01.364558 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7jkjm" event={"ID":"7342a9f6-72b1-4e3c-8d50-0b0326454a86","Type":"ContainerStarted","Data":"0920b32683cf25f692f3ffa41a49ebf9bc58cf57185c8ae8934939c4ec5dc501"} Jul 23 21:52:01.364705 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:01.364653 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7jkjm" event={"ID":"7342a9f6-72b1-4e3c-8d50-0b0326454a86","Type":"ContainerStarted","Data":"616fb5021ff7e33821c940a07bfdfa6c61b6742fa4b1e356bfda0b127c3b2a43"} Jul 23 21:52:01.364705 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:01.364691 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7jkjm" event={"ID":"7342a9f6-72b1-4e3c-8d50-0b0326454a86","Type":"ContainerStarted","Data":"90d4ab0fe49c471927fec331311dea08faf06b3bc284e7bafee6ab8fd0e4e1ec"} Jul 23 21:52:01.364797 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:01.364732 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7jkjm" event={"ID":"7342a9f6-72b1-4e3c-8d50-0b0326454a86","Type":"ContainerStarted","Data":"ad067f395f5581dfd5765ff407f8180652e622e3f770b48f7314a248f1704d20"} Jul 23 21:52:01.366102 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:01.366073 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-cmqcr" event={"ID":"93c54b15-009c-43ca-93a6-5a03909fc703","Type":"ContainerStarted","Data":"2f0fd25958538bfa043349e9c46a87e11693fda3509ab1eb2099d95bd7df6ccf"} Jul 23 21:52:01.368019 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:01.367956 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4pdct" event={"ID":"6d542711-813a-4501-a243-b799ddd73340","Type":"ContainerStarted","Data":"2686c195d5502fae752abb58cef5bdfdc1ee6e709d9f364f61b42bda9eb43b3d"} Jul 23 21:52:01.383359 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:01.383294 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-cmqcr" podStartSLOduration=4.603625173 podStartE2EDuration="23.38327743s" totalImagesPullingTime="18.779652257s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 21:51:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:52:01.383144224 +0000 UTC m=+23.625052098" watchObservedRunningTime="2026-07-23 21:52:01.38327743 +0000 UTC m=+23.625185304" Jul 23 21:52:01.839099 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:01.839068 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-rpgtk" Jul 23 21:52:01.839937 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:01.839901 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-rpgtk" Jul 23 21:52:02.173966 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:02.173876 2578 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-23T21:52:01.257307378Z","UUID":"49e67ca1-13bd-41d3-af48-cd829a7f762f","Handler":null,"Name":"","Endpoint":""} Jul 23 21:52:02.175528 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:02.175507 2578 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 23 21:52:02.175671 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:02.175563 2578 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 23 21:52:02.245578 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:02.245534 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kl5st" Jul 23 21:52:02.245745 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:52:02.245662 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kl5st" podUID="7c998f5f-33c9-44c6-abf7-f042f767fc4f" Jul 23 21:52:02.373048 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:02.372951 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4pdct" event={"ID":"6d542711-813a-4501-a243-b799ddd73340","Type":"ContainerStarted","Data":"c9416e97fcfd26f84d8eaf3c23ff6be06fe1511d01cbc14601e8b01115fd1b61"} Jul 23 21:52:02.391430 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:02.391370 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4pdct" podStartSLOduration=3.821671232 podStartE2EDuration="24.391355965s" totalImagesPullingTime="20.569684733s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 21:51:38 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:52:02.391278313 +0000 UTC m=+24.633186187" watchObservedRunningTime="2026-07-23 21:52:02.391355965 +0000 UTC m=+24.633263838" Jul 23 21:52:03.245728 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:03.245508 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-26shl" Jul 23 21:52:03.245882 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:03.245559 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4cq9t" Jul 23 21:52:03.245882 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:52:03.245801 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-26shl" podUID="500a30f7-ac15-47da-adf9-d0485f80af93" Jul 23 21:52:03.245990 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:52:03.245896 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4cq9t" podUID="ef1bd359-5601-4a79-9b52-03d6e6a49bc6" Jul 23 21:52:03.382124 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:03.382098 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7jkjm_7342a9f6-72b1-4e3c-8d50-0b0326454a86/ovn-acl-logging/0.log" Jul 23 21:52:03.382578 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:03.382516 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7jkjm" event={"ID":"7342a9f6-72b1-4e3c-8d50-0b0326454a86","Type":"ContainerStarted","Data":"d0ff1d9b1e56d064b24c47d152bab999ecbf2b55c995935fd7a8b780515c04b2"} Jul 23 21:52:03.382707 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:03.382532 2578 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 23 21:52:03.773331 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:03.773296 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-rpgtk" Jul 23 21:52:03.774082 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:03.774042 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-rpgtk" Jul 23 21:52:04.246222 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:04.246133 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kl5st" Jul 23 21:52:04.246380 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:52:04.246281 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kl5st" podUID="7c998f5f-33c9-44c6-abf7-f042f767fc4f" Jul 23 21:52:05.245483 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:05.245218 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4cq9t" Jul 23 21:52:05.245483 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:05.245293 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-26shl" Jul 23 21:52:05.246205 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:52:05.245494 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4cq9t" podUID="ef1bd359-5601-4a79-9b52-03d6e6a49bc6" Jul 23 21:52:05.246205 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:52:05.245533 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-26shl" podUID="500a30f7-ac15-47da-adf9-d0485f80af93" Jul 23 21:52:05.389681 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:05.389659 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7jkjm_7342a9f6-72b1-4e3c-8d50-0b0326454a86/ovn-acl-logging/0.log" Jul 23 21:52:05.390039 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:05.390016 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7jkjm" event={"ID":"7342a9f6-72b1-4e3c-8d50-0b0326454a86","Type":"ContainerStarted","Data":"550164abda49f3642035f1e7f04d96a7cbf17c190e4a777a68b429556a69b608"} Jul 23 21:52:05.390370 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:05.390327 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7jkjm" Jul 23 21:52:05.390492 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:05.390477 2578 scope.go:117] "RemoveContainer" containerID="6d236190bedd736d8df45ac078370757e1e7bc0ddb109dba6a2cb79fa43ff4a0" Jul 23 21:52:05.392106 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:05.392076 2578 generic.go:356] "Generic (PLEG): container finished" podID="1e27638c-41d9-441b-8cea-dd134bf51985" containerID="e235b88dc602273a0624ca53a72658112bf95104f764b5fdd777321ce730c905" exitCode=0 Jul 23 21:52:05.392944 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:05.392922 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ntjs5" event={"ID":"1e27638c-41d9-441b-8cea-dd134bf51985","Type":"ContainerDied","Data":"e235b88dc602273a0624ca53a72658112bf95104f764b5fdd777321ce730c905"} Jul 23 21:52:05.408264 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:05.408246 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7jkjm" Jul 23 21:52:06.245202 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:06.245169 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kl5st" Jul 23 21:52:06.245324 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:52:06.245278 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kl5st" podUID="7c998f5f-33c9-44c6-abf7-f042f767fc4f" Jul 23 21:52:06.396440 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:06.396408 2578 generic.go:356] "Generic (PLEG): container finished" podID="1e27638c-41d9-441b-8cea-dd134bf51985" containerID="206248957af39ff735ca921877836e9da8f8e8b71c3505d4ee098a4f8459dae7" exitCode=0 Jul 23 21:52:06.396886 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:06.396520 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ntjs5" event={"ID":"1e27638c-41d9-441b-8cea-dd134bf51985","Type":"ContainerDied","Data":"206248957af39ff735ca921877836e9da8f8e8b71c3505d4ee098a4f8459dae7"} Jul 23 21:52:06.400749 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:06.400731 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7jkjm_7342a9f6-72b1-4e3c-8d50-0b0326454a86/ovn-acl-logging/0.log" Jul 23 21:52:06.401139 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:06.401114 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7jkjm" event={"ID":"7342a9f6-72b1-4e3c-8d50-0b0326454a86","Type":"ContainerStarted","Data":"2d8247ec8e88522a9d6aec0dd34d96ef06585bd914d2d9e6a3ad44782344ea42"} Jul 23 21:52:06.401235 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:06.401224 2578 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 23 21:52:06.401474 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:06.401459 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7jkjm" Jul 23 21:52:06.416894 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:06.416872 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7jkjm" Jul 23 21:52:06.447205 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:06.447161 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-7jkjm" podStartSLOduration=9.625399541 podStartE2EDuration="28.44714776s" totalImagesPullingTime="18.821748219s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 21:51:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:52:06.445778149 +0000 UTC m=+28.687686019" watchObservedRunningTime="2026-07-23 21:52:06.44714776 +0000 UTC m=+28.689055663" Jul 23 21:52:06.792942 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:06.792730 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-26shl"] Jul 23 21:52:06.793080 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:06.793039 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-26shl" Jul 23 21:52:06.793151 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:52:06.793132 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-26shl" podUID="500a30f7-ac15-47da-adf9-d0485f80af93" Jul 23 21:52:06.797109 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:06.797081 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-kl5st"] Jul 23 21:52:06.797252 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:06.797223 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kl5st" Jul 23 21:52:06.797378 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:52:06.797347 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kl5st" podUID="7c998f5f-33c9-44c6-abf7-f042f767fc4f" Jul 23 21:52:06.797378 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:06.797934 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-4cq9t"] Jul 23 21:52:06.797378 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:06.798016 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4cq9t" Jul 23 21:52:06.797378 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:52:06.798099 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4cq9t" podUID="ef1bd359-5601-4a79-9b52-03d6e6a49bc6" Jul 23 21:52:07.405489 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:07.405452 2578 generic.go:356] "Generic (PLEG): container finished" podID="1e27638c-41d9-441b-8cea-dd134bf51985" containerID="d17199c57bc2258f16d1285681b336b644c1233790efd0168b7aa84b4036af6c" exitCode=0 Jul 23 21:52:07.405955 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:07.405520 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ntjs5" event={"ID":"1e27638c-41d9-441b-8cea-dd134bf51985","Type":"ContainerDied","Data":"d17199c57bc2258f16d1285681b336b644c1233790efd0168b7aa84b4036af6c"} Jul 23 21:52:07.405955 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:07.405638 2578 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 23 21:52:08.247139 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:08.247105 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4cq9t" Jul 23 21:52:08.247299 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:08.247199 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-26shl" Jul 23 21:52:08.247299 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:08.247241 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kl5st" Jul 23 21:52:08.247299 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:52:08.247254 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4cq9t" podUID="ef1bd359-5601-4a79-9b52-03d6e6a49bc6" Jul 23 21:52:08.247444 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:52:08.247329 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kl5st" podUID="7c998f5f-33c9-44c6-abf7-f042f767fc4f" Jul 23 21:52:08.247444 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:52:08.247418 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-26shl" podUID="500a30f7-ac15-47da-adf9-d0485f80af93" Jul 23 21:52:08.407855 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:08.407830 2578 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 23 21:52:09.190030 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:09.189945 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7jkjm" Jul 23 21:52:09.201652 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:09.201220 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-7jkjm" podUID="7342a9f6-72b1-4e3c-8d50-0b0326454a86" containerName="ovnkube-controller" probeResult="failure" output="" Jul 23 21:52:09.212295 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:09.212261 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-7jkjm" podUID="7342a9f6-72b1-4e3c-8d50-0b0326454a86" containerName="ovnkube-controller" probeResult="failure" output="" Jul 23 21:52:09.420794 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:09.420757 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-7jkjm" podUID="7342a9f6-72b1-4e3c-8d50-0b0326454a86" containerName="ovnkube-controller" probeResult="failure" output="" Jul 23 21:52:10.245496 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:10.245461 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-26shl" Jul 23 21:52:10.245686 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:10.245466 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kl5st" Jul 23 21:52:10.245686 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:10.245601 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4cq9t" Jul 23 21:52:10.245686 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:52:10.245624 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-26shl" podUID="500a30f7-ac15-47da-adf9-d0485f80af93" Jul 23 21:52:10.245846 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:52:10.245714 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4cq9t" podUID="ef1bd359-5601-4a79-9b52-03d6e6a49bc6" Jul 23 21:52:10.245846 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:52:10.245789 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kl5st" podUID="7c998f5f-33c9-44c6-abf7-f042f767fc4f" Jul 23 21:52:11.870598 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:11.870515 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c998f5f-33c9-44c6-abf7-f042f767fc4f-metrics-certs\") pod \"network-metrics-daemon-kl5st\" (UID: \"7c998f5f-33c9-44c6-abf7-f042f767fc4f\") " pod="openshift-multus/network-metrics-daemon-kl5st" Jul 23 21:52:11.870949 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:52:11.870638 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 21:52:11.870949 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:52:11.870692 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c998f5f-33c9-44c6-abf7-f042f767fc4f-metrics-certs podName:7c998f5f-33c9-44c6-abf7-f042f767fc4f nodeName:}" failed. No retries permitted until 2026-07-23 21:52:43.870677548 +0000 UTC m=+66.112585400 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7c998f5f-33c9-44c6-abf7-f042f767fc4f-metrics-certs") pod "network-metrics-daemon-kl5st" (UID: "7c998f5f-33c9-44c6-abf7-f042f767fc4f") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 21:52:11.971572 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:11.971517 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/500a30f7-ac15-47da-adf9-d0485f80af93-original-pull-secret\") pod \"global-pull-secret-syncer-26shl\" (UID: \"500a30f7-ac15-47da-adf9-d0485f80af93\") " pod="kube-system/global-pull-secret-syncer-26shl" Jul 23 21:52:11.971728 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:52:11.971662 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 23 21:52:11.971728 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:52:11.971725 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/500a30f7-ac15-47da-adf9-d0485f80af93-original-pull-secret podName:500a30f7-ac15-47da-adf9-d0485f80af93 nodeName:}" failed. No retries permitted until 2026-07-23 21:52:43.971709477 +0000 UTC m=+66.213617331 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/500a30f7-ac15-47da-adf9-d0485f80af93-original-pull-secret") pod "global-pull-secret-syncer-26shl" (UID: "500a30f7-ac15-47da-adf9-d0485f80af93") : object "kube-system"/"original-pull-secret" not registered Jul 23 21:52:12.072218 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:12.072182 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-st5zb\" (UniqueName: \"kubernetes.io/projected/ef1bd359-5601-4a79-9b52-03d6e6a49bc6-kube-api-access-st5zb\") pod \"network-check-target-4cq9t\" (UID: \"ef1bd359-5601-4a79-9b52-03d6e6a49bc6\") " pod="openshift-network-diagnostics/network-check-target-4cq9t" Jul 23 21:52:12.072397 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:52:12.072351 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 23 21:52:12.072397 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:52:12.072373 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 23 21:52:12.072397 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:52:12.072383 2578 projected.go:196] Error preparing data for projected volume kube-api-access-st5zb for pod openshift-network-diagnostics/network-check-target-4cq9t: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 21:52:12.072508 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:52:12.072434 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ef1bd359-5601-4a79-9b52-03d6e6a49bc6-kube-api-access-st5zb podName:ef1bd359-5601-4a79-9b52-03d6e6a49bc6 nodeName:}" failed. No retries permitted until 2026-07-23 21:52:44.072420751 +0000 UTC m=+66.314328607 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-st5zb" (UniqueName: "kubernetes.io/projected/ef1bd359-5601-4a79-9b52-03d6e6a49bc6-kube-api-access-st5zb") pod "network-check-target-4cq9t" (UID: "ef1bd359-5601-4a79-9b52-03d6e6a49bc6") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 21:52:12.246197 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:12.246123 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kl5st" Jul 23 21:52:12.246370 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:12.246124 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4cq9t" Jul 23 21:52:12.246370 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:52:12.246329 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-4cq9t" podUID="ef1bd359-5601-4a79-9b52-03d6e6a49bc6" Jul 23 21:52:12.246370 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:52:12.246231 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kl5st" podUID="7c998f5f-33c9-44c6-abf7-f042f767fc4f" Jul 23 21:52:12.246527 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:12.246383 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-26shl" Jul 23 21:52:12.246527 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:52:12.246480 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-26shl" podUID="500a30f7-ac15-47da-adf9-d0485f80af93" Jul 23 21:52:12.419311 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:12.419057 2578 generic.go:356] "Generic (PLEG): container finished" podID="1e27638c-41d9-441b-8cea-dd134bf51985" containerID="ef95b8d34a363c6659c07981a4e115c358429890bc2ed2b889ffa0ef0bc33a94" exitCode=0 Jul 23 21:52:12.419311 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:12.419125 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ntjs5" event={"ID":"1e27638c-41d9-441b-8cea-dd134bf51985","Type":"ContainerDied","Data":"ef95b8d34a363c6659c07981a4e115c358429890bc2ed2b889ffa0ef0bc33a94"} Jul 23 21:52:12.602881 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:12.602858 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-157.ec2.internal" event="NodeReady" Jul 23 21:52:12.603039 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:12.603029 2578 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 23 21:52:12.640455 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:12.640425 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-fbbbb55d8-pz9t5"] Jul 23 21:52:12.653685 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:12.653658 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-5gh5l"] Jul 23 21:52:12.653815 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:12.653768 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-fbbbb55d8-pz9t5" Jul 23 21:52:12.656237 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:12.656216 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 23 21:52:12.656364 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:12.656240 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 23 21:52:12.656364 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:12.656240 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-hkh5w\"" Jul 23 21:52:12.656474 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:12.656217 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 23 21:52:12.662981 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:12.662962 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 23 21:52:12.671967 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:12.671949 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-6mgzw"] Jul 23 21:52:12.672104 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:12.672088 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-5gh5l" Jul 23 21:52:12.674509 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:12.674488 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 23 21:52:12.674772 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:12.674755 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-swdkn\"" Jul 23 21:52:12.674816 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:12.674784 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 23 21:52:12.675032 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:12.675019 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 23 21:52:12.681002 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:12.680980 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-fbbbb55d8-pz9t5"] Jul 23 21:52:12.681131 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:12.681017 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-5gh5l"] Jul 23 21:52:12.681131 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:12.681041 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-6mgzw"] Jul 23 21:52:12.681232 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:12.681137 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-6mgzw" Jul 23 21:52:12.683491 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:12.683475 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 23 21:52:12.683624 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:12.683612 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-wq952\"" Jul 23 21:52:12.683861 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:12.683849 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 23 21:52:12.777365 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:12.777288 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3-bound-sa-token\") pod \"image-registry-fbbbb55d8-pz9t5\" (UID: \"88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3\") " pod="openshift-image-registry/image-registry-fbbbb55d8-pz9t5" Jul 23 21:52:12.777365 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:12.777321 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/634a7674-21bb-4fd9-8ce8-1d6357c4500d-cert\") pod \"ingress-canary-5gh5l\" (UID: \"634a7674-21bb-4fd9-8ce8-1d6357c4500d\") " pod="openshift-ingress-canary/ingress-canary-5gh5l" Jul 23 21:52:12.777365 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:12.777353 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c8c008d7-7d6c-42e2-b9c6-4c9e9557c7c4-tmp-dir\") pod \"dns-default-6mgzw\" (UID: \"c8c008d7-7d6c-42e2-b9c6-4c9e9557c7c4\") " pod="openshift-dns/dns-default-6mgzw" Jul 23 21:52:12.777638 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:12.777407 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3-registry-certificates\") pod \"image-registry-fbbbb55d8-pz9t5\" (UID: \"88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3\") " pod="openshift-image-registry/image-registry-fbbbb55d8-pz9t5" Jul 23 21:52:12.777638 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:12.777437 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rzwzj\" (UniqueName: \"kubernetes.io/projected/c8c008d7-7d6c-42e2-b9c6-4c9e9557c7c4-kube-api-access-rzwzj\") pod \"dns-default-6mgzw\" (UID: \"c8c008d7-7d6c-42e2-b9c6-4c9e9557c7c4\") " pod="openshift-dns/dns-default-6mgzw" Jul 23 21:52:12.777638 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:12.777455 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3-installation-pull-secrets\") pod \"image-registry-fbbbb55d8-pz9t5\" (UID: \"88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3\") " pod="openshift-image-registry/image-registry-fbbbb55d8-pz9t5" Jul 23 21:52:12.777638 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:12.777474 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7tg7w\" (UniqueName: \"kubernetes.io/projected/88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3-kube-api-access-7tg7w\") pod \"image-registry-fbbbb55d8-pz9t5\" (UID: \"88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3\") " pod="openshift-image-registry/image-registry-fbbbb55d8-pz9t5" Jul 23 21:52:12.777638 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:12.777498 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3-image-registry-private-configuration\") pod \"image-registry-fbbbb55d8-pz9t5\" (UID: \"88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3\") " pod="openshift-image-registry/image-registry-fbbbb55d8-pz9t5" Jul 23 21:52:12.777638 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:12.777557 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3-registry-tls\") pod \"image-registry-fbbbb55d8-pz9t5\" (UID: \"88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3\") " pod="openshift-image-registry/image-registry-fbbbb55d8-pz9t5" Jul 23 21:52:12.777638 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:12.777612 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3-ca-trust-extracted\") pod \"image-registry-fbbbb55d8-pz9t5\" (UID: \"88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3\") " pod="openshift-image-registry/image-registry-fbbbb55d8-pz9t5" Jul 23 21:52:12.777850 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:12.777643 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c8c008d7-7d6c-42e2-b9c6-4c9e9557c7c4-config-volume\") pod \"dns-default-6mgzw\" (UID: \"c8c008d7-7d6c-42e2-b9c6-4c9e9557c7c4\") " pod="openshift-dns/dns-default-6mgzw" Jul 23 21:52:12.777850 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:12.777664 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3-trusted-ca\") pod \"image-registry-fbbbb55d8-pz9t5\" (UID: \"88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3\") " pod="openshift-image-registry/image-registry-fbbbb55d8-pz9t5" Jul 23 21:52:12.777850 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:12.777683 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cs5z8\" (UniqueName: \"kubernetes.io/projected/634a7674-21bb-4fd9-8ce8-1d6357c4500d-kube-api-access-cs5z8\") pod \"ingress-canary-5gh5l\" (UID: \"634a7674-21bb-4fd9-8ce8-1d6357c4500d\") " pod="openshift-ingress-canary/ingress-canary-5gh5l" Jul 23 21:52:12.777850 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:12.777704 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c8c008d7-7d6c-42e2-b9c6-4c9e9557c7c4-metrics-tls\") pod \"dns-default-6mgzw\" (UID: \"c8c008d7-7d6c-42e2-b9c6-4c9e9557c7c4\") " pod="openshift-dns/dns-default-6mgzw" Jul 23 21:52:12.878193 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:12.878155 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3-ca-trust-extracted\") pod \"image-registry-fbbbb55d8-pz9t5\" (UID: \"88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3\") " pod="openshift-image-registry/image-registry-fbbbb55d8-pz9t5" Jul 23 21:52:12.878193 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:12.878191 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c8c008d7-7d6c-42e2-b9c6-4c9e9557c7c4-config-volume\") pod \"dns-default-6mgzw\" (UID: \"c8c008d7-7d6c-42e2-b9c6-4c9e9557c7c4\") " pod="openshift-dns/dns-default-6mgzw" Jul 23 21:52:12.878698 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:12.878220 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3-trusted-ca\") pod \"image-registry-fbbbb55d8-pz9t5\" (UID: \"88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3\") " pod="openshift-image-registry/image-registry-fbbbb55d8-pz9t5" Jul 23 21:52:12.878698 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:12.878284 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cs5z8\" (UniqueName: \"kubernetes.io/projected/634a7674-21bb-4fd9-8ce8-1d6357c4500d-kube-api-access-cs5z8\") pod \"ingress-canary-5gh5l\" (UID: \"634a7674-21bb-4fd9-8ce8-1d6357c4500d\") " pod="openshift-ingress-canary/ingress-canary-5gh5l" Jul 23 21:52:12.878698 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:12.878310 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c8c008d7-7d6c-42e2-b9c6-4c9e9557c7c4-metrics-tls\") pod \"dns-default-6mgzw\" (UID: \"c8c008d7-7d6c-42e2-b9c6-4c9e9557c7c4\") " pod="openshift-dns/dns-default-6mgzw" Jul 23 21:52:12.878698 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:52:12.878391 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 23 21:52:12.878698 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:52:12.878446 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c8c008d7-7d6c-42e2-b9c6-4c9e9557c7c4-metrics-tls podName:c8c008d7-7d6c-42e2-b9c6-4c9e9557c7c4 nodeName:}" failed. No retries permitted until 2026-07-23 21:52:13.37842988 +0000 UTC m=+35.620337732 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c8c008d7-7d6c-42e2-b9c6-4c9e9557c7c4-metrics-tls") pod "dns-default-6mgzw" (UID: "c8c008d7-7d6c-42e2-b9c6-4c9e9557c7c4") : secret "dns-default-metrics-tls" not found Jul 23 21:52:12.878698 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:12.878478 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3-bound-sa-token\") pod \"image-registry-fbbbb55d8-pz9t5\" (UID: \"88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3\") " pod="openshift-image-registry/image-registry-fbbbb55d8-pz9t5" Jul 23 21:52:12.878698 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:12.878509 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/634a7674-21bb-4fd9-8ce8-1d6357c4500d-cert\") pod \"ingress-canary-5gh5l\" (UID: \"634a7674-21bb-4fd9-8ce8-1d6357c4500d\") " pod="openshift-ingress-canary/ingress-canary-5gh5l" Jul 23 21:52:12.878698 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:12.878554 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c8c008d7-7d6c-42e2-b9c6-4c9e9557c7c4-tmp-dir\") pod \"dns-default-6mgzw\" (UID: \"c8c008d7-7d6c-42e2-b9c6-4c9e9557c7c4\") " pod="openshift-dns/dns-default-6mgzw" Jul 23 21:52:12.878698 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:12.878587 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3-registry-certificates\") pod \"image-registry-fbbbb55d8-pz9t5\" (UID: \"88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3\") " pod="openshift-image-registry/image-registry-fbbbb55d8-pz9t5" Jul 23 21:52:12.878698 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:12.878613 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rzwzj\" (UniqueName: \"kubernetes.io/projected/c8c008d7-7d6c-42e2-b9c6-4c9e9557c7c4-kube-api-access-rzwzj\") pod \"dns-default-6mgzw\" (UID: \"c8c008d7-7d6c-42e2-b9c6-4c9e9557c7c4\") " pod="openshift-dns/dns-default-6mgzw" Jul 23 21:52:12.878698 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:12.878624 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3-ca-trust-extracted\") pod \"image-registry-fbbbb55d8-pz9t5\" (UID: \"88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3\") " pod="openshift-image-registry/image-registry-fbbbb55d8-pz9t5" Jul 23 21:52:12.878698 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:12.878639 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3-installation-pull-secrets\") pod \"image-registry-fbbbb55d8-pz9t5\" (UID: \"88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3\") " pod="openshift-image-registry/image-registry-fbbbb55d8-pz9t5" Jul 23 21:52:12.878698 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:12.878682 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7tg7w\" (UniqueName: \"kubernetes.io/projected/88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3-kube-api-access-7tg7w\") pod \"image-registry-fbbbb55d8-pz9t5\" (UID: \"88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3\") " pod="openshift-image-registry/image-registry-fbbbb55d8-pz9t5" Jul 23 21:52:12.879292 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:12.878725 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3-image-registry-private-configuration\") pod \"image-registry-fbbbb55d8-pz9t5\" (UID: \"88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3\") " pod="openshift-image-registry/image-registry-fbbbb55d8-pz9t5" Jul 23 21:52:12.879292 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:12.878943 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3-registry-tls\") pod \"image-registry-fbbbb55d8-pz9t5\" (UID: \"88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3\") " pod="openshift-image-registry/image-registry-fbbbb55d8-pz9t5" Jul 23 21:52:12.879292 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:12.878948 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c8c008d7-7d6c-42e2-b9c6-4c9e9557c7c4-tmp-dir\") pod \"dns-default-6mgzw\" (UID: \"c8c008d7-7d6c-42e2-b9c6-4c9e9557c7c4\") " pod="openshift-dns/dns-default-6mgzw" Jul 23 21:52:12.879292 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:12.879020 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c8c008d7-7d6c-42e2-b9c6-4c9e9557c7c4-config-volume\") pod \"dns-default-6mgzw\" (UID: \"c8c008d7-7d6c-42e2-b9c6-4c9e9557c7c4\") " pod="openshift-dns/dns-default-6mgzw" Jul 23 21:52:12.879292 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:52:12.879069 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 23 21:52:12.879292 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:52:12.879083 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-fbbbb55d8-pz9t5: secret "image-registry-tls" not found Jul 23 21:52:12.879292 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:12.879133 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3-registry-certificates\") pod \"image-registry-fbbbb55d8-pz9t5\" (UID: \"88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3\") " pod="openshift-image-registry/image-registry-fbbbb55d8-pz9t5" Jul 23 21:52:12.879292 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:52:12.879139 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3-registry-tls podName:88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3 nodeName:}" failed. No retries permitted until 2026-07-23 21:52:13.379123786 +0000 UTC m=+35.621031657 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3-registry-tls") pod "image-registry-fbbbb55d8-pz9t5" (UID: "88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3") : secret "image-registry-tls" not found Jul 23 21:52:12.879686 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:12.879416 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3-trusted-ca\") pod \"image-registry-fbbbb55d8-pz9t5\" (UID: \"88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3\") " pod="openshift-image-registry/image-registry-fbbbb55d8-pz9t5" Jul 23 21:52:12.882916 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:12.882884 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3-image-registry-private-configuration\") pod \"image-registry-fbbbb55d8-pz9t5\" (UID: \"88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3\") " pod="openshift-image-registry/image-registry-fbbbb55d8-pz9t5" Jul 23 21:52:12.882916 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:12.882910 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/634a7674-21bb-4fd9-8ce8-1d6357c4500d-cert\") pod \"ingress-canary-5gh5l\" (UID: \"634a7674-21bb-4fd9-8ce8-1d6357c4500d\") " pod="openshift-ingress-canary/ingress-canary-5gh5l" Jul 23 21:52:12.883053 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:12.882925 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3-installation-pull-secrets\") pod \"image-registry-fbbbb55d8-pz9t5\" (UID: \"88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3\") " pod="openshift-image-registry/image-registry-fbbbb55d8-pz9t5" Jul 23 21:52:12.887149 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:12.887123 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cs5z8\" (UniqueName: \"kubernetes.io/projected/634a7674-21bb-4fd9-8ce8-1d6357c4500d-kube-api-access-cs5z8\") pod \"ingress-canary-5gh5l\" (UID: \"634a7674-21bb-4fd9-8ce8-1d6357c4500d\") " pod="openshift-ingress-canary/ingress-canary-5gh5l" Jul 23 21:52:12.888162 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:12.888110 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3-bound-sa-token\") pod \"image-registry-fbbbb55d8-pz9t5\" (UID: \"88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3\") " pod="openshift-image-registry/image-registry-fbbbb55d8-pz9t5" Jul 23 21:52:12.888276 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:12.888158 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7tg7w\" (UniqueName: \"kubernetes.io/projected/88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3-kube-api-access-7tg7w\") pod \"image-registry-fbbbb55d8-pz9t5\" (UID: \"88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3\") " pod="openshift-image-registry/image-registry-fbbbb55d8-pz9t5" Jul 23 21:52:12.888754 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:12.888724 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rzwzj\" (UniqueName: \"kubernetes.io/projected/c8c008d7-7d6c-42e2-b9c6-4c9e9557c7c4-kube-api-access-rzwzj\") pod \"dns-default-6mgzw\" (UID: \"c8c008d7-7d6c-42e2-b9c6-4c9e9557c7c4\") " pod="openshift-dns/dns-default-6mgzw" Jul 23 21:52:12.982930 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:12.982898 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-5gh5l" Jul 23 21:52:13.154643 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:13.154606 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-5gh5l"] Jul 23 21:52:13.158820 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:52:13.158793 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod634a7674_21bb_4fd9_8ce8_1d6357c4500d.slice/crio-2674e36b9565d6d9c158297088c9458c1379b7203bcc61a51ba60d16e1d614b4 WatchSource:0}: Error finding container 2674e36b9565d6d9c158297088c9458c1379b7203bcc61a51ba60d16e1d614b4: Status 404 returned error can't find the container with id 2674e36b9565d6d9c158297088c9458c1379b7203bcc61a51ba60d16e1d614b4 Jul 23 21:52:13.383354 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:13.383264 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3-registry-tls\") pod \"image-registry-fbbbb55d8-pz9t5\" (UID: \"88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3\") " pod="openshift-image-registry/image-registry-fbbbb55d8-pz9t5" Jul 23 21:52:13.383354 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:13.383338 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c8c008d7-7d6c-42e2-b9c6-4c9e9557c7c4-metrics-tls\") pod \"dns-default-6mgzw\" (UID: \"c8c008d7-7d6c-42e2-b9c6-4c9e9557c7c4\") " pod="openshift-dns/dns-default-6mgzw" Jul 23 21:52:13.383694 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:52:13.383454 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 23 21:52:13.383694 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:52:13.383477 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-fbbbb55d8-pz9t5: secret "image-registry-tls" not found Jul 23 21:52:13.383694 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:52:13.383491 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 23 21:52:13.383694 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:52:13.383558 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3-registry-tls podName:88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3 nodeName:}" failed. No retries permitted until 2026-07-23 21:52:14.383522582 +0000 UTC m=+36.625430437 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3-registry-tls") pod "image-registry-fbbbb55d8-pz9t5" (UID: "88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3") : secret "image-registry-tls" not found Jul 23 21:52:13.383694 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:52:13.383576 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c8c008d7-7d6c-42e2-b9c6-4c9e9557c7c4-metrics-tls podName:c8c008d7-7d6c-42e2-b9c6-4c9e9557c7c4 nodeName:}" failed. No retries permitted until 2026-07-23 21:52:14.383567735 +0000 UTC m=+36.625475587 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c8c008d7-7d6c-42e2-b9c6-4c9e9557c7c4-metrics-tls") pod "dns-default-6mgzw" (UID: "c8c008d7-7d6c-42e2-b9c6-4c9e9557c7c4") : secret "dns-default-metrics-tls" not found Jul 23 21:52:13.424857 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:13.424821 2578 generic.go:356] "Generic (PLEG): container finished" podID="1e27638c-41d9-441b-8cea-dd134bf51985" containerID="b863e279b508a7f77a1d2020322efb14de656cd5596053af7c362fe58fa29793" exitCode=0 Jul 23 21:52:13.425007 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:13.424895 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ntjs5" event={"ID":"1e27638c-41d9-441b-8cea-dd134bf51985","Type":"ContainerDied","Data":"b863e279b508a7f77a1d2020322efb14de656cd5596053af7c362fe58fa29793"} Jul 23 21:52:13.426037 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:13.426003 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-5gh5l" event={"ID":"634a7674-21bb-4fd9-8ce8-1d6357c4500d","Type":"ContainerStarted","Data":"2674e36b9565d6d9c158297088c9458c1379b7203bcc61a51ba60d16e1d614b4"} Jul 23 21:52:14.249561 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:14.249359 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-26shl" Jul 23 21:52:14.250142 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:14.249389 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kl5st" Jul 23 21:52:14.250142 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:14.249435 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4cq9t" Jul 23 21:52:14.252617 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:14.252475 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 23 21:52:14.252617 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:14.252487 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 23 21:52:14.253385 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:14.253362 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-fxtft\"" Jul 23 21:52:14.253516 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:14.253424 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-n9zwh\"" Jul 23 21:52:14.253675 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:14.253597 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 23 21:52:14.253675 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:14.253658 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 23 21:52:14.391510 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:14.391476 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3-registry-tls\") pod \"image-registry-fbbbb55d8-pz9t5\" (UID: \"88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3\") " pod="openshift-image-registry/image-registry-fbbbb55d8-pz9t5" Jul 23 21:52:14.391711 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:52:14.391624 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 23 21:52:14.391711 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:52:14.391638 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-fbbbb55d8-pz9t5: secret "image-registry-tls" not found Jul 23 21:52:14.391711 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:14.391681 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c8c008d7-7d6c-42e2-b9c6-4c9e9557c7c4-metrics-tls\") pod \"dns-default-6mgzw\" (UID: \"c8c008d7-7d6c-42e2-b9c6-4c9e9557c7c4\") " pod="openshift-dns/dns-default-6mgzw" Jul 23 21:52:14.391711 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:52:14.391694 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3-registry-tls podName:88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3 nodeName:}" failed. No retries permitted until 2026-07-23 21:52:16.391675614 +0000 UTC m=+38.633583488 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3-registry-tls") pod "image-registry-fbbbb55d8-pz9t5" (UID: "88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3") : secret "image-registry-tls" not found Jul 23 21:52:14.391916 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:52:14.391752 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 23 21:52:14.391916 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:52:14.391786 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c8c008d7-7d6c-42e2-b9c6-4c9e9557c7c4-metrics-tls podName:c8c008d7-7d6c-42e2-b9c6-4c9e9557c7c4 nodeName:}" failed. No retries permitted until 2026-07-23 21:52:16.391775959 +0000 UTC m=+38.633683815 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c8c008d7-7d6c-42e2-b9c6-4c9e9557c7c4-metrics-tls") pod "dns-default-6mgzw" (UID: "c8c008d7-7d6c-42e2-b9c6-4c9e9557c7c4") : secret "dns-default-metrics-tls" not found Jul 23 21:52:14.431822 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:14.431792 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ntjs5" event={"ID":"1e27638c-41d9-441b-8cea-dd134bf51985","Type":"ContainerStarted","Data":"2dba59f626acbbd7892573e727ea826d69cacc3b500ed7eb7844a3c3ba999d11"} Jul 23 21:52:14.456968 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:14.456919 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-ntjs5" podStartSLOduration=8.053366354 podStartE2EDuration="36.456903559s" totalImagesPullingTime="28.403537205s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 21:51:38 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:52:14.455832804 +0000 UTC m=+36.697740677" watchObservedRunningTime="2026-07-23 21:52:14.456903559 +0000 UTC m=+36.698811433" Jul 23 21:52:16.412976 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:16.412942 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3-registry-tls\") pod \"image-registry-fbbbb55d8-pz9t5\" (UID: \"88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3\") " pod="openshift-image-registry/image-registry-fbbbb55d8-pz9t5" Jul 23 21:52:16.413456 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:16.413016 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c8c008d7-7d6c-42e2-b9c6-4c9e9557c7c4-metrics-tls\") pod \"dns-default-6mgzw\" (UID: \"c8c008d7-7d6c-42e2-b9c6-4c9e9557c7c4\") " pod="openshift-dns/dns-default-6mgzw" Jul 23 21:52:16.413456 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:52:16.413091 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 23 21:52:16.413456 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:52:16.413107 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-fbbbb55d8-pz9t5: secret "image-registry-tls" not found Jul 23 21:52:16.413456 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:52:16.413139 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 23 21:52:16.413456 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:52:16.413182 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3-registry-tls podName:88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3 nodeName:}" failed. No retries permitted until 2026-07-23 21:52:20.41316381 +0000 UTC m=+42.655071662 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3-registry-tls") pod "image-registry-fbbbb55d8-pz9t5" (UID: "88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3") : secret "image-registry-tls" not found Jul 23 21:52:16.413456 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:52:16.413196 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c8c008d7-7d6c-42e2-b9c6-4c9e9557c7c4-metrics-tls podName:c8c008d7-7d6c-42e2-b9c6-4c9e9557c7c4 nodeName:}" failed. No retries permitted until 2026-07-23 21:52:20.413190681 +0000 UTC m=+42.655098533 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c8c008d7-7d6c-42e2-b9c6-4c9e9557c7c4-metrics-tls") pod "dns-default-6mgzw" (UID: "c8c008d7-7d6c-42e2-b9c6-4c9e9557c7c4") : secret "dns-default-metrics-tls" not found Jul 23 21:52:16.438279 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:16.438246 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-5gh5l" event={"ID":"634a7674-21bb-4fd9-8ce8-1d6357c4500d","Type":"ContainerStarted","Data":"bbd33ad2ecd0b13784ac768f59098a58c9b57618a7713bd72b9fcf4a5443683d"} Jul 23 21:52:16.456066 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:16.456018 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-5gh5l" podStartSLOduration=2.017816493 podStartE2EDuration="4.456003244s" totalImagesPullingTime="2.438186751s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:52:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:52:16.455224574 +0000 UTC m=+38.697132451" watchObservedRunningTime="2026-07-23 21:52:16.456003244 +0000 UTC m=+38.697911130" Jul 23 21:52:18.429550 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:18.429514 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-fkqwr"] Jul 23 21:52:18.462095 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:18.462062 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-fkqwr"] Jul 23 21:52:18.462322 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:18.462177 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-fkqwr" Jul 23 21:52:18.465007 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:18.464984 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 23 21:52:18.465115 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:18.464986 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 23 21:52:18.465863 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:18.465846 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 23 21:52:18.465863 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:18.465853 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 23 21:52:18.465984 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:18.465896 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-jtqd5\"" Jul 23 21:52:18.630923 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:18.630888 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cth4l\" (UniqueName: \"kubernetes.io/projected/f1e7deb0-a030-4479-b82a-5dbbc864104b-kube-api-access-cth4l\") pod \"insights-runtime-extractor-fkqwr\" (UID: \"f1e7deb0-a030-4479-b82a-5dbbc864104b\") " pod="openshift-insights/insights-runtime-extractor-fkqwr" Jul 23 21:52:18.630923 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:18.630929 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f1e7deb0-a030-4479-b82a-5dbbc864104b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fkqwr\" (UID: \"f1e7deb0-a030-4479-b82a-5dbbc864104b\") " pod="openshift-insights/insights-runtime-extractor-fkqwr" Jul 23 21:52:18.631125 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:18.630948 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f1e7deb0-a030-4479-b82a-5dbbc864104b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fkqwr\" (UID: \"f1e7deb0-a030-4479-b82a-5dbbc864104b\") " pod="openshift-insights/insights-runtime-extractor-fkqwr" Jul 23 21:52:18.631125 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:18.631074 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f1e7deb0-a030-4479-b82a-5dbbc864104b-crio-socket\") pod \"insights-runtime-extractor-fkqwr\" (UID: \"f1e7deb0-a030-4479-b82a-5dbbc864104b\") " pod="openshift-insights/insights-runtime-extractor-fkqwr" Jul 23 21:52:18.631125 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:18.631104 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f1e7deb0-a030-4479-b82a-5dbbc864104b-data-volume\") pod \"insights-runtime-extractor-fkqwr\" (UID: \"f1e7deb0-a030-4479-b82a-5dbbc864104b\") " pod="openshift-insights/insights-runtime-extractor-fkqwr" Jul 23 21:52:18.731504 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:18.731433 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f1e7deb0-a030-4479-b82a-5dbbc864104b-crio-socket\") pod \"insights-runtime-extractor-fkqwr\" (UID: \"f1e7deb0-a030-4479-b82a-5dbbc864104b\") " pod="openshift-insights/insights-runtime-extractor-fkqwr" Jul 23 21:52:18.731504 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:18.731470 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f1e7deb0-a030-4479-b82a-5dbbc864104b-data-volume\") pod \"insights-runtime-extractor-fkqwr\" (UID: \"f1e7deb0-a030-4479-b82a-5dbbc864104b\") " pod="openshift-insights/insights-runtime-extractor-fkqwr" Jul 23 21:52:18.731504 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:18.731496 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cth4l\" (UniqueName: \"kubernetes.io/projected/f1e7deb0-a030-4479-b82a-5dbbc864104b-kube-api-access-cth4l\") pod \"insights-runtime-extractor-fkqwr\" (UID: \"f1e7deb0-a030-4479-b82a-5dbbc864104b\") " pod="openshift-insights/insights-runtime-extractor-fkqwr" Jul 23 21:52:18.731656 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:18.731521 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f1e7deb0-a030-4479-b82a-5dbbc864104b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fkqwr\" (UID: \"f1e7deb0-a030-4479-b82a-5dbbc864104b\") " pod="openshift-insights/insights-runtime-extractor-fkqwr" Jul 23 21:52:18.731656 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:18.731563 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f1e7deb0-a030-4479-b82a-5dbbc864104b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fkqwr\" (UID: \"f1e7deb0-a030-4479-b82a-5dbbc864104b\") " pod="openshift-insights/insights-runtime-extractor-fkqwr" Jul 23 21:52:18.731656 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:52:18.731622 2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 23 21:52:18.731737 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:52:18.731682 2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 23 21:52:18.731737 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:18.731684 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f1e7deb0-a030-4479-b82a-5dbbc864104b-crio-socket\") pod \"insights-runtime-extractor-fkqwr\" (UID: \"f1e7deb0-a030-4479-b82a-5dbbc864104b\") " pod="openshift-insights/insights-runtime-extractor-fkqwr" Jul 23 21:52:18.731737 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:52:18.731706 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f1e7deb0-a030-4479-b82a-5dbbc864104b-kube-rbac-proxy-cm podName:f1e7deb0-a030-4479-b82a-5dbbc864104b nodeName:}" failed. No retries permitted until 2026-07-23 21:52:19.231684228 +0000 UTC m=+41.473592104 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f1e7deb0-a030-4479-b82a-5dbbc864104b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-fkqwr" (UID: "f1e7deb0-a030-4479-b82a-5dbbc864104b") : configmap "kube-rbac-proxy" not found Jul 23 21:52:18.731737 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:52:18.731726 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1e7deb0-a030-4479-b82a-5dbbc864104b-insights-runtime-extractor-tls podName:f1e7deb0-a030-4479-b82a-5dbbc864104b nodeName:}" failed. No retries permitted until 2026-07-23 21:52:19.231716681 +0000 UTC m=+41.473624533 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f1e7deb0-a030-4479-b82a-5dbbc864104b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fkqwr" (UID: "f1e7deb0-a030-4479-b82a-5dbbc864104b") : secret "insights-runtime-extractor-tls" not found Jul 23 21:52:18.731858 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:18.731844 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f1e7deb0-a030-4479-b82a-5dbbc864104b-data-volume\") pod \"insights-runtime-extractor-fkqwr\" (UID: \"f1e7deb0-a030-4479-b82a-5dbbc864104b\") " pod="openshift-insights/insights-runtime-extractor-fkqwr" Jul 23 21:52:18.741650 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:18.741622 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cth4l\" (UniqueName: \"kubernetes.io/projected/f1e7deb0-a030-4479-b82a-5dbbc864104b-kube-api-access-cth4l\") pod \"insights-runtime-extractor-fkqwr\" (UID: \"f1e7deb0-a030-4479-b82a-5dbbc864104b\") " pod="openshift-insights/insights-runtime-extractor-fkqwr" Jul 23 21:52:18.878517 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:18.878483 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-gz5x4"] Jul 23 21:52:18.894264 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:18.894240 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-gz5x4" Jul 23 21:52:18.902502 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:18.902482 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 23 21:52:18.902641 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:18.902509 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 23 21:52:18.902641 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:18.902588 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-ttt9l\"" Jul 23 21:52:18.910908 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:18.910888 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-gz5x4"] Jul 23 21:52:19.034747 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:19.034649 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lp9d9\" (UniqueName: \"kubernetes.io/projected/8fbea949-92d3-469d-b6e1-7bf6c90aa4e8-kube-api-access-lp9d9\") pod \"migrator-7cc9c7ffd7-gz5x4\" (UID: \"8fbea949-92d3-469d-b6e1-7bf6c90aa4e8\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-gz5x4" Jul 23 21:52:19.135043 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:19.135007 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lp9d9\" (UniqueName: \"kubernetes.io/projected/8fbea949-92d3-469d-b6e1-7bf6c90aa4e8-kube-api-access-lp9d9\") pod \"migrator-7cc9c7ffd7-gz5x4\" (UID: \"8fbea949-92d3-469d-b6e1-7bf6c90aa4e8\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-gz5x4" Jul 23 21:52:19.144321 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:19.144296 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lp9d9\" (UniqueName: \"kubernetes.io/projected/8fbea949-92d3-469d-b6e1-7bf6c90aa4e8-kube-api-access-lp9d9\") pod \"migrator-7cc9c7ffd7-gz5x4\" (UID: \"8fbea949-92d3-469d-b6e1-7bf6c90aa4e8\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-gz5x4" Jul 23 21:52:19.202485 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:19.202451 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-gz5x4" Jul 23 21:52:19.236125 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:19.236092 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f1e7deb0-a030-4479-b82a-5dbbc864104b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fkqwr\" (UID: \"f1e7deb0-a030-4479-b82a-5dbbc864104b\") " pod="openshift-insights/insights-runtime-extractor-fkqwr" Jul 23 21:52:19.236125 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:19.236124 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f1e7deb0-a030-4479-b82a-5dbbc864104b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fkqwr\" (UID: \"f1e7deb0-a030-4479-b82a-5dbbc864104b\") " pod="openshift-insights/insights-runtime-extractor-fkqwr" Jul 23 21:52:19.236318 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:52:19.236216 2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 23 21:52:19.236318 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:52:19.236238 2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 23 21:52:19.236318 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:52:19.236273 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f1e7deb0-a030-4479-b82a-5dbbc864104b-kube-rbac-proxy-cm podName:f1e7deb0-a030-4479-b82a-5dbbc864104b nodeName:}" failed. No retries permitted until 2026-07-23 21:52:20.236257793 +0000 UTC m=+42.478165645 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f1e7deb0-a030-4479-b82a-5dbbc864104b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-fkqwr" (UID: "f1e7deb0-a030-4479-b82a-5dbbc864104b") : configmap "kube-rbac-proxy" not found Jul 23 21:52:19.236417 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:52:19.236334 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1e7deb0-a030-4479-b82a-5dbbc864104b-insights-runtime-extractor-tls podName:f1e7deb0-a030-4479-b82a-5dbbc864104b nodeName:}" failed. No retries permitted until 2026-07-23 21:52:20.236323741 +0000 UTC m=+42.478231594 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f1e7deb0-a030-4479-b82a-5dbbc864104b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fkqwr" (UID: "f1e7deb0-a030-4479-b82a-5dbbc864104b") : secret "insights-runtime-extractor-tls" not found Jul 23 21:52:19.332102 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:19.332061 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-gz5x4"] Jul 23 21:52:19.335075 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:52:19.335048 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8fbea949_92d3_469d_b6e1_7bf6c90aa4e8.slice/crio-6c4777d396ecc880b6ee5c13709e4f909e49c8e3252d436bb84ea3e463f92332 WatchSource:0}: Error finding container 6c4777d396ecc880b6ee5c13709e4f909e49c8e3252d436bb84ea3e463f92332: Status 404 returned error can't find the container with id 6c4777d396ecc880b6ee5c13709e4f909e49c8e3252d436bb84ea3e463f92332 Jul 23 21:52:19.445576 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:19.445533 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-gz5x4" event={"ID":"8fbea949-92d3-469d-b6e1-7bf6c90aa4e8","Type":"ContainerStarted","Data":"6c4777d396ecc880b6ee5c13709e4f909e49c8e3252d436bb84ea3e463f92332"} Jul 23 21:52:20.231338 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:20.231310 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-w6jzd_4e42ce8b-422d-4281-b12e-6371c9744f8a/dns-node-resolver/0.log" Jul 23 21:52:20.244726 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:20.244693 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f1e7deb0-a030-4479-b82a-5dbbc864104b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fkqwr\" (UID: \"f1e7deb0-a030-4479-b82a-5dbbc864104b\") " pod="openshift-insights/insights-runtime-extractor-fkqwr" Jul 23 21:52:20.244894 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:20.244741 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f1e7deb0-a030-4479-b82a-5dbbc864104b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fkqwr\" (UID: \"f1e7deb0-a030-4479-b82a-5dbbc864104b\") " pod="openshift-insights/insights-runtime-extractor-fkqwr" Jul 23 21:52:20.244894 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:52:20.244841 2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 23 21:52:20.245014 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:52:20.244913 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f1e7deb0-a030-4479-b82a-5dbbc864104b-kube-rbac-proxy-cm podName:f1e7deb0-a030-4479-b82a-5dbbc864104b nodeName:}" failed. No retries permitted until 2026-07-23 21:52:22.244894034 +0000 UTC m=+44.486801887 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f1e7deb0-a030-4479-b82a-5dbbc864104b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-fkqwr" (UID: "f1e7deb0-a030-4479-b82a-5dbbc864104b") : configmap "kube-rbac-proxy" not found Jul 23 21:52:20.245014 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:52:20.244938 2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 23 21:52:20.245014 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:52:20.245001 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1e7deb0-a030-4479-b82a-5dbbc864104b-insights-runtime-extractor-tls podName:f1e7deb0-a030-4479-b82a-5dbbc864104b nodeName:}" failed. No retries permitted until 2026-07-23 21:52:22.244983141 +0000 UTC m=+44.486891014 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f1e7deb0-a030-4479-b82a-5dbbc864104b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fkqwr" (UID: "f1e7deb0-a030-4479-b82a-5dbbc864104b") : secret "insights-runtime-extractor-tls" not found Jul 23 21:52:20.446452 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:20.446424 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3-registry-tls\") pod \"image-registry-fbbbb55d8-pz9t5\" (UID: \"88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3\") " pod="openshift-image-registry/image-registry-fbbbb55d8-pz9t5" Jul 23 21:52:20.446924 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:20.446486 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c8c008d7-7d6c-42e2-b9c6-4c9e9557c7c4-metrics-tls\") pod \"dns-default-6mgzw\" (UID: \"c8c008d7-7d6c-42e2-b9c6-4c9e9557c7c4\") " pod="openshift-dns/dns-default-6mgzw" Jul 23 21:52:20.446924 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:52:20.446593 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 23 21:52:20.446924 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:52:20.446608 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-fbbbb55d8-pz9t5: secret "image-registry-tls" not found Jul 23 21:52:20.446924 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:52:20.446648 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 23 21:52:20.446924 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:52:20.446663 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3-registry-tls podName:88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3 nodeName:}" failed. No retries permitted until 2026-07-23 21:52:28.446648999 +0000 UTC m=+50.688556851 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3-registry-tls") pod "image-registry-fbbbb55d8-pz9t5" (UID: "88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3") : secret "image-registry-tls" not found Jul 23 21:52:20.446924 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:52:20.446694 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c8c008d7-7d6c-42e2-b9c6-4c9e9557c7c4-metrics-tls podName:c8c008d7-7d6c-42e2-b9c6-4c9e9557c7c4 nodeName:}" failed. No retries permitted until 2026-07-23 21:52:28.446680965 +0000 UTC m=+50.688588821 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c8c008d7-7d6c-42e2-b9c6-4c9e9557c7c4-metrics-tls") pod "dns-default-6mgzw" (UID: "c8c008d7-7d6c-42e2-b9c6-4c9e9557c7c4") : secret "dns-default-metrics-tls" not found Jul 23 21:52:21.031172 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:21.031154 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-nt77r_49757977-27cf-4177-b295-c830e4a71b03/node-ca/0.log" Jul 23 21:52:21.192778 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:21.192697 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-58598695f7-mf5hw"] Jul 23 21:52:21.208719 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:21.208693 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-58598695f7-mf5hw"] Jul 23 21:52:21.208851 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:21.208805 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-58598695f7-mf5hw" Jul 23 21:52:21.211250 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:21.211226 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 23 21:52:21.211376 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:21.211237 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 23 21:52:21.211376 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:21.211277 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 23 21:52:21.212047 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:21.212032 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 23 21:52:21.212121 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:21.212108 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-s2vpp\"" Jul 23 21:52:21.353261 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:21.353228 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6wrz9\" (UniqueName: \"kubernetes.io/projected/c2589998-6427-46b2-93bc-23c33e2dc0e6-kube-api-access-6wrz9\") pod \"service-ca-58598695f7-mf5hw\" (UID: \"c2589998-6427-46b2-93bc-23c33e2dc0e6\") " pod="openshift-service-ca/service-ca-58598695f7-mf5hw" Jul 23 21:52:21.353261 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:21.353264 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/c2589998-6427-46b2-93bc-23c33e2dc0e6-signing-key\") pod \"service-ca-58598695f7-mf5hw\" (UID: \"c2589998-6427-46b2-93bc-23c33e2dc0e6\") " pod="openshift-service-ca/service-ca-58598695f7-mf5hw" Jul 23 21:52:21.353460 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:21.353296 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/c2589998-6427-46b2-93bc-23c33e2dc0e6-signing-cabundle\") pod \"service-ca-58598695f7-mf5hw\" (UID: \"c2589998-6427-46b2-93bc-23c33e2dc0e6\") " pod="openshift-service-ca/service-ca-58598695f7-mf5hw" Jul 23 21:52:21.452171 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:21.452080 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-gz5x4" event={"ID":"8fbea949-92d3-469d-b6e1-7bf6c90aa4e8","Type":"ContainerStarted","Data":"39a0c45cfe047fd04cb705f5485bf4bced5797728e1c5e73b91ca311681709bd"} Jul 23 21:52:21.452171 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:21.452134 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-gz5x4" event={"ID":"8fbea949-92d3-469d-b6e1-7bf6c90aa4e8","Type":"ContainerStarted","Data":"2db936daf5832f3a79aff5b8b9ee348c8158b093f2630298b2babbd798f2ca9e"} Jul 23 21:52:21.454622 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:21.454605 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/c2589998-6427-46b2-93bc-23c33e2dc0e6-signing-key\") pod \"service-ca-58598695f7-mf5hw\" (UID: \"c2589998-6427-46b2-93bc-23c33e2dc0e6\") " pod="openshift-service-ca/service-ca-58598695f7-mf5hw" Jul 23 21:52:21.454674 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:21.454652 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/c2589998-6427-46b2-93bc-23c33e2dc0e6-signing-cabundle\") pod \"service-ca-58598695f7-mf5hw\" (UID: \"c2589998-6427-46b2-93bc-23c33e2dc0e6\") " pod="openshift-service-ca/service-ca-58598695f7-mf5hw" Jul 23 21:52:21.454866 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:21.454847 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6wrz9\" (UniqueName: \"kubernetes.io/projected/c2589998-6427-46b2-93bc-23c33e2dc0e6-kube-api-access-6wrz9\") pod \"service-ca-58598695f7-mf5hw\" (UID: \"c2589998-6427-46b2-93bc-23c33e2dc0e6\") " pod="openshift-service-ca/service-ca-58598695f7-mf5hw" Jul 23 21:52:21.455389 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:21.455361 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/c2589998-6427-46b2-93bc-23c33e2dc0e6-signing-cabundle\") pod \"service-ca-58598695f7-mf5hw\" (UID: \"c2589998-6427-46b2-93bc-23c33e2dc0e6\") " pod="openshift-service-ca/service-ca-58598695f7-mf5hw" Jul 23 21:52:21.457177 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:21.457161 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/c2589998-6427-46b2-93bc-23c33e2dc0e6-signing-key\") pod \"service-ca-58598695f7-mf5hw\" (UID: \"c2589998-6427-46b2-93bc-23c33e2dc0e6\") " pod="openshift-service-ca/service-ca-58598695f7-mf5hw" Jul 23 21:52:21.466408 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:21.466384 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6wrz9\" (UniqueName: \"kubernetes.io/projected/c2589998-6427-46b2-93bc-23c33e2dc0e6-kube-api-access-6wrz9\") pod \"service-ca-58598695f7-mf5hw\" (UID: \"c2589998-6427-46b2-93bc-23c33e2dc0e6\") " pod="openshift-service-ca/service-ca-58598695f7-mf5hw" Jul 23 21:52:21.468665 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:21.468622 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-gz5x4" podStartSLOduration=1.983464447 podStartE2EDuration="3.468606293s" totalImagesPullingTime="1.485141846s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:52:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:52:21.46748741 +0000 UTC m=+43.709395285" watchObservedRunningTime="2026-07-23 21:52:21.468606293 +0000 UTC m=+43.710514167" Jul 23 21:52:21.517020 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:21.516991 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-58598695f7-mf5hw" Jul 23 21:52:21.630996 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:21.630970 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-5gh5l_634a7674-21bb-4fd9-8ce8-1d6357c4500d/serve-healthcheck-canary/0.log" Jul 23 21:52:21.662351 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:21.662324 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-58598695f7-mf5hw"] Jul 23 21:52:21.665927 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:52:21.665902 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2589998_6427_46b2_93bc_23c33e2dc0e6.slice/crio-3b5cd3a663a517f5ea523c88df01a12be910db061e869b0cf1dd115ec7eefab6 WatchSource:0}: Error finding container 3b5cd3a663a517f5ea523c88df01a12be910db061e869b0cf1dd115ec7eefab6: Status 404 returned error can't find the container with id 3b5cd3a663a517f5ea523c88df01a12be910db061e869b0cf1dd115ec7eefab6 Jul 23 21:52:22.262074 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:22.262040 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f1e7deb0-a030-4479-b82a-5dbbc864104b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fkqwr\" (UID: \"f1e7deb0-a030-4479-b82a-5dbbc864104b\") " pod="openshift-insights/insights-runtime-extractor-fkqwr" Jul 23 21:52:22.262074 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:22.262079 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f1e7deb0-a030-4479-b82a-5dbbc864104b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fkqwr\" (UID: \"f1e7deb0-a030-4479-b82a-5dbbc864104b\") " pod="openshift-insights/insights-runtime-extractor-fkqwr" Jul 23 21:52:22.262318 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:52:22.262178 2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 23 21:52:22.262318 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:52:22.262196 2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 23 21:52:22.262318 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:52:22.262256 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f1e7deb0-a030-4479-b82a-5dbbc864104b-kube-rbac-proxy-cm podName:f1e7deb0-a030-4479-b82a-5dbbc864104b nodeName:}" failed. No retries permitted until 2026-07-23 21:52:26.26223514 +0000 UTC m=+48.504142992 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f1e7deb0-a030-4479-b82a-5dbbc864104b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-fkqwr" (UID: "f1e7deb0-a030-4479-b82a-5dbbc864104b") : configmap "kube-rbac-proxy" not found Jul 23 21:52:22.262468 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:52:22.262351 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1e7deb0-a030-4479-b82a-5dbbc864104b-insights-runtime-extractor-tls podName:f1e7deb0-a030-4479-b82a-5dbbc864104b nodeName:}" failed. No retries permitted until 2026-07-23 21:52:26.262309412 +0000 UTC m=+48.504217264 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f1e7deb0-a030-4479-b82a-5dbbc864104b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fkqwr" (UID: "f1e7deb0-a030-4479-b82a-5dbbc864104b") : secret "insights-runtime-extractor-tls" not found Jul 23 21:52:22.455894 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:22.455853 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-58598695f7-mf5hw" event={"ID":"c2589998-6427-46b2-93bc-23c33e2dc0e6","Type":"ContainerStarted","Data":"3b5cd3a663a517f5ea523c88df01a12be910db061e869b0cf1dd115ec7eefab6"} Jul 23 21:52:24.462628 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:24.462595 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-58598695f7-mf5hw" event={"ID":"c2589998-6427-46b2-93bc-23c33e2dc0e6","Type":"ContainerStarted","Data":"b97e04fbf7dec498b801ccaaa64875985c13ccae83bae8fce9b213035af79261"} Jul 23 21:52:26.297842 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:26.297809 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f1e7deb0-a030-4479-b82a-5dbbc864104b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fkqwr\" (UID: \"f1e7deb0-a030-4479-b82a-5dbbc864104b\") " pod="openshift-insights/insights-runtime-extractor-fkqwr" Jul 23 21:52:26.297842 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:26.297844 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f1e7deb0-a030-4479-b82a-5dbbc864104b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fkqwr\" (UID: \"f1e7deb0-a030-4479-b82a-5dbbc864104b\") " pod="openshift-insights/insights-runtime-extractor-fkqwr" Jul 23 21:52:26.298392 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:52:26.297932 2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 23 21:52:26.298392 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:52:26.297946 2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 23 21:52:26.298392 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:52:26.297996 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f1e7deb0-a030-4479-b82a-5dbbc864104b-kube-rbac-proxy-cm podName:f1e7deb0-a030-4479-b82a-5dbbc864104b nodeName:}" failed. No retries permitted until 2026-07-23 21:52:34.297981158 +0000 UTC m=+56.539889014 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f1e7deb0-a030-4479-b82a-5dbbc864104b-kube-rbac-proxy-cm") pod "insights-runtime-extractor-fkqwr" (UID: "f1e7deb0-a030-4479-b82a-5dbbc864104b") : configmap "kube-rbac-proxy" not found Jul 23 21:52:26.298392 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:52:26.298012 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1e7deb0-a030-4479-b82a-5dbbc864104b-insights-runtime-extractor-tls podName:f1e7deb0-a030-4479-b82a-5dbbc864104b nodeName:}" failed. No retries permitted until 2026-07-23 21:52:34.298004349 +0000 UTC m=+56.539912201 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f1e7deb0-a030-4479-b82a-5dbbc864104b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fkqwr" (UID: "f1e7deb0-a030-4479-b82a-5dbbc864104b") : secret "insights-runtime-extractor-tls" not found Jul 23 21:52:28.518492 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:28.518458 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3-registry-tls\") pod \"image-registry-fbbbb55d8-pz9t5\" (UID: \"88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3\") " pod="openshift-image-registry/image-registry-fbbbb55d8-pz9t5" Jul 23 21:52:28.518863 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:28.518509 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c8c008d7-7d6c-42e2-b9c6-4c9e9557c7c4-metrics-tls\") pod \"dns-default-6mgzw\" (UID: \"c8c008d7-7d6c-42e2-b9c6-4c9e9557c7c4\") " pod="openshift-dns/dns-default-6mgzw" Jul 23 21:52:28.520875 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:28.520854 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c8c008d7-7d6c-42e2-b9c6-4c9e9557c7c4-metrics-tls\") pod \"dns-default-6mgzw\" (UID: \"c8c008d7-7d6c-42e2-b9c6-4c9e9557c7c4\") " pod="openshift-dns/dns-default-6mgzw" Jul 23 21:52:28.521032 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:28.521014 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3-registry-tls\") pod \"image-registry-fbbbb55d8-pz9t5\" (UID: \"88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3\") " pod="openshift-image-registry/image-registry-fbbbb55d8-pz9t5" Jul 23 21:52:28.576663 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:28.576632 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-fbbbb55d8-pz9t5" Jul 23 21:52:28.589736 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:28.589704 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-6mgzw" Jul 23 21:52:28.739889 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:28.739837 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-58598695f7-mf5hw" podStartSLOduration=5.751791774 podStartE2EDuration="7.739817873s" totalImagesPullingTime="1.988026099s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:52:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:52:24.483992895 +0000 UTC m=+46.725900769" watchObservedRunningTime="2026-07-23 21:52:28.739817873 +0000 UTC m=+50.981725741" Jul 23 21:52:28.740251 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:28.740235 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-fbbbb55d8-pz9t5"] Jul 23 21:52:28.743855 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:52:28.743830 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88186fa0_4ba3_4bbc_a9ba_7c13d3e125f3.slice/crio-25ad7a6ce3d3cea00d0887c5b40aff62b46e72ab2fd922696ae119086a6c264c WatchSource:0}: Error finding container 25ad7a6ce3d3cea00d0887c5b40aff62b46e72ab2fd922696ae119086a6c264c: Status 404 returned error can't find the container with id 25ad7a6ce3d3cea00d0887c5b40aff62b46e72ab2fd922696ae119086a6c264c Jul 23 21:52:28.752616 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:28.752593 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-6mgzw"] Jul 23 21:52:28.755790 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:52:28.755758 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8c008d7_7d6c_42e2_b9c6_4c9e9557c7c4.slice/crio-dbb3044e9813ce0cd8bfdc5db5bfa79e796b8fe5b37af143fd6789c86cbaf845 WatchSource:0}: Error finding container dbb3044e9813ce0cd8bfdc5db5bfa79e796b8fe5b37af143fd6789c86cbaf845: Status 404 returned error can't find the container with id dbb3044e9813ce0cd8bfdc5db5bfa79e796b8fe5b37af143fd6789c86cbaf845 Jul 23 21:52:29.477488 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:29.477455 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-fbbbb55d8-pz9t5" event={"ID":"88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3","Type":"ContainerStarted","Data":"25ff99f7d0736b7918e8bcdec808981d4771823aa557e088283a6ef8beca0691"} Jul 23 21:52:29.477703 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:29.477509 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-fbbbb55d8-pz9t5" event={"ID":"88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3","Type":"ContainerStarted","Data":"25ad7a6ce3d3cea00d0887c5b40aff62b46e72ab2fd922696ae119086a6c264c"} Jul 23 21:52:29.477703 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:29.477591 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-fbbbb55d8-pz9t5" Jul 23 21:52:29.478716 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:29.478676 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-6mgzw" event={"ID":"c8c008d7-7d6c-42e2-b9c6-4c9e9557c7c4","Type":"ContainerStarted","Data":"dbb3044e9813ce0cd8bfdc5db5bfa79e796b8fe5b37af143fd6789c86cbaf845"} Jul 23 21:52:29.497706 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:29.497647 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-fbbbb55d8-pz9t5" podStartSLOduration=19.497628883 podStartE2EDuration="19.497628883s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:52:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:52:29.496877338 +0000 UTC m=+51.738785215" watchObservedRunningTime="2026-07-23 21:52:29.497628883 +0000 UTC m=+51.739536759" Jul 23 21:52:30.483186 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:30.483154 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-6mgzw" event={"ID":"c8c008d7-7d6c-42e2-b9c6-4c9e9557c7c4","Type":"ContainerStarted","Data":"3a97d8d373585180758d496e26a58f5b24b078db46c0823f8c066df94d6de364"} Jul 23 21:52:30.483696 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:30.483206 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-6mgzw" event={"ID":"c8c008d7-7d6c-42e2-b9c6-4c9e9557c7c4","Type":"ContainerStarted","Data":"a3b708fd118f7939eb081c3598187d26c453858f497abd53c1065ee53cbdeee7"} Jul 23 21:52:30.501089 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:30.501007 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-6mgzw" podStartSLOduration=17.006528104 podStartE2EDuration="18.500991917s" totalImagesPullingTime="1.494463813s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:52:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:52:30.500168011 +0000 UTC m=+52.742075885" watchObservedRunningTime="2026-07-23 21:52:30.500991917 +0000 UTC m=+52.742899769" Jul 23 21:52:31.485902 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:31.485872 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-6mgzw" Jul 23 21:52:34.367529 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:34.367486 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f1e7deb0-a030-4479-b82a-5dbbc864104b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fkqwr\" (UID: \"f1e7deb0-a030-4479-b82a-5dbbc864104b\") " pod="openshift-insights/insights-runtime-extractor-fkqwr" Jul 23 21:52:34.368020 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:34.367557 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f1e7deb0-a030-4479-b82a-5dbbc864104b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fkqwr\" (UID: \"f1e7deb0-a030-4479-b82a-5dbbc864104b\") " pod="openshift-insights/insights-runtime-extractor-fkqwr" Jul 23 21:52:34.368020 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:52:34.367702 2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 23 21:52:34.368020 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:52:34.367776 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1e7deb0-a030-4479-b82a-5dbbc864104b-insights-runtime-extractor-tls podName:f1e7deb0-a030-4479-b82a-5dbbc864104b nodeName:}" failed. No retries permitted until 2026-07-23 21:52:50.367755005 +0000 UTC m=+72.609662863 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f1e7deb0-a030-4479-b82a-5dbbc864104b-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fkqwr" (UID: "f1e7deb0-a030-4479-b82a-5dbbc864104b") : secret "insights-runtime-extractor-tls" not found Jul 23 21:52:34.368020 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:34.367974 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f1e7deb0-a030-4479-b82a-5dbbc864104b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fkqwr\" (UID: \"f1e7deb0-a030-4479-b82a-5dbbc864104b\") " pod="openshift-insights/insights-runtime-extractor-fkqwr" Jul 23 21:52:39.420444 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:39.420415 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7jkjm" Jul 23 21:52:40.791912 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:40.791877 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-qkfwc"] Jul 23 21:52:40.795282 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:40.795261 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-qkfwc" Jul 23 21:52:40.799023 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:40.799000 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 23 21:52:40.800020 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:40.799999 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-9jx8z\"" Jul 23 21:52:40.800020 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:40.800024 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 23 21:52:40.814593 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:40.814567 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5f69cf99d4-r6sr7"] Jul 23 21:52:40.816323 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:40.816308 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5f69cf99d4-r6sr7" Jul 23 21:52:40.818858 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:40.818837 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 23 21:52:40.818947 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:40.818899 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-ptlq4\"" Jul 23 21:52:40.818947 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:40.818904 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 23 21:52:40.818947 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:40.818918 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 23 21:52:40.819144 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:40.818922 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 23 21:52:40.823660 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:40.823639 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-qkfwc"] Jul 23 21:52:40.832972 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:40.832946 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5f69cf99d4-r6sr7"] Jul 23 21:52:40.847532 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:40.847511 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-fbbbb55d8-pz9t5"] Jul 23 21:52:40.852008 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:40.851990 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-5bc67dcd46-zxc85"] Jul 23 21:52:40.854069 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:40.854051 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-5bc67dcd46-zxc85" Jul 23 21:52:40.856660 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:40.856644 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 23 21:52:40.857134 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:40.857120 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-wh69k\"" Jul 23 21:52:40.858709 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:40.858692 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 23 21:52:40.869042 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:40.869020 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-5bc67dcd46-zxc85"] Jul 23 21:52:40.915516 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:40.915480 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vf7ls\" (UniqueName: \"kubernetes.io/projected/ca691a1f-6865-40b2-96fe-9bb8782b7e18-kube-api-access-vf7ls\") pod \"downloads-5bc67dcd46-zxc85\" (UID: \"ca691a1f-6865-40b2-96fe-9bb8782b7e18\") " pod="openshift-console/downloads-5bc67dcd46-zxc85" Jul 23 21:52:40.915724 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:40.915533 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kq2vd\" (UniqueName: \"kubernetes.io/projected/dd1401ba-e227-43ad-ac19-8b1d4e643f20-kube-api-access-kq2vd\") pod \"managed-serviceaccount-addon-agent-5f69cf99d4-r6sr7\" (UID: \"dd1401ba-e227-43ad-ac19-8b1d4e643f20\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5f69cf99d4-r6sr7" Jul 23 21:52:40.915724 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:40.915674 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/3dd824e2-8ab1-4a94-8d99-a0eec78aa59d-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-qkfwc\" (UID: \"3dd824e2-8ab1-4a94-8d99-a0eec78aa59d\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-qkfwc" Jul 23 21:52:40.915824 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:40.915727 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3dd824e2-8ab1-4a94-8d99-a0eec78aa59d-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-qkfwc\" (UID: \"3dd824e2-8ab1-4a94-8d99-a0eec78aa59d\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-qkfwc" Jul 23 21:52:40.915824 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:40.915777 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/dd1401ba-e227-43ad-ac19-8b1d4e643f20-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5f69cf99d4-r6sr7\" (UID: \"dd1401ba-e227-43ad-ac19-8b1d4e643f20\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5f69cf99d4-r6sr7" Jul 23 21:52:40.950075 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:40.950037 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5899879c95-5w8d7"] Jul 23 21:52:40.952095 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:40.952073 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5899879c95-5w8d7" Jul 23 21:52:40.974711 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:40.974680 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5899879c95-5w8d7"] Jul 23 21:52:41.016512 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:41.016474 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3dd824e2-8ab1-4a94-8d99-a0eec78aa59d-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-qkfwc\" (UID: \"3dd824e2-8ab1-4a94-8d99-a0eec78aa59d\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-qkfwc" Jul 23 21:52:41.016512 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:41.016513 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/dd1401ba-e227-43ad-ac19-8b1d4e643f20-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5f69cf99d4-r6sr7\" (UID: \"dd1401ba-e227-43ad-ac19-8b1d4e643f20\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5f69cf99d4-r6sr7" Jul 23 21:52:41.016795 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:41.016554 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0936cb55-c4b0-484c-8ff0-6f457ce7e5b1-registry-certificates\") pod \"image-registry-5899879c95-5w8d7\" (UID: \"0936cb55-c4b0-484c-8ff0-6f457ce7e5b1\") " pod="openshift-image-registry/image-registry-5899879c95-5w8d7" Jul 23 21:52:41.016795 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:41.016660 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0936cb55-c4b0-484c-8ff0-6f457ce7e5b1-ca-trust-extracted\") pod \"image-registry-5899879c95-5w8d7\" (UID: \"0936cb55-c4b0-484c-8ff0-6f457ce7e5b1\") " pod="openshift-image-registry/image-registry-5899879c95-5w8d7" Jul 23 21:52:41.016795 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:41.016705 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0936cb55-c4b0-484c-8ff0-6f457ce7e5b1-bound-sa-token\") pod \"image-registry-5899879c95-5w8d7\" (UID: \"0936cb55-c4b0-484c-8ff0-6f457ce7e5b1\") " pod="openshift-image-registry/image-registry-5899879c95-5w8d7" Jul 23 21:52:41.016795 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:41.016733 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vf7ls\" (UniqueName: \"kubernetes.io/projected/ca691a1f-6865-40b2-96fe-9bb8782b7e18-kube-api-access-vf7ls\") pod \"downloads-5bc67dcd46-zxc85\" (UID: \"ca691a1f-6865-40b2-96fe-9bb8782b7e18\") " pod="openshift-console/downloads-5bc67dcd46-zxc85" Jul 23 21:52:41.016795 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:41.016761 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0936cb55-c4b0-484c-8ff0-6f457ce7e5b1-image-registry-private-configuration\") pod \"image-registry-5899879c95-5w8d7\" (UID: \"0936cb55-c4b0-484c-8ff0-6f457ce7e5b1\") " pod="openshift-image-registry/image-registry-5899879c95-5w8d7" Jul 23 21:52:41.017038 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:41.016802 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kq2vd\" (UniqueName: \"kubernetes.io/projected/dd1401ba-e227-43ad-ac19-8b1d4e643f20-kube-api-access-kq2vd\") pod \"managed-serviceaccount-addon-agent-5f69cf99d4-r6sr7\" (UID: \"dd1401ba-e227-43ad-ac19-8b1d4e643f20\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5f69cf99d4-r6sr7" Jul 23 21:52:41.017038 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:41.016824 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s2znj\" (UniqueName: \"kubernetes.io/projected/0936cb55-c4b0-484c-8ff0-6f457ce7e5b1-kube-api-access-s2znj\") pod \"image-registry-5899879c95-5w8d7\" (UID: \"0936cb55-c4b0-484c-8ff0-6f457ce7e5b1\") " pod="openshift-image-registry/image-registry-5899879c95-5w8d7" Jul 23 21:52:41.017038 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:41.016848 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0936cb55-c4b0-484c-8ff0-6f457ce7e5b1-installation-pull-secrets\") pod \"image-registry-5899879c95-5w8d7\" (UID: \"0936cb55-c4b0-484c-8ff0-6f457ce7e5b1\") " pod="openshift-image-registry/image-registry-5899879c95-5w8d7" Jul 23 21:52:41.017038 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:41.017007 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0936cb55-c4b0-484c-8ff0-6f457ce7e5b1-trusted-ca\") pod \"image-registry-5899879c95-5w8d7\" (UID: \"0936cb55-c4b0-484c-8ff0-6f457ce7e5b1\") " pod="openshift-image-registry/image-registry-5899879c95-5w8d7" Jul 23 21:52:41.017201 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:41.017058 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/3dd824e2-8ab1-4a94-8d99-a0eec78aa59d-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-qkfwc\" (UID: \"3dd824e2-8ab1-4a94-8d99-a0eec78aa59d\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-qkfwc" Jul 23 21:52:41.017201 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:41.017086 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0936cb55-c4b0-484c-8ff0-6f457ce7e5b1-registry-tls\") pod \"image-registry-5899879c95-5w8d7\" (UID: \"0936cb55-c4b0-484c-8ff0-6f457ce7e5b1\") " pod="openshift-image-registry/image-registry-5899879c95-5w8d7" Jul 23 21:52:41.017652 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:41.017637 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/3dd824e2-8ab1-4a94-8d99-a0eec78aa59d-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-qkfwc\" (UID: \"3dd824e2-8ab1-4a94-8d99-a0eec78aa59d\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-qkfwc" Jul 23 21:52:41.019120 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:41.019099 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3dd824e2-8ab1-4a94-8d99-a0eec78aa59d-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-qkfwc\" (UID: \"3dd824e2-8ab1-4a94-8d99-a0eec78aa59d\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-qkfwc" Jul 23 21:52:41.019218 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:41.019174 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/dd1401ba-e227-43ad-ac19-8b1d4e643f20-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5f69cf99d4-r6sr7\" (UID: \"dd1401ba-e227-43ad-ac19-8b1d4e643f20\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5f69cf99d4-r6sr7" Jul 23 21:52:41.026038 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:41.026012 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kq2vd\" (UniqueName: \"kubernetes.io/projected/dd1401ba-e227-43ad-ac19-8b1d4e643f20-kube-api-access-kq2vd\") pod \"managed-serviceaccount-addon-agent-5f69cf99d4-r6sr7\" (UID: \"dd1401ba-e227-43ad-ac19-8b1d4e643f20\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5f69cf99d4-r6sr7" Jul 23 21:52:41.026246 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:41.026228 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vf7ls\" (UniqueName: \"kubernetes.io/projected/ca691a1f-6865-40b2-96fe-9bb8782b7e18-kube-api-access-vf7ls\") pod \"downloads-5bc67dcd46-zxc85\" (UID: \"ca691a1f-6865-40b2-96fe-9bb8782b7e18\") " pod="openshift-console/downloads-5bc67dcd46-zxc85" Jul 23 21:52:41.104302 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:41.104270 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-qkfwc" Jul 23 21:52:41.117712 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:41.117675 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0936cb55-c4b0-484c-8ff0-6f457ce7e5b1-trusted-ca\") pod \"image-registry-5899879c95-5w8d7\" (UID: \"0936cb55-c4b0-484c-8ff0-6f457ce7e5b1\") " pod="openshift-image-registry/image-registry-5899879c95-5w8d7" Jul 23 21:52:41.117827 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:41.117714 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0936cb55-c4b0-484c-8ff0-6f457ce7e5b1-registry-tls\") pod \"image-registry-5899879c95-5w8d7\" (UID: \"0936cb55-c4b0-484c-8ff0-6f457ce7e5b1\") " pod="openshift-image-registry/image-registry-5899879c95-5w8d7" Jul 23 21:52:41.117827 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:41.117760 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0936cb55-c4b0-484c-8ff0-6f457ce7e5b1-registry-certificates\") pod \"image-registry-5899879c95-5w8d7\" (UID: \"0936cb55-c4b0-484c-8ff0-6f457ce7e5b1\") " pod="openshift-image-registry/image-registry-5899879c95-5w8d7" Jul 23 21:52:41.117827 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:41.117777 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0936cb55-c4b0-484c-8ff0-6f457ce7e5b1-ca-trust-extracted\") pod \"image-registry-5899879c95-5w8d7\" (UID: \"0936cb55-c4b0-484c-8ff0-6f457ce7e5b1\") " pod="openshift-image-registry/image-registry-5899879c95-5w8d7" Jul 23 21:52:41.117827 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:41.117819 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0936cb55-c4b0-484c-8ff0-6f457ce7e5b1-bound-sa-token\") pod \"image-registry-5899879c95-5w8d7\" (UID: \"0936cb55-c4b0-484c-8ff0-6f457ce7e5b1\") " pod="openshift-image-registry/image-registry-5899879c95-5w8d7" Jul 23 21:52:41.118034 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:41.117847 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0936cb55-c4b0-484c-8ff0-6f457ce7e5b1-image-registry-private-configuration\") pod \"image-registry-5899879c95-5w8d7\" (UID: \"0936cb55-c4b0-484c-8ff0-6f457ce7e5b1\") " pod="openshift-image-registry/image-registry-5899879c95-5w8d7" Jul 23 21:52:41.118034 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:41.117886 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s2znj\" (UniqueName: \"kubernetes.io/projected/0936cb55-c4b0-484c-8ff0-6f457ce7e5b1-kube-api-access-s2znj\") pod \"image-registry-5899879c95-5w8d7\" (UID: \"0936cb55-c4b0-484c-8ff0-6f457ce7e5b1\") " pod="openshift-image-registry/image-registry-5899879c95-5w8d7" Jul 23 21:52:41.118034 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:41.117909 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0936cb55-c4b0-484c-8ff0-6f457ce7e5b1-installation-pull-secrets\") pod \"image-registry-5899879c95-5w8d7\" (UID: \"0936cb55-c4b0-484c-8ff0-6f457ce7e5b1\") " pod="openshift-image-registry/image-registry-5899879c95-5w8d7" Jul 23 21:52:41.118992 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:41.118971 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0936cb55-c4b0-484c-8ff0-6f457ce7e5b1-ca-trust-extracted\") pod \"image-registry-5899879c95-5w8d7\" (UID: \"0936cb55-c4b0-484c-8ff0-6f457ce7e5b1\") " pod="openshift-image-registry/image-registry-5899879c95-5w8d7" Jul 23 21:52:41.119142 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:41.119118 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0936cb55-c4b0-484c-8ff0-6f457ce7e5b1-registry-certificates\") pod \"image-registry-5899879c95-5w8d7\" (UID: \"0936cb55-c4b0-484c-8ff0-6f457ce7e5b1\") " pod="openshift-image-registry/image-registry-5899879c95-5w8d7" Jul 23 21:52:41.119327 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:41.119307 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0936cb55-c4b0-484c-8ff0-6f457ce7e5b1-trusted-ca\") pod \"image-registry-5899879c95-5w8d7\" (UID: \"0936cb55-c4b0-484c-8ff0-6f457ce7e5b1\") " pod="openshift-image-registry/image-registry-5899879c95-5w8d7" Jul 23 21:52:41.120648 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:41.120621 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0936cb55-c4b0-484c-8ff0-6f457ce7e5b1-registry-tls\") pod \"image-registry-5899879c95-5w8d7\" (UID: \"0936cb55-c4b0-484c-8ff0-6f457ce7e5b1\") " pod="openshift-image-registry/image-registry-5899879c95-5w8d7" Jul 23 21:52:41.120772 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:41.120751 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0936cb55-c4b0-484c-8ff0-6f457ce7e5b1-installation-pull-secrets\") pod \"image-registry-5899879c95-5w8d7\" (UID: \"0936cb55-c4b0-484c-8ff0-6f457ce7e5b1\") " pod="openshift-image-registry/image-registry-5899879c95-5w8d7" Jul 23 21:52:41.121815 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:41.121796 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0936cb55-c4b0-484c-8ff0-6f457ce7e5b1-image-registry-private-configuration\") pod \"image-registry-5899879c95-5w8d7\" (UID: \"0936cb55-c4b0-484c-8ff0-6f457ce7e5b1\") " pod="openshift-image-registry/image-registry-5899879c95-5w8d7" Jul 23 21:52:41.130587 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:41.130560 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s2znj\" (UniqueName: \"kubernetes.io/projected/0936cb55-c4b0-484c-8ff0-6f457ce7e5b1-kube-api-access-s2znj\") pod \"image-registry-5899879c95-5w8d7\" (UID: \"0936cb55-c4b0-484c-8ff0-6f457ce7e5b1\") " pod="openshift-image-registry/image-registry-5899879c95-5w8d7" Jul 23 21:52:41.130682 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:41.130567 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0936cb55-c4b0-484c-8ff0-6f457ce7e5b1-bound-sa-token\") pod \"image-registry-5899879c95-5w8d7\" (UID: \"0936cb55-c4b0-484c-8ff0-6f457ce7e5b1\") " pod="openshift-image-registry/image-registry-5899879c95-5w8d7" Jul 23 21:52:41.135405 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:41.135384 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5f69cf99d4-r6sr7" Jul 23 21:52:41.162936 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:41.162900 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-5bc67dcd46-zxc85" Jul 23 21:52:41.259105 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:41.258640 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-qkfwc"] Jul 23 21:52:41.262665 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:41.262515 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5899879c95-5w8d7" Jul 23 21:52:41.264025 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:52:41.263981 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3dd824e2_8ab1_4a94_8d99_a0eec78aa59d.slice/crio-9bd5cf9ba2255f7ee52e2faa6ab2b9548c39e5e02651de005b9fb548d87acbed WatchSource:0}: Error finding container 9bd5cf9ba2255f7ee52e2faa6ab2b9548c39e5e02651de005b9fb548d87acbed: Status 404 returned error can't find the container with id 9bd5cf9ba2255f7ee52e2faa6ab2b9548c39e5e02651de005b9fb548d87acbed Jul 23 21:52:41.283060 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:41.283025 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5f69cf99d4-r6sr7"] Jul 23 21:52:41.286531 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:52:41.286496 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddd1401ba_e227_43ad_ac19_8b1d4e643f20.slice/crio-c0553da7d5fbbddfe61ce9db62acb419868f34a50e1691f328a88921d5caa154 WatchSource:0}: Error finding container c0553da7d5fbbddfe61ce9db62acb419868f34a50e1691f328a88921d5caa154: Status 404 returned error can't find the container with id c0553da7d5fbbddfe61ce9db62acb419868f34a50e1691f328a88921d5caa154 Jul 23 21:52:41.312394 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:41.312360 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-5bc67dcd46-zxc85"] Jul 23 21:52:41.316194 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:52:41.316085 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca691a1f_6865_40b2_96fe_9bb8782b7e18.slice/crio-361e9e3f6ed8800f5f11a8d811f2b4e46d9f8a0d48e6ab04510fba782ce0816d WatchSource:0}: Error finding container 361e9e3f6ed8800f5f11a8d811f2b4e46d9f8a0d48e6ab04510fba782ce0816d: Status 404 returned error can't find the container with id 361e9e3f6ed8800f5f11a8d811f2b4e46d9f8a0d48e6ab04510fba782ce0816d Jul 23 21:52:41.399676 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:41.399599 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5899879c95-5w8d7"] Jul 23 21:52:41.403651 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:52:41.403619 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0936cb55_c4b0_484c_8ff0_6f457ce7e5b1.slice/crio-a575a10cc02852296f1f97db37c29cfc67de4538d33f19aebe6d8a598bf424f9 WatchSource:0}: Error finding container a575a10cc02852296f1f97db37c29cfc67de4538d33f19aebe6d8a598bf424f9: Status 404 returned error can't find the container with id a575a10cc02852296f1f97db37c29cfc67de4538d33f19aebe6d8a598bf424f9 Jul 23 21:52:41.491438 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:41.491399 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-6mgzw" Jul 23 21:52:41.519448 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:41.519393 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-5bc67dcd46-zxc85" event={"ID":"ca691a1f-6865-40b2-96fe-9bb8782b7e18","Type":"ContainerStarted","Data":"361e9e3f6ed8800f5f11a8d811f2b4e46d9f8a0d48e6ab04510fba782ce0816d"} Jul 23 21:52:41.520997 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:41.520934 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-qkfwc" event={"ID":"3dd824e2-8ab1-4a94-8d99-a0eec78aa59d","Type":"ContainerStarted","Data":"9bd5cf9ba2255f7ee52e2faa6ab2b9548c39e5e02651de005b9fb548d87acbed"} Jul 23 21:52:41.522589 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:41.522520 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5f69cf99d4-r6sr7" event={"ID":"dd1401ba-e227-43ad-ac19-8b1d4e643f20","Type":"ContainerStarted","Data":"c0553da7d5fbbddfe61ce9db62acb419868f34a50e1691f328a88921d5caa154"} Jul 23 21:52:41.524409 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:41.524385 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5899879c95-5w8d7" event={"ID":"0936cb55-c4b0-484c-8ff0-6f457ce7e5b1","Type":"ContainerStarted","Data":"543a44f5a18d6cb9098f41dc01c809038b03bf6c074ce0afc00d5dc59a60e80d"} Jul 23 21:52:41.524504 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:41.524428 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5899879c95-5w8d7" event={"ID":"0936cb55-c4b0-484c-8ff0-6f457ce7e5b1","Type":"ContainerStarted","Data":"a575a10cc02852296f1f97db37c29cfc67de4538d33f19aebe6d8a598bf424f9"} Jul 23 21:52:41.524684 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:41.524665 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5899879c95-5w8d7" Jul 23 21:52:41.545445 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:41.544602 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-5899879c95-5w8d7" podStartSLOduration=1.544582307 podStartE2EDuration="1.544582307s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:52:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:52:41.543392922 +0000 UTC m=+63.785300798" watchObservedRunningTime="2026-07-23 21:52:41.544582307 +0000 UTC m=+63.786490175" Jul 23 21:52:43.944719 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:43.944674 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c998f5f-33c9-44c6-abf7-f042f767fc4f-metrics-certs\") pod \"network-metrics-daemon-kl5st\" (UID: \"7c998f5f-33c9-44c6-abf7-f042f767fc4f\") " pod="openshift-multus/network-metrics-daemon-kl5st" Jul 23 21:52:43.947403 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:43.947370 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 23 21:52:43.957892 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:43.957862 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7c998f5f-33c9-44c6-abf7-f042f767fc4f-metrics-certs\") pod \"network-metrics-daemon-kl5st\" (UID: \"7c998f5f-33c9-44c6-abf7-f042f767fc4f\") " pod="openshift-multus/network-metrics-daemon-kl5st" Jul 23 21:52:43.971557 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:43.971512 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-fxtft\"" Jul 23 21:52:43.979963 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:43.979936 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kl5st" Jul 23 21:52:44.045810 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:44.045737 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/500a30f7-ac15-47da-adf9-d0485f80af93-original-pull-secret\") pod \"global-pull-secret-syncer-26shl\" (UID: \"500a30f7-ac15-47da-adf9-d0485f80af93\") " pod="kube-system/global-pull-secret-syncer-26shl" Jul 23 21:52:44.048307 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:44.048284 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 23 21:52:44.059144 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:44.059109 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/500a30f7-ac15-47da-adf9-d0485f80af93-original-pull-secret\") pod \"global-pull-secret-syncer-26shl\" (UID: \"500a30f7-ac15-47da-adf9-d0485f80af93\") " pod="kube-system/global-pull-secret-syncer-26shl" Jul 23 21:52:44.147010 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:44.146978 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-st5zb\" (UniqueName: \"kubernetes.io/projected/ef1bd359-5601-4a79-9b52-03d6e6a49bc6-kube-api-access-st5zb\") pod \"network-check-target-4cq9t\" (UID: \"ef1bd359-5601-4a79-9b52-03d6e6a49bc6\") " pod="openshift-network-diagnostics/network-check-target-4cq9t" Jul 23 21:52:44.149981 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:44.149960 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 23 21:52:44.160763 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:44.160733 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 23 21:52:44.172498 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:44.172464 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-st5zb\" (UniqueName: \"kubernetes.io/projected/ef1bd359-5601-4a79-9b52-03d6e6a49bc6-kube-api-access-st5zb\") pod \"network-check-target-4cq9t\" (UID: \"ef1bd359-5601-4a79-9b52-03d6e6a49bc6\") " pod="openshift-network-diagnostics/network-check-target-4cq9t" Jul 23 21:52:44.262314 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:44.262286 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-26shl" Jul 23 21:52:44.278308 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:44.278282 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-kl5st"] Jul 23 21:52:44.278708 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:44.278674 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-n9zwh\"" Jul 23 21:52:44.282672 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:52:44.282641 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c998f5f_33c9_44c6_abf7_f042f767fc4f.slice/crio-870f46ae2980e5b61604aabd2ee77f837212450fccbf3bc446f9f86efc68d82a WatchSource:0}: Error finding container 870f46ae2980e5b61604aabd2ee77f837212450fccbf3bc446f9f86efc68d82a: Status 404 returned error can't find the container with id 870f46ae2980e5b61604aabd2ee77f837212450fccbf3bc446f9f86efc68d82a Jul 23 21:52:44.286617 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:44.286593 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-4cq9t" Jul 23 21:52:44.428507 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:44.428380 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-26shl"] Jul 23 21:52:44.431512 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:52:44.431480 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod500a30f7_ac15_47da_adf9_d0485f80af93.slice/crio-bd6b9143bcbf9291165660cda08d21201826bbb9af328d9802e5e2cadc6bf5d7 WatchSource:0}: Error finding container bd6b9143bcbf9291165660cda08d21201826bbb9af328d9802e5e2cadc6bf5d7: Status 404 returned error can't find the container with id bd6b9143bcbf9291165660cda08d21201826bbb9af328d9802e5e2cadc6bf5d7 Jul 23 21:52:44.457698 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:44.457455 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-4cq9t"] Jul 23 21:52:44.460148 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:52:44.460122 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef1bd359_5601_4a79_9b52_03d6e6a49bc6.slice/crio-d7fe22d4f6f9e118393fa2495210662f66eda8a06bdd7092b2853d34e6f885c3 WatchSource:0}: Error finding container d7fe22d4f6f9e118393fa2495210662f66eda8a06bdd7092b2853d34e6f885c3: Status 404 returned error can't find the container with id d7fe22d4f6f9e118393fa2495210662f66eda8a06bdd7092b2853d34e6f885c3 Jul 23 21:52:44.539370 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:44.539297 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-qkfwc" event={"ID":"3dd824e2-8ab1-4a94-8d99-a0eec78aa59d","Type":"ContainerStarted","Data":"edd445996d18b92dc2b04df824ccc9a5d9ae54f5b5cb93386167ac5f5e3a32fc"} Jul 23 21:52:44.541212 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:44.541163 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5f69cf99d4-r6sr7" event={"ID":"dd1401ba-e227-43ad-ac19-8b1d4e643f20","Type":"ContainerStarted","Data":"014c5e8806a2074ce6c967983ce010b3b66f410da53b0fd91562733364aa2c70"} Jul 23 21:52:44.542394 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:44.542369 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-kl5st" event={"ID":"7c998f5f-33c9-44c6-abf7-f042f767fc4f","Type":"ContainerStarted","Data":"870f46ae2980e5b61604aabd2ee77f837212450fccbf3bc446f9f86efc68d82a"} Jul 23 21:52:44.543605 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:44.543575 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-4cq9t" event={"ID":"ef1bd359-5601-4a79-9b52-03d6e6a49bc6","Type":"ContainerStarted","Data":"d7fe22d4f6f9e118393fa2495210662f66eda8a06bdd7092b2853d34e6f885c3"} Jul 23 21:52:44.544807 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:44.544783 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-26shl" event={"ID":"500a30f7-ac15-47da-adf9-d0485f80af93","Type":"ContainerStarted","Data":"bd6b9143bcbf9291165660cda08d21201826bbb9af328d9802e5e2cadc6bf5d7"} Jul 23 21:52:44.555644 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:44.555577 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-qkfwc" podStartSLOduration=1.694039927 podStartE2EDuration="4.555555324s" totalImagesPullingTime="2.861515397s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:52:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:52:44.554696215 +0000 UTC m=+66.796604090" watchObservedRunningTime="2026-07-23 21:52:44.555555324 +0000 UTC m=+66.797463199" Jul 23 21:52:44.569747 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:44.569688 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5f69cf99d4-r6sr7" podStartSLOduration=1.6873842369999998 podStartE2EDuration="4.569673296s" totalImagesPullingTime="2.882289059s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:52:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:52:44.569269526 +0000 UTC m=+66.811177401" watchObservedRunningTime="2026-07-23 21:52:44.569673296 +0000 UTC m=+66.811581174" Jul 23 21:52:45.556163 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:45.555762 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-kl5st" event={"ID":"7c998f5f-33c9-44c6-abf7-f042f767fc4f","Type":"ContainerStarted","Data":"d9050b6fa296faaaa5c8e11714ed670103f2955b7aa148c1ba4473a018cbc224"} Jul 23 21:52:46.562637 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:46.562533 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-kl5st" event={"ID":"7c998f5f-33c9-44c6-abf7-f042f767fc4f","Type":"ContainerStarted","Data":"a675c3e55afa4f7d32cd58d2465f6f93337674806a4cbfdd42bd546efc1dfde1"} Jul 23 21:52:46.583088 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:46.582727 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-kl5st" podStartSLOduration=67.494227952 podStartE2EDuration="1m8.582708761s" totalImagesPullingTime="1.088480809s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:51:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:52:46.580673801 +0000 UTC m=+68.822581676" watchObservedRunningTime="2026-07-23 21:52:46.582708761 +0000 UTC m=+68.824616637" Jul 23 21:52:48.753810 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:48.753773 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-j9rv8"] Jul 23 21:52:48.757196 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:48.756574 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-5945b5cfd-j9rv8" Jul 23 21:52:48.769562 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:48.769208 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 23 21:52:48.769562 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:48.769439 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 23 21:52:48.769896 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:48.769875 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-fgn9l\"" Jul 23 21:52:48.771192 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:48.770105 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 23 21:52:48.771192 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:48.770416 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 23 21:52:48.771192 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:48.770627 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 23 21:52:48.771192 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:48.770970 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-j9rv8"] Jul 23 21:52:48.888348 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:48.888310 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7b56052e-4592-4797-ba17-50254d728b8a-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-j9rv8\" (UID: \"7b56052e-4592-4797-ba17-50254d728b8a\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-j9rv8" Jul 23 21:52:48.888531 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:48.888356 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7b56052e-4592-4797-ba17-50254d728b8a-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-j9rv8\" (UID: \"7b56052e-4592-4797-ba17-50254d728b8a\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-j9rv8" Jul 23 21:52:48.888531 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:48.888387 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/7b56052e-4592-4797-ba17-50254d728b8a-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-j9rv8\" (UID: \"7b56052e-4592-4797-ba17-50254d728b8a\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-j9rv8" Jul 23 21:52:48.888531 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:48.888426 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8mklz\" (UniqueName: \"kubernetes.io/projected/7b56052e-4592-4797-ba17-50254d728b8a-kube-api-access-8mklz\") pod \"prometheus-operator-5945b5cfd-j9rv8\" (UID: \"7b56052e-4592-4797-ba17-50254d728b8a\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-j9rv8" Jul 23 21:52:48.989738 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:48.989701 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7b56052e-4592-4797-ba17-50254d728b8a-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-j9rv8\" (UID: \"7b56052e-4592-4797-ba17-50254d728b8a\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-j9rv8" Jul 23 21:52:48.989921 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:48.989761 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7b56052e-4592-4797-ba17-50254d728b8a-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-j9rv8\" (UID: \"7b56052e-4592-4797-ba17-50254d728b8a\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-j9rv8" Jul 23 21:52:48.989921 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:48.989815 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/7b56052e-4592-4797-ba17-50254d728b8a-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-j9rv8\" (UID: \"7b56052e-4592-4797-ba17-50254d728b8a\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-j9rv8" Jul 23 21:52:48.989921 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:48.989870 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8mklz\" (UniqueName: \"kubernetes.io/projected/7b56052e-4592-4797-ba17-50254d728b8a-kube-api-access-8mklz\") pod \"prometheus-operator-5945b5cfd-j9rv8\" (UID: \"7b56052e-4592-4797-ba17-50254d728b8a\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-j9rv8" Jul 23 21:52:48.990523 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:48.990492 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7b56052e-4592-4797-ba17-50254d728b8a-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-j9rv8\" (UID: \"7b56052e-4592-4797-ba17-50254d728b8a\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-j9rv8" Jul 23 21:52:48.992551 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:48.992491 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7b56052e-4592-4797-ba17-50254d728b8a-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-j9rv8\" (UID: \"7b56052e-4592-4797-ba17-50254d728b8a\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-j9rv8" Jul 23 21:52:48.992655 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:48.992616 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/7b56052e-4592-4797-ba17-50254d728b8a-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-j9rv8\" (UID: \"7b56052e-4592-4797-ba17-50254d728b8a\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-j9rv8" Jul 23 21:52:48.998342 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:48.998314 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8mklz\" (UniqueName: \"kubernetes.io/projected/7b56052e-4592-4797-ba17-50254d728b8a-kube-api-access-8mklz\") pod \"prometheus-operator-5945b5cfd-j9rv8\" (UID: \"7b56052e-4592-4797-ba17-50254d728b8a\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-j9rv8" Jul 23 21:52:49.084820 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:49.084793 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-5945b5cfd-j9rv8" Jul 23 21:52:49.847314 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:49.847194 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-j9rv8"] Jul 23 21:52:49.850652 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:52:49.850617 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b56052e_4592_4797_ba17_50254d728b8a.slice/crio-6d60248b98a43d536c86b5edfd5b80e8f6ee8cdf62869fc5c410bc9b142f091c WatchSource:0}: Error finding container 6d60248b98a43d536c86b5edfd5b80e8f6ee8cdf62869fc5c410bc9b142f091c: Status 404 returned error can't find the container with id 6d60248b98a43d536c86b5edfd5b80e8f6ee8cdf62869fc5c410bc9b142f091c Jul 23 21:52:50.402703 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:50.402662 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f1e7deb0-a030-4479-b82a-5dbbc864104b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fkqwr\" (UID: \"f1e7deb0-a030-4479-b82a-5dbbc864104b\") " pod="openshift-insights/insights-runtime-extractor-fkqwr" Jul 23 21:52:50.405495 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:50.405465 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f1e7deb0-a030-4479-b82a-5dbbc864104b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fkqwr\" (UID: \"f1e7deb0-a030-4479-b82a-5dbbc864104b\") " pod="openshift-insights/insights-runtime-extractor-fkqwr" Jul 23 21:52:50.573693 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:50.573651 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-jtqd5\"" Jul 23 21:52:50.580466 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:50.580045 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-4cq9t" event={"ID":"ef1bd359-5601-4a79-9b52-03d6e6a49bc6","Type":"ContainerStarted","Data":"c8031a8d4b67eaa9389943911847b1bd4a44af053ef367c610d59db23365a59b"} Jul 23 21:52:50.580466 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:50.580152 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-4cq9t" Jul 23 21:52:50.581592 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:50.581571 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-fkqwr" Jul 23 21:52:50.583673 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:50.583650 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-26shl" event={"ID":"500a30f7-ac15-47da-adf9-d0485f80af93","Type":"ContainerStarted","Data":"d693b3341a8530c0661c2648d0db75be10a66e02c8a96fd3d16c60b8206f63f4"} Jul 23 21:52:50.586480 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:50.586455 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-j9rv8" event={"ID":"7b56052e-4592-4797-ba17-50254d728b8a","Type":"ContainerStarted","Data":"6d60248b98a43d536c86b5edfd5b80e8f6ee8cdf62869fc5c410bc9b142f091c"} Jul 23 21:52:50.613553 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:50.613487 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-4cq9t" podStartSLOduration=67.374262364 podStartE2EDuration="1m12.613472748s" totalImagesPullingTime="5.239210384s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:51:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:52:50.596663403 +0000 UTC m=+72.838571278" watchObservedRunningTime="2026-07-23 21:52:50.613472748 +0000 UTC m=+72.855380621" Jul 23 21:52:50.614500 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:50.614467 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-26shl" podStartSLOduration=67.333489381 podStartE2EDuration="1m12.614457194s" totalImagesPullingTime="5.280967813s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 21:51:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:52:50.613620212 +0000 UTC m=+72.855528082" watchObservedRunningTime="2026-07-23 21:52:50.614457194 +0000 UTC m=+72.856365067" Jul 23 21:52:50.854560 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:50.854517 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-fbbbb55d8-pz9t5" Jul 23 21:52:51.103158 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:51.103135 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-fkqwr"] Jul 23 21:52:51.106476 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:52:51.106447 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1e7deb0_a030_4479_b82a_5dbbc864104b.slice/crio-d03346963f0a5b5d34da8a0de33065d4e9e396d934619bdcf92793fc30115940 WatchSource:0}: Error finding container d03346963f0a5b5d34da8a0de33065d4e9e396d934619bdcf92793fc30115940: Status 404 returned error can't find the container with id d03346963f0a5b5d34da8a0de33065d4e9e396d934619bdcf92793fc30115940 Jul 23 21:52:51.598936 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:51.598119 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-j9rv8" event={"ID":"7b56052e-4592-4797-ba17-50254d728b8a","Type":"ContainerStarted","Data":"c587ba1aeb8fc0fb9a933f6d6115ecdaadb3d59deb4803973315ff7b1b14d2e3"} Jul 23 21:52:51.598936 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:51.598186 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-j9rv8" event={"ID":"7b56052e-4592-4797-ba17-50254d728b8a","Type":"ContainerStarted","Data":"9757d163c6b8a248e5768ab514aff7c37fcf9e600017c0c91cbfd41d847800a6"} Jul 23 21:52:51.601732 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:51.601217 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fkqwr" event={"ID":"f1e7deb0-a030-4479-b82a-5dbbc864104b","Type":"ContainerStarted","Data":"d03346963f0a5b5d34da8a0de33065d4e9e396d934619bdcf92793fc30115940"} Jul 23 21:52:51.618599 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:51.617434 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-5945b5cfd-j9rv8" podStartSLOduration=2.4960024499999998 podStartE2EDuration="3.617414805s" totalImagesPullingTime="1.121412355s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:52:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:52:51.61581455 +0000 UTC m=+73.857722434" watchObservedRunningTime="2026-07-23 21:52:51.617414805 +0000 UTC m=+73.859322679" Jul 23 21:52:53.310985 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:53.310952 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-9szvj"] Jul 23 21:52:53.315533 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:53.315511 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-9szvj" Jul 23 21:52:53.323097 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:53.322917 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 23 21:52:53.323097 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:53.322968 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 23 21:52:53.323097 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:53.322968 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 23 21:52:53.323328 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:53.323232 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-ktmzs\"" Jul 23 21:52:53.433669 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:53.433627 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/98fdd345-9020-4d2f-9ce2-9a392c19218c-sys\") pod \"node-exporter-9szvj\" (UID: \"98fdd345-9020-4d2f-9ce2-9a392c19218c\") " pod="openshift-monitoring/node-exporter-9szvj" Jul 23 21:52:53.433862 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:53.433701 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/98fdd345-9020-4d2f-9ce2-9a392c19218c-node-exporter-textfile\") pod \"node-exporter-9szvj\" (UID: \"98fdd345-9020-4d2f-9ce2-9a392c19218c\") " pod="openshift-monitoring/node-exporter-9szvj" Jul 23 21:52:53.433862 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:53.433730 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/98fdd345-9020-4d2f-9ce2-9a392c19218c-node-exporter-tls\") pod \"node-exporter-9szvj\" (UID: \"98fdd345-9020-4d2f-9ce2-9a392c19218c\") " pod="openshift-monitoring/node-exporter-9szvj" Jul 23 21:52:53.433862 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:53.433757 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/98fdd345-9020-4d2f-9ce2-9a392c19218c-node-exporter-wtmp\") pod \"node-exporter-9szvj\" (UID: \"98fdd345-9020-4d2f-9ce2-9a392c19218c\") " pod="openshift-monitoring/node-exporter-9szvj" Jul 23 21:52:53.433862 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:53.433792 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/98fdd345-9020-4d2f-9ce2-9a392c19218c-node-exporter-accelerators-collector-config\") pod \"node-exporter-9szvj\" (UID: \"98fdd345-9020-4d2f-9ce2-9a392c19218c\") " pod="openshift-monitoring/node-exporter-9szvj" Jul 23 21:52:53.433862 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:53.433811 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/98fdd345-9020-4d2f-9ce2-9a392c19218c-root\") pod \"node-exporter-9szvj\" (UID: \"98fdd345-9020-4d2f-9ce2-9a392c19218c\") " pod="openshift-monitoring/node-exporter-9szvj" Jul 23 21:52:53.433862 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:53.433835 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/98fdd345-9020-4d2f-9ce2-9a392c19218c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-9szvj\" (UID: \"98fdd345-9020-4d2f-9ce2-9a392c19218c\") " pod="openshift-monitoring/node-exporter-9szvj" Jul 23 21:52:53.433862 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:53.433858 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nnzbk\" (UniqueName: \"kubernetes.io/projected/98fdd345-9020-4d2f-9ce2-9a392c19218c-kube-api-access-nnzbk\") pod \"node-exporter-9szvj\" (UID: \"98fdd345-9020-4d2f-9ce2-9a392c19218c\") " pod="openshift-monitoring/node-exporter-9szvj" Jul 23 21:52:53.434348 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:53.433900 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/98fdd345-9020-4d2f-9ce2-9a392c19218c-metrics-client-ca\") pod \"node-exporter-9szvj\" (UID: \"98fdd345-9020-4d2f-9ce2-9a392c19218c\") " pod="openshift-monitoring/node-exporter-9szvj" Jul 23 21:52:53.534513 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:53.534479 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/98fdd345-9020-4d2f-9ce2-9a392c19218c-node-exporter-wtmp\") pod \"node-exporter-9szvj\" (UID: \"98fdd345-9020-4d2f-9ce2-9a392c19218c\") " pod="openshift-monitoring/node-exporter-9szvj" Jul 23 21:52:53.534721 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:53.534553 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/98fdd345-9020-4d2f-9ce2-9a392c19218c-node-exporter-accelerators-collector-config\") pod \"node-exporter-9szvj\" (UID: \"98fdd345-9020-4d2f-9ce2-9a392c19218c\") " pod="openshift-monitoring/node-exporter-9szvj" Jul 23 21:52:53.534721 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:53.534588 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/98fdd345-9020-4d2f-9ce2-9a392c19218c-root\") pod \"node-exporter-9szvj\" (UID: \"98fdd345-9020-4d2f-9ce2-9a392c19218c\") " pod="openshift-monitoring/node-exporter-9szvj" Jul 23 21:52:53.534721 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:53.534609 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/98fdd345-9020-4d2f-9ce2-9a392c19218c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-9szvj\" (UID: \"98fdd345-9020-4d2f-9ce2-9a392c19218c\") " pod="openshift-monitoring/node-exporter-9szvj" Jul 23 21:52:53.534721 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:53.534632 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nnzbk\" (UniqueName: \"kubernetes.io/projected/98fdd345-9020-4d2f-9ce2-9a392c19218c-kube-api-access-nnzbk\") pod \"node-exporter-9szvj\" (UID: \"98fdd345-9020-4d2f-9ce2-9a392c19218c\") " pod="openshift-monitoring/node-exporter-9szvj" Jul 23 21:52:53.534721 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:53.534666 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/98fdd345-9020-4d2f-9ce2-9a392c19218c-metrics-client-ca\") pod \"node-exporter-9szvj\" (UID: \"98fdd345-9020-4d2f-9ce2-9a392c19218c\") " pod="openshift-monitoring/node-exporter-9szvj" Jul 23 21:52:53.534721 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:53.534703 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/98fdd345-9020-4d2f-9ce2-9a392c19218c-node-exporter-wtmp\") pod \"node-exporter-9szvj\" (UID: \"98fdd345-9020-4d2f-9ce2-9a392c19218c\") " pod="openshift-monitoring/node-exporter-9szvj" Jul 23 21:52:53.535014 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:53.534712 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/98fdd345-9020-4d2f-9ce2-9a392c19218c-root\") pod \"node-exporter-9szvj\" (UID: \"98fdd345-9020-4d2f-9ce2-9a392c19218c\") " pod="openshift-monitoring/node-exporter-9szvj" Jul 23 21:52:53.535014 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:53.534741 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/98fdd345-9020-4d2f-9ce2-9a392c19218c-sys\") pod \"node-exporter-9szvj\" (UID: \"98fdd345-9020-4d2f-9ce2-9a392c19218c\") " pod="openshift-monitoring/node-exporter-9szvj" Jul 23 21:52:53.535014 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:53.534707 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/98fdd345-9020-4d2f-9ce2-9a392c19218c-sys\") pod \"node-exporter-9szvj\" (UID: \"98fdd345-9020-4d2f-9ce2-9a392c19218c\") " pod="openshift-monitoring/node-exporter-9szvj" Jul 23 21:52:53.535014 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:53.534797 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/98fdd345-9020-4d2f-9ce2-9a392c19218c-node-exporter-textfile\") pod \"node-exporter-9szvj\" (UID: \"98fdd345-9020-4d2f-9ce2-9a392c19218c\") " pod="openshift-monitoring/node-exporter-9szvj" Jul 23 21:52:53.535014 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:53.534820 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/98fdd345-9020-4d2f-9ce2-9a392c19218c-node-exporter-tls\") pod \"node-exporter-9szvj\" (UID: \"98fdd345-9020-4d2f-9ce2-9a392c19218c\") " pod="openshift-monitoring/node-exporter-9szvj" Jul 23 21:52:53.535014 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:52:53.534911 2578 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 23 21:52:53.535014 ip-10-0-136-157 kubenswrapper[2578]: E0723 21:52:53.534956 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/98fdd345-9020-4d2f-9ce2-9a392c19218c-node-exporter-tls podName:98fdd345-9020-4d2f-9ce2-9a392c19218c nodeName:}" failed. No retries permitted until 2026-07-23 21:52:54.034940918 +0000 UTC m=+76.276848770 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/98fdd345-9020-4d2f-9ce2-9a392c19218c-node-exporter-tls") pod "node-exporter-9szvj" (UID: "98fdd345-9020-4d2f-9ce2-9a392c19218c") : secret "node-exporter-tls" not found Jul 23 21:52:53.535391 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:53.535336 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/98fdd345-9020-4d2f-9ce2-9a392c19218c-metrics-client-ca\") pod \"node-exporter-9szvj\" (UID: \"98fdd345-9020-4d2f-9ce2-9a392c19218c\") " pod="openshift-monitoring/node-exporter-9szvj" Jul 23 21:52:53.535609 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:53.535555 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/98fdd345-9020-4d2f-9ce2-9a392c19218c-node-exporter-accelerators-collector-config\") pod \"node-exporter-9szvj\" (UID: \"98fdd345-9020-4d2f-9ce2-9a392c19218c\") " pod="openshift-monitoring/node-exporter-9szvj" Jul 23 21:52:53.535609 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:53.535561 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/98fdd345-9020-4d2f-9ce2-9a392c19218c-node-exporter-textfile\") pod \"node-exporter-9szvj\" (UID: \"98fdd345-9020-4d2f-9ce2-9a392c19218c\") " pod="openshift-monitoring/node-exporter-9szvj" Jul 23 21:52:53.537955 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:53.537914 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/98fdd345-9020-4d2f-9ce2-9a392c19218c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-9szvj\" (UID: \"98fdd345-9020-4d2f-9ce2-9a392c19218c\") " pod="openshift-monitoring/node-exporter-9szvj" Jul 23 21:52:53.546899 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:53.546858 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nnzbk\" (UniqueName: \"kubernetes.io/projected/98fdd345-9020-4d2f-9ce2-9a392c19218c-kube-api-access-nnzbk\") pod \"node-exporter-9szvj\" (UID: \"98fdd345-9020-4d2f-9ce2-9a392c19218c\") " pod="openshift-monitoring/node-exporter-9szvj" Jul 23 21:52:54.039513 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:54.039480 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/98fdd345-9020-4d2f-9ce2-9a392c19218c-node-exporter-tls\") pod \"node-exporter-9szvj\" (UID: \"98fdd345-9020-4d2f-9ce2-9a392c19218c\") " pod="openshift-monitoring/node-exporter-9szvj" Jul 23 21:52:54.043913 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:54.042134 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/98fdd345-9020-4d2f-9ce2-9a392c19218c-node-exporter-tls\") pod \"node-exporter-9szvj\" (UID: \"98fdd345-9020-4d2f-9ce2-9a392c19218c\") " pod="openshift-monitoring/node-exporter-9szvj" Jul 23 21:52:54.233655 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:54.233615 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-9szvj" Jul 23 21:52:55.247998 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:55.247962 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-55ff89fbc9-t5zhk"] Jul 23 21:52:55.251248 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:55.251228 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-55ff89fbc9-t5zhk" Jul 23 21:52:55.253792 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:55.253770 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 23 21:52:55.253913 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:55.253807 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 23 21:52:55.253913 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:55.253813 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 23 21:52:55.253913 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:55.253811 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 23 21:52:55.253913 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:55.253811 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-h5rvz\"" Jul 23 21:52:55.254362 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:55.254341 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 23 21:52:55.254596 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:55.254580 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-76u6n5f6v3jk9\"" Jul 23 21:52:55.265576 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:55.265556 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-55ff89fbc9-t5zhk"] Jul 23 21:52:55.351771 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:55.351736 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zkkn8\" (UniqueName: \"kubernetes.io/projected/034d2dc9-a711-493f-b5bc-6f772cf1641f-kube-api-access-zkkn8\") pod \"thanos-querier-55ff89fbc9-t5zhk\" (UID: \"034d2dc9-a711-493f-b5bc-6f772cf1641f\") " pod="openshift-monitoring/thanos-querier-55ff89fbc9-t5zhk" Jul 23 21:52:55.351771 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:55.351775 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/034d2dc9-a711-493f-b5bc-6f772cf1641f-metrics-client-ca\") pod \"thanos-querier-55ff89fbc9-t5zhk\" (UID: \"034d2dc9-a711-493f-b5bc-6f772cf1641f\") " pod="openshift-monitoring/thanos-querier-55ff89fbc9-t5zhk" Jul 23 21:52:55.352004 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:55.351810 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/034d2dc9-a711-493f-b5bc-6f772cf1641f-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-55ff89fbc9-t5zhk\" (UID: \"034d2dc9-a711-493f-b5bc-6f772cf1641f\") " pod="openshift-monitoring/thanos-querier-55ff89fbc9-t5zhk" Jul 23 21:52:55.352004 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:55.351882 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/034d2dc9-a711-493f-b5bc-6f772cf1641f-secret-grpc-tls\") pod \"thanos-querier-55ff89fbc9-t5zhk\" (UID: \"034d2dc9-a711-493f-b5bc-6f772cf1641f\") " pod="openshift-monitoring/thanos-querier-55ff89fbc9-t5zhk" Jul 23 21:52:55.352004 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:55.351914 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/034d2dc9-a711-493f-b5bc-6f772cf1641f-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-55ff89fbc9-t5zhk\" (UID: \"034d2dc9-a711-493f-b5bc-6f772cf1641f\") " pod="openshift-monitoring/thanos-querier-55ff89fbc9-t5zhk" Jul 23 21:52:55.352004 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:55.351936 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/034d2dc9-a711-493f-b5bc-6f772cf1641f-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-55ff89fbc9-t5zhk\" (UID: \"034d2dc9-a711-493f-b5bc-6f772cf1641f\") " pod="openshift-monitoring/thanos-querier-55ff89fbc9-t5zhk" Jul 23 21:52:55.352004 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:55.351964 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/034d2dc9-a711-493f-b5bc-6f772cf1641f-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-55ff89fbc9-t5zhk\" (UID: \"034d2dc9-a711-493f-b5bc-6f772cf1641f\") " pod="openshift-monitoring/thanos-querier-55ff89fbc9-t5zhk" Jul 23 21:52:55.352004 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:55.351982 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/034d2dc9-a711-493f-b5bc-6f772cf1641f-secret-thanos-querier-tls\") pod \"thanos-querier-55ff89fbc9-t5zhk\" (UID: \"034d2dc9-a711-493f-b5bc-6f772cf1641f\") " pod="openshift-monitoring/thanos-querier-55ff89fbc9-t5zhk" Jul 23 21:52:55.452670 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:55.452628 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/034d2dc9-a711-493f-b5bc-6f772cf1641f-secret-grpc-tls\") pod \"thanos-querier-55ff89fbc9-t5zhk\" (UID: \"034d2dc9-a711-493f-b5bc-6f772cf1641f\") " pod="openshift-monitoring/thanos-querier-55ff89fbc9-t5zhk" Jul 23 21:52:55.452814 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:55.452703 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/034d2dc9-a711-493f-b5bc-6f772cf1641f-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-55ff89fbc9-t5zhk\" (UID: \"034d2dc9-a711-493f-b5bc-6f772cf1641f\") " pod="openshift-monitoring/thanos-querier-55ff89fbc9-t5zhk" Jul 23 21:52:55.452814 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:55.452724 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/034d2dc9-a711-493f-b5bc-6f772cf1641f-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-55ff89fbc9-t5zhk\" (UID: \"034d2dc9-a711-493f-b5bc-6f772cf1641f\") " pod="openshift-monitoring/thanos-querier-55ff89fbc9-t5zhk" Jul 23 21:52:55.452814 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:55.452744 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/034d2dc9-a711-493f-b5bc-6f772cf1641f-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-55ff89fbc9-t5zhk\" (UID: \"034d2dc9-a711-493f-b5bc-6f772cf1641f\") " pod="openshift-monitoring/thanos-querier-55ff89fbc9-t5zhk" Jul 23 21:52:55.452814 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:55.452763 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/034d2dc9-a711-493f-b5bc-6f772cf1641f-secret-thanos-querier-tls\") pod \"thanos-querier-55ff89fbc9-t5zhk\" (UID: \"034d2dc9-a711-493f-b5bc-6f772cf1641f\") " pod="openshift-monitoring/thanos-querier-55ff89fbc9-t5zhk" Jul 23 21:52:55.452814 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:55.452785 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zkkn8\" (UniqueName: \"kubernetes.io/projected/034d2dc9-a711-493f-b5bc-6f772cf1641f-kube-api-access-zkkn8\") pod \"thanos-querier-55ff89fbc9-t5zhk\" (UID: \"034d2dc9-a711-493f-b5bc-6f772cf1641f\") " pod="openshift-monitoring/thanos-querier-55ff89fbc9-t5zhk" Jul 23 21:52:55.452814 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:55.452801 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/034d2dc9-a711-493f-b5bc-6f772cf1641f-metrics-client-ca\") pod \"thanos-querier-55ff89fbc9-t5zhk\" (UID: \"034d2dc9-a711-493f-b5bc-6f772cf1641f\") " pod="openshift-monitoring/thanos-querier-55ff89fbc9-t5zhk" Jul 23 21:52:55.453001 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:55.452826 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/034d2dc9-a711-493f-b5bc-6f772cf1641f-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-55ff89fbc9-t5zhk\" (UID: \"034d2dc9-a711-493f-b5bc-6f772cf1641f\") " pod="openshift-monitoring/thanos-querier-55ff89fbc9-t5zhk" Jul 23 21:52:55.453997 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:55.453953 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/034d2dc9-a711-493f-b5bc-6f772cf1641f-metrics-client-ca\") pod \"thanos-querier-55ff89fbc9-t5zhk\" (UID: \"034d2dc9-a711-493f-b5bc-6f772cf1641f\") " pod="openshift-monitoring/thanos-querier-55ff89fbc9-t5zhk" Jul 23 21:52:55.455716 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:55.455664 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/034d2dc9-a711-493f-b5bc-6f772cf1641f-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-55ff89fbc9-t5zhk\" (UID: \"034d2dc9-a711-493f-b5bc-6f772cf1641f\") " pod="openshift-monitoring/thanos-querier-55ff89fbc9-t5zhk" Jul 23 21:52:55.455919 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:55.455878 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/034d2dc9-a711-493f-b5bc-6f772cf1641f-secret-grpc-tls\") pod \"thanos-querier-55ff89fbc9-t5zhk\" (UID: \"034d2dc9-a711-493f-b5bc-6f772cf1641f\") " pod="openshift-monitoring/thanos-querier-55ff89fbc9-t5zhk" Jul 23 21:52:55.456914 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:55.456894 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/034d2dc9-a711-493f-b5bc-6f772cf1641f-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-55ff89fbc9-t5zhk\" (UID: \"034d2dc9-a711-493f-b5bc-6f772cf1641f\") " pod="openshift-monitoring/thanos-querier-55ff89fbc9-t5zhk" Jul 23 21:52:55.457087 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:55.457070 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/034d2dc9-a711-493f-b5bc-6f772cf1641f-secret-thanos-querier-tls\") pod \"thanos-querier-55ff89fbc9-t5zhk\" (UID: \"034d2dc9-a711-493f-b5bc-6f772cf1641f\") " pod="openshift-monitoring/thanos-querier-55ff89fbc9-t5zhk" Jul 23 21:52:55.457171 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:55.457070 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/034d2dc9-a711-493f-b5bc-6f772cf1641f-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-55ff89fbc9-t5zhk\" (UID: \"034d2dc9-a711-493f-b5bc-6f772cf1641f\") " pod="openshift-monitoring/thanos-querier-55ff89fbc9-t5zhk" Jul 23 21:52:55.459235 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:55.459216 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/034d2dc9-a711-493f-b5bc-6f772cf1641f-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-55ff89fbc9-t5zhk\" (UID: \"034d2dc9-a711-493f-b5bc-6f772cf1641f\") " pod="openshift-monitoring/thanos-querier-55ff89fbc9-t5zhk" Jul 23 21:52:55.462207 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:55.462183 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zkkn8\" (UniqueName: \"kubernetes.io/projected/034d2dc9-a711-493f-b5bc-6f772cf1641f-kube-api-access-zkkn8\") pod \"thanos-querier-55ff89fbc9-t5zhk\" (UID: \"034d2dc9-a711-493f-b5bc-6f772cf1641f\") " pod="openshift-monitoring/thanos-querier-55ff89fbc9-t5zhk" Jul 23 21:52:55.563575 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:55.562346 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-55ff89fbc9-t5zhk" Jul 23 21:52:57.695253 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:57.695218 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-5cb6c9bd46-gwkr9"] Jul 23 21:52:57.697507 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:57.697486 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5cb6c9bd46-gwkr9" Jul 23 21:52:57.700186 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:57.700164 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 23 21:52:57.701529 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:57.701214 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 23 21:52:57.701529 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:57.701250 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 23 21:52:57.701529 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:57.701289 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-mf965\"" Jul 23 21:52:57.701529 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:57.701296 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 23 21:52:57.701529 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:57.701293 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-5mm5b8vrsnq4p\"" Jul 23 21:52:57.711017 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:57.710993 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5cb6c9bd46-gwkr9"] Jul 23 21:52:57.874780 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:57.874746 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/a82ef530-e2e9-4234-b23f-1230edd4b83c-secret-metrics-server-tls\") pod \"metrics-server-5cb6c9bd46-gwkr9\" (UID: \"a82ef530-e2e9-4234-b23f-1230edd4b83c\") " pod="openshift-monitoring/metrics-server-5cb6c9bd46-gwkr9" Jul 23 21:52:57.874958 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:57.874789 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a82ef530-e2e9-4234-b23f-1230edd4b83c-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5cb6c9bd46-gwkr9\" (UID: \"a82ef530-e2e9-4234-b23f-1230edd4b83c\") " pod="openshift-monitoring/metrics-server-5cb6c9bd46-gwkr9" Jul 23 21:52:57.874958 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:57.874816 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mgm8b\" (UniqueName: \"kubernetes.io/projected/a82ef530-e2e9-4234-b23f-1230edd4b83c-kube-api-access-mgm8b\") pod \"metrics-server-5cb6c9bd46-gwkr9\" (UID: \"a82ef530-e2e9-4234-b23f-1230edd4b83c\") " pod="openshift-monitoring/metrics-server-5cb6c9bd46-gwkr9" Jul 23 21:52:57.874958 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:57.874921 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/a82ef530-e2e9-4234-b23f-1230edd4b83c-secret-metrics-server-client-certs\") pod \"metrics-server-5cb6c9bd46-gwkr9\" (UID: \"a82ef530-e2e9-4234-b23f-1230edd4b83c\") " pod="openshift-monitoring/metrics-server-5cb6c9bd46-gwkr9" Jul 23 21:52:57.875076 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:57.874987 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/a82ef530-e2e9-4234-b23f-1230edd4b83c-metrics-server-audit-profiles\") pod \"metrics-server-5cb6c9bd46-gwkr9\" (UID: \"a82ef530-e2e9-4234-b23f-1230edd4b83c\") " pod="openshift-monitoring/metrics-server-5cb6c9bd46-gwkr9" Jul 23 21:52:57.875076 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:57.875020 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a82ef530-e2e9-4234-b23f-1230edd4b83c-client-ca-bundle\") pod \"metrics-server-5cb6c9bd46-gwkr9\" (UID: \"a82ef530-e2e9-4234-b23f-1230edd4b83c\") " pod="openshift-monitoring/metrics-server-5cb6c9bd46-gwkr9" Jul 23 21:52:57.875076 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:57.875059 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/a82ef530-e2e9-4234-b23f-1230edd4b83c-audit-log\") pod \"metrics-server-5cb6c9bd46-gwkr9\" (UID: \"a82ef530-e2e9-4234-b23f-1230edd4b83c\") " pod="openshift-monitoring/metrics-server-5cb6c9bd46-gwkr9" Jul 23 21:52:57.976474 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:57.976385 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mgm8b\" (UniqueName: \"kubernetes.io/projected/a82ef530-e2e9-4234-b23f-1230edd4b83c-kube-api-access-mgm8b\") pod \"metrics-server-5cb6c9bd46-gwkr9\" (UID: \"a82ef530-e2e9-4234-b23f-1230edd4b83c\") " pod="openshift-monitoring/metrics-server-5cb6c9bd46-gwkr9" Jul 23 21:52:57.976474 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:57.976447 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/a82ef530-e2e9-4234-b23f-1230edd4b83c-secret-metrics-server-client-certs\") pod \"metrics-server-5cb6c9bd46-gwkr9\" (UID: \"a82ef530-e2e9-4234-b23f-1230edd4b83c\") " pod="openshift-monitoring/metrics-server-5cb6c9bd46-gwkr9" Jul 23 21:52:57.976698 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:57.976490 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/a82ef530-e2e9-4234-b23f-1230edd4b83c-metrics-server-audit-profiles\") pod \"metrics-server-5cb6c9bd46-gwkr9\" (UID: \"a82ef530-e2e9-4234-b23f-1230edd4b83c\") " pod="openshift-monitoring/metrics-server-5cb6c9bd46-gwkr9" Jul 23 21:52:57.976698 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:57.976519 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a82ef530-e2e9-4234-b23f-1230edd4b83c-client-ca-bundle\") pod \"metrics-server-5cb6c9bd46-gwkr9\" (UID: \"a82ef530-e2e9-4234-b23f-1230edd4b83c\") " pod="openshift-monitoring/metrics-server-5cb6c9bd46-gwkr9" Jul 23 21:52:57.976698 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:57.976570 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/a82ef530-e2e9-4234-b23f-1230edd4b83c-audit-log\") pod \"metrics-server-5cb6c9bd46-gwkr9\" (UID: \"a82ef530-e2e9-4234-b23f-1230edd4b83c\") " pod="openshift-monitoring/metrics-server-5cb6c9bd46-gwkr9" Jul 23 21:52:57.976698 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:57.976628 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/a82ef530-e2e9-4234-b23f-1230edd4b83c-secret-metrics-server-tls\") pod \"metrics-server-5cb6c9bd46-gwkr9\" (UID: \"a82ef530-e2e9-4234-b23f-1230edd4b83c\") " pod="openshift-monitoring/metrics-server-5cb6c9bd46-gwkr9" Jul 23 21:52:57.976698 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:57.976661 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a82ef530-e2e9-4234-b23f-1230edd4b83c-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5cb6c9bd46-gwkr9\" (UID: \"a82ef530-e2e9-4234-b23f-1230edd4b83c\") " pod="openshift-monitoring/metrics-server-5cb6c9bd46-gwkr9" Jul 23 21:52:57.977139 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:57.977114 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/a82ef530-e2e9-4234-b23f-1230edd4b83c-audit-log\") pod \"metrics-server-5cb6c9bd46-gwkr9\" (UID: \"a82ef530-e2e9-4234-b23f-1230edd4b83c\") " pod="openshift-monitoring/metrics-server-5cb6c9bd46-gwkr9" Jul 23 21:52:57.977342 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:57.977322 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a82ef530-e2e9-4234-b23f-1230edd4b83c-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5cb6c9bd46-gwkr9\" (UID: \"a82ef530-e2e9-4234-b23f-1230edd4b83c\") " pod="openshift-monitoring/metrics-server-5cb6c9bd46-gwkr9" Jul 23 21:52:57.977627 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:57.977606 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/a82ef530-e2e9-4234-b23f-1230edd4b83c-metrics-server-audit-profiles\") pod \"metrics-server-5cb6c9bd46-gwkr9\" (UID: \"a82ef530-e2e9-4234-b23f-1230edd4b83c\") " pod="openshift-monitoring/metrics-server-5cb6c9bd46-gwkr9" Jul 23 21:52:57.979982 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:57.979957 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/a82ef530-e2e9-4234-b23f-1230edd4b83c-client-ca-bundle\") pod \"metrics-server-5cb6c9bd46-gwkr9\" (UID: \"a82ef530-e2e9-4234-b23f-1230edd4b83c\") " pod="openshift-monitoring/metrics-server-5cb6c9bd46-gwkr9" Jul 23 21:52:57.980560 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:57.980520 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/a82ef530-e2e9-4234-b23f-1230edd4b83c-secret-metrics-server-tls\") pod \"metrics-server-5cb6c9bd46-gwkr9\" (UID: \"a82ef530-e2e9-4234-b23f-1230edd4b83c\") " pod="openshift-monitoring/metrics-server-5cb6c9bd46-gwkr9" Jul 23 21:52:57.982265 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:57.982242 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/a82ef530-e2e9-4234-b23f-1230edd4b83c-secret-metrics-server-client-certs\") pod \"metrics-server-5cb6c9bd46-gwkr9\" (UID: \"a82ef530-e2e9-4234-b23f-1230edd4b83c\") " pod="openshift-monitoring/metrics-server-5cb6c9bd46-gwkr9" Jul 23 21:52:57.994341 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:57.994314 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mgm8b\" (UniqueName: \"kubernetes.io/projected/a82ef530-e2e9-4234-b23f-1230edd4b83c-kube-api-access-mgm8b\") pod \"metrics-server-5cb6c9bd46-gwkr9\" (UID: \"a82ef530-e2e9-4234-b23f-1230edd4b83c\") " pod="openshift-monitoring/metrics-server-5cb6c9bd46-gwkr9" Jul 23 21:52:58.009176 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:52:58.009147 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5cb6c9bd46-gwkr9" Jul 23 21:53:02.535810 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:53:02.535508 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-5899879c95-5w8d7" Jul 23 21:53:02.902614 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:53:02.902584 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98fdd345_9020_4d2f_9ce2_9a392c19218c.slice/crio-43b846149c227d40426d2aec72be2c139c9b55d6412cc78e89395b1b58f91eec WatchSource:0}: Error finding container 43b846149c227d40426d2aec72be2c139c9b55d6412cc78e89395b1b58f91eec: Status 404 returned error can't find the container with id 43b846149c227d40426d2aec72be2c139c9b55d6412cc78e89395b1b58f91eec Jul 23 21:53:03.052081 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:53:03.052046 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-55ff89fbc9-t5zhk"] Jul 23 21:53:03.056221 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:53:03.056199 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod034d2dc9_a711_493f_b5bc_6f772cf1641f.slice/crio-857c477f91ebe5fc3c1862cd194f85b7c016b3fd6d2001425b901ec4150273d7 WatchSource:0}: Error finding container 857c477f91ebe5fc3c1862cd194f85b7c016b3fd6d2001425b901ec4150273d7: Status 404 returned error can't find the container with id 857c477f91ebe5fc3c1862cd194f85b7c016b3fd6d2001425b901ec4150273d7 Jul 23 21:53:03.067611 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:53:03.067587 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5cb6c9bd46-gwkr9"] Jul 23 21:53:03.070049 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:53:03.070024 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda82ef530_e2e9_4234_b23f_1230edd4b83c.slice/crio-06e7e542fba1d8d661439dd1d669de1441b257f034f71fb94fec4e494f93c84b WatchSource:0}: Error finding container 06e7e542fba1d8d661439dd1d669de1441b257f034f71fb94fec4e494f93c84b: Status 404 returned error can't find the container with id 06e7e542fba1d8d661439dd1d669de1441b257f034f71fb94fec4e494f93c84b Jul 23 21:53:03.671165 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:53:03.668299 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-5bc67dcd46-zxc85" event={"ID":"ca691a1f-6865-40b2-96fe-9bb8782b7e18","Type":"ContainerStarted","Data":"4427c046bccd4a5e2ec0914723d918cfdf2b48a1ccbbc0ea9bb4decb34ef6d2d"} Jul 23 21:53:03.671165 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:53:03.669094 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-5bc67dcd46-zxc85" Jul 23 21:53:03.673676 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:53:03.673333 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5cb6c9bd46-gwkr9" event={"ID":"a82ef530-e2e9-4234-b23f-1230edd4b83c","Type":"ContainerStarted","Data":"06e7e542fba1d8d661439dd1d669de1441b257f034f71fb94fec4e494f93c84b"} Jul 23 21:53:03.675380 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:53:03.675297 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-5bc67dcd46-zxc85" Jul 23 21:53:03.680182 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:53:03.680153 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-55ff89fbc9-t5zhk" event={"ID":"034d2dc9-a711-493f-b5bc-6f772cf1641f","Type":"ContainerStarted","Data":"857c477f91ebe5fc3c1862cd194f85b7c016b3fd6d2001425b901ec4150273d7"} Jul 23 21:53:03.688531 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:53:03.688475 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9szvj" event={"ID":"98fdd345-9020-4d2f-9ce2-9a392c19218c","Type":"ContainerStarted","Data":"43b846149c227d40426d2aec72be2c139c9b55d6412cc78e89395b1b58f91eec"} Jul 23 21:53:03.695350 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:53:03.695263 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fkqwr" event={"ID":"f1e7deb0-a030-4479-b82a-5dbbc864104b","Type":"ContainerStarted","Data":"872e0135602d13e203f4c4c8fe7a3d393ef6ce295ec82d849664695afd6e3b9b"} Jul 23 21:53:03.744615 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:53:03.740323 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-5bc67dcd46-zxc85" podStartSLOduration=2.159328038 podStartE2EDuration="23.740301282s" totalImagesPullingTime="21.580973244s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:52:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:53:03.702139389 +0000 UTC m=+85.944047263" watchObservedRunningTime="2026-07-23 21:53:03.740301282 +0000 UTC m=+85.982209155" Jul 23 21:53:05.867589 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:53:05.867549 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-fbbbb55d8-pz9t5" podUID="88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3" containerName="registry" containerID="cri-o://25ff99f7d0736b7918e8bcdec808981d4771823aa557e088283a6ef8beca0691" gracePeriod=30 Jul 23 21:53:08.725134 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:53:08.725089 2578 generic.go:356] "Generic (PLEG): container finished" podID="88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3" containerID="25ff99f7d0736b7918e8bcdec808981d4771823aa557e088283a6ef8beca0691" exitCode=0 Jul 23 21:53:08.725645 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:53:08.725152 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-fbbbb55d8-pz9t5" event={"ID":"88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3","Type":"ContainerDied","Data":"25ff99f7d0736b7918e8bcdec808981d4771823aa557e088283a6ef8beca0691"} Jul 23 21:53:09.629932 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:53:09.629900 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-fbbbb55d8-pz9t5" Jul 23 21:53:09.683065 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:53:09.682360 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3-registry-certificates\") pod \"88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3\" (UID: \"88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3\") " Jul 23 21:53:09.683065 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:53:09.682408 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3-image-registry-private-configuration\") pod \"88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3\" (UID: \"88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3\") " Jul 23 21:53:09.683065 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:53:09.682456 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3-trusted-ca\") pod \"88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3\" (UID: \"88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3\") " Jul 23 21:53:09.683065 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:53:09.682489 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7tg7w\" (UniqueName: \"kubernetes.io/projected/88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3-kube-api-access-7tg7w\") pod \"88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3\" (UID: \"88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3\") " Jul 23 21:53:09.683065 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:53:09.682520 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3-installation-pull-secrets\") pod \"88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3\" (UID: \"88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3\") " Jul 23 21:53:09.683065 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:53:09.682592 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3-bound-sa-token\") pod \"88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3\" (UID: \"88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3\") " Jul 23 21:53:09.683065 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:53:09.682632 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3-registry-tls\") pod \"88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3\" (UID: \"88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3\") " Jul 23 21:53:09.683065 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:53:09.682672 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3-ca-trust-extracted\") pod \"88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3\" (UID: \"88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3\") " Jul 23 21:53:09.683559 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:53:09.683506 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3" (UID: "88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 21:53:09.685216 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:53:09.685184 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3" (UID: "88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 21:53:09.688396 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:53:09.688357 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3" (UID: "88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 21:53:09.693827 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:53:09.693157 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3" (UID: "88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 21:53:09.694151 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:53:09.694109 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3-kube-api-access-7tg7w" (OuterVolumeSpecName: "kube-api-access-7tg7w") pod "88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3" (UID: "88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3"). InnerVolumeSpecName "kube-api-access-7tg7w". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 21:53:09.694239 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:53:09.694200 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3" (UID: "88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 21:53:09.694488 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:53:09.694325 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3" (UID: "88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 21:53:09.696147 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:53:09.696129 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3" (UID: "88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 21:53:09.730067 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:53:09.730046 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-fbbbb55d8-pz9t5" Jul 23 21:53:09.730412 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:53:09.730046 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-fbbbb55d8-pz9t5" event={"ID":"88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3","Type":"ContainerDied","Data":"25ad7a6ce3d3cea00d0887c5b40aff62b46e72ab2fd922696ae119086a6c264c"} Jul 23 21:53:09.730412 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:53:09.730194 2578 scope.go:117] "RemoveContainer" containerID="25ff99f7d0736b7918e8bcdec808981d4771823aa557e088283a6ef8beca0691" Jul 23 21:53:09.753995 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:53:09.753973 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-fbbbb55d8-pz9t5"] Jul 23 21:53:09.762056 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:53:09.762032 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-fbbbb55d8-pz9t5"] Jul 23 21:53:09.783199 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:53:09.783179 2578 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3-bound-sa-token\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\"" Jul 23 21:53:09.783295 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:53:09.783204 2578 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3-registry-tls\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\"" Jul 23 21:53:09.783295 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:53:09.783217 2578 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3-ca-trust-extracted\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\"" Jul 23 21:53:09.783295 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:53:09.783229 2578 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3-registry-certificates\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\"" Jul 23 21:53:09.783295 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:53:09.783244 2578 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3-image-registry-private-configuration\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\"" Jul 23 21:53:09.783295 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:53:09.783257 2578 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3-trusted-ca\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\"" Jul 23 21:53:09.783295 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:53:09.783269 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7tg7w\" (UniqueName: \"kubernetes.io/projected/88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3-kube-api-access-7tg7w\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\"" Jul 23 21:53:09.783295 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:53:09.783283 2578 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3-installation-pull-secrets\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\"" Jul 23 21:53:10.253761 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:53:10.253727 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3" path="/var/lib/kubelet/pods/88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3/volumes" Jul 23 21:53:10.741164 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:53:10.740645 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5cb6c9bd46-gwkr9" event={"ID":"a82ef530-e2e9-4234-b23f-1230edd4b83c","Type":"ContainerStarted","Data":"cf4bf651db75f61e383baa47f49a58bb7a8ba9211b60adee178f1c171ac06618"} Jul 23 21:53:10.744937 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:53:10.744903 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-55ff89fbc9-t5zhk" event={"ID":"034d2dc9-a711-493f-b5bc-6f772cf1641f","Type":"ContainerStarted","Data":"364a4f2bfc3caf3564ff5d0dbecf1cb3acb41726af81ab2892e446a1f2be7f0e"} Jul 23 21:53:10.745072 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:53:10.744980 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-55ff89fbc9-t5zhk" event={"ID":"034d2dc9-a711-493f-b5bc-6f772cf1641f","Type":"ContainerStarted","Data":"6603ed6c5600879f23c4f9c3172780736168a46ea19deaa7e3072874f8a9234a"} Jul 23 21:53:10.745072 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:53:10.745023 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-55ff89fbc9-t5zhk" event={"ID":"034d2dc9-a711-493f-b5bc-6f772cf1641f","Type":"ContainerStarted","Data":"41385c4314efda9886b807236f1576733369832c4b51be6f74dc552a0a3c295c"} Jul 23 21:53:10.747282 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:53:10.747237 2578 generic.go:356] "Generic (PLEG): container finished" podID="98fdd345-9020-4d2f-9ce2-9a392c19218c" containerID="8aaa385fe0eab3d8a073c16f7177292ec74df05fe3f122a51dc1b5f62ef59ff7" exitCode=0 Jul 23 21:53:10.747532 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:53:10.747321 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9szvj" event={"ID":"98fdd345-9020-4d2f-9ce2-9a392c19218c","Type":"ContainerDied","Data":"8aaa385fe0eab3d8a073c16f7177292ec74df05fe3f122a51dc1b5f62ef59ff7"} Jul 23 21:53:10.750581 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:53:10.750522 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fkqwr" event={"ID":"f1e7deb0-a030-4479-b82a-5dbbc864104b","Type":"ContainerStarted","Data":"6153d766c607fbe2a94d99a7cd87d15b5458e9fab7f6467c13899846e2a6ffeb"} Jul 23 21:53:10.760482 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:53:10.760440 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-5cb6c9bd46-gwkr9" podStartSLOduration=7.017551797 podStartE2EDuration="13.760425691s" totalImagesPullingTime="6.742873894s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:52:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:53:10.758866925 +0000 UTC m=+93.000774799" watchObservedRunningTime="2026-07-23 21:53:10.760425691 +0000 UTC m=+93.002333565" Jul 23 21:53:11.757592 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:53:11.757560 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9szvj" event={"ID":"98fdd345-9020-4d2f-9ce2-9a392c19218c","Type":"ContainerStarted","Data":"fc49b89f10454020073eb5188da35810f08b4045719980ad83f2c17f609cb3a9"} Jul 23 21:53:11.757982 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:53:11.757624 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9szvj" event={"ID":"98fdd345-9020-4d2f-9ce2-9a392c19218c","Type":"ContainerStarted","Data":"98d1886f5e5bf95da5ae48add8e4038e1e72df46f1cb7ce2ee8cdbf291ce9025"} Jul 23 21:53:12.766171 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:53:12.766128 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-55ff89fbc9-t5zhk" event={"ID":"034d2dc9-a711-493f-b5bc-6f772cf1641f","Type":"ContainerStarted","Data":"ee09acdb645c8047af2c5616f17e90ed96001da39ea4edabde2424f829ebd9d2"} Jul 23 21:53:12.766728 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:53:12.766220 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-55ff89fbc9-t5zhk" event={"ID":"034d2dc9-a711-493f-b5bc-6f772cf1641f","Type":"ContainerStarted","Data":"94bba1bd582a41725fd9f344aa474ae28b7b80b7d945fc9e86e8367843b27421"} Jul 23 21:53:12.766728 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:53:12.766268 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-55ff89fbc9-t5zhk" event={"ID":"034d2dc9-a711-493f-b5bc-6f772cf1641f","Type":"ContainerStarted","Data":"d4ce52890d8a24f5d93ef9eb981fb24d32488db38079efc40fb18221fa6ee354"} Jul 23 21:53:12.794842 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:53:12.794676 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-55ff89fbc9-t5zhk" podStartSLOduration=9.648946997 podStartE2EDuration="17.794661012s" totalImagesPullingTime="8.145714015s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:52:55 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:53:12.792420901 +0000 UTC m=+95.034328776" watchObservedRunningTime="2026-07-23 21:53:12.794661012 +0000 UTC m=+95.036568885" Jul 23 21:53:12.795011 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:53:12.794866 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-9szvj" podStartSLOduration=13.988326248 podStartE2EDuration="19.794857683s" totalImagesPullingTime="5.694589695s" totalInitContainerRuntime="111.94174ms" isStatefulPod=true podCreationTimestamp="2026-07-23 21:52:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:53:11.790152813 +0000 UTC m=+94.032060686" watchObservedRunningTime="2026-07-23 21:53:12.794857683 +0000 UTC m=+95.036765558" Jul 23 21:53:13.772962 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:53:13.772931 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-55ff89fbc9-t5zhk" Jul 23 21:53:14.783604 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:53:14.783490 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fkqwr" event={"ID":"f1e7deb0-a030-4479-b82a-5dbbc864104b","Type":"ContainerStarted","Data":"4a9c6d532e544fee9caa3ecc39eafbd9f3e589ebe80b9151b821e91bf1eaa111"} Jul 23 21:53:14.801576 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:53:14.801519 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-fkqwr" podStartSLOduration=33.645543666 podStartE2EDuration="56.801506412s" totalImagesPullingTime="23.155962746s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 21:52:18 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:53:14.800522818 +0000 UTC m=+97.042430692" watchObservedRunningTime="2026-07-23 21:53:14.801506412 +0000 UTC m=+97.043414285" Jul 23 21:53:18.009710 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:53:18.009678 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-5cb6c9bd46-gwkr9" Jul 23 21:53:18.010126 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:53:18.009736 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-5cb6c9bd46-gwkr9" Jul 23 21:53:19.789470 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:53:19.789443 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-55ff89fbc9-t5zhk" Jul 23 21:53:21.603618 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:53:21.603587 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-4cq9t" Jul 23 21:53:38.014289 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:53:38.014258 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-5cb6c9bd46-gwkr9" Jul 23 21:53:38.017825 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:53:38.017802 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-5cb6c9bd46-gwkr9" Jul 23 21:56:29.453690 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:56:29.453659 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gqvlh"] Jul 23 21:56:29.454103 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:56:29.453939 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3" containerName="registry" Jul 23 21:56:29.454103 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:56:29.453949 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3" containerName="registry" Jul 23 21:56:29.454103 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:56:29.453998 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="88186fa0-4ba3-4bbc-a9ba-7c13d3e125f3" containerName="registry" Jul 23 21:56:29.456109 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:56:29.456092 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gqvlh" Jul 23 21:56:29.458774 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:56:29.458751 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 23 21:56:29.459683 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:56:29.459656 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 23 21:56:29.459810 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:56:29.459656 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-mwnm7\"" Jul 23 21:56:29.466819 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:56:29.466791 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gqvlh"] Jul 23 21:56:29.553238 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:56:29.553198 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f6999e23-3782-4b40-b2b6-5f5b3c80b444-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gqvlh\" (UID: \"f6999e23-3782-4b40-b2b6-5f5b3c80b444\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gqvlh" Jul 23 21:56:29.553429 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:56:29.553262 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ghg5q\" (UniqueName: \"kubernetes.io/projected/f6999e23-3782-4b40-b2b6-5f5b3c80b444-kube-api-access-ghg5q\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gqvlh\" (UID: \"f6999e23-3782-4b40-b2b6-5f5b3c80b444\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gqvlh" Jul 23 21:56:29.553429 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:56:29.553364 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f6999e23-3782-4b40-b2b6-5f5b3c80b444-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gqvlh\" (UID: \"f6999e23-3782-4b40-b2b6-5f5b3c80b444\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gqvlh" Jul 23 21:56:29.653907 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:56:29.653870 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f6999e23-3782-4b40-b2b6-5f5b3c80b444-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gqvlh\" (UID: \"f6999e23-3782-4b40-b2b6-5f5b3c80b444\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gqvlh" Jul 23 21:56:29.654048 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:56:29.653946 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f6999e23-3782-4b40-b2b6-5f5b3c80b444-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gqvlh\" (UID: \"f6999e23-3782-4b40-b2b6-5f5b3c80b444\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gqvlh" Jul 23 21:56:29.654048 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:56:29.654000 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ghg5q\" (UniqueName: \"kubernetes.io/projected/f6999e23-3782-4b40-b2b6-5f5b3c80b444-kube-api-access-ghg5q\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gqvlh\" (UID: \"f6999e23-3782-4b40-b2b6-5f5b3c80b444\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gqvlh" Jul 23 21:56:29.654164 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:56:29.654144 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f6999e23-3782-4b40-b2b6-5f5b3c80b444-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gqvlh\" (UID: \"f6999e23-3782-4b40-b2b6-5f5b3c80b444\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gqvlh" Jul 23 21:56:29.654347 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:56:29.654326 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f6999e23-3782-4b40-b2b6-5f5b3c80b444-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gqvlh\" (UID: \"f6999e23-3782-4b40-b2b6-5f5b3c80b444\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gqvlh" Jul 23 21:56:29.662953 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:56:29.662921 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ghg5q\" (UniqueName: \"kubernetes.io/projected/f6999e23-3782-4b40-b2b6-5f5b3c80b444-kube-api-access-ghg5q\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gqvlh\" (UID: \"f6999e23-3782-4b40-b2b6-5f5b3c80b444\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gqvlh" Jul 23 21:56:29.764653 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:56:29.764560 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gqvlh" Jul 23 21:56:29.893625 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:56:29.893592 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gqvlh"] Jul 23 21:56:29.897296 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:56:29.897264 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6999e23_3782_4b40_b2b6_5f5b3c80b444.slice/crio-61aeac3a8b01cd595dbb7eaccc583675c67cf3d0de1166c1fc472bebf3318ad1 WatchSource:0}: Error finding container 61aeac3a8b01cd595dbb7eaccc583675c67cf3d0de1166c1fc472bebf3318ad1: Status 404 returned error can't find the container with id 61aeac3a8b01cd595dbb7eaccc583675c67cf3d0de1166c1fc472bebf3318ad1 Jul 23 21:56:30.451053 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:56:30.451015 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gqvlh" event={"ID":"f6999e23-3782-4b40-b2b6-5f5b3c80b444","Type":"ContainerStarted","Data":"61aeac3a8b01cd595dbb7eaccc583675c67cf3d0de1166c1fc472bebf3318ad1"} Jul 23 21:56:36.477612 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:56:36.477570 2578 generic.go:356] "Generic (PLEG): container finished" podID="f6999e23-3782-4b40-b2b6-5f5b3c80b444" containerID="430b3e0c76c665882cd6f2f3b11df56fea6799f1c67171f2ec8ec99d2bcfeb79" exitCode=0 Jul 23 21:56:36.478014 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:56:36.477646 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gqvlh" event={"ID":"f6999e23-3782-4b40-b2b6-5f5b3c80b444","Type":"ContainerDied","Data":"430b3e0c76c665882cd6f2f3b11df56fea6799f1c67171f2ec8ec99d2bcfeb79"} Jul 23 21:56:38.496775 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:56:38.496740 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7jkjm_7342a9f6-72b1-4e3c-8d50-0b0326454a86/ovn-acl-logging/0.log" Jul 23 21:56:38.499300 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:56:38.496814 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7jkjm_7342a9f6-72b1-4e3c-8d50-0b0326454a86/ovn-acl-logging/0.log" Jul 23 21:56:38.503027 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:56:38.503008 2578 kubelet.go:1648] "Image garbage collection succeeded" Jul 23 21:56:39.492286 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:56:39.492254 2578 generic.go:356] "Generic (PLEG): container finished" podID="f6999e23-3782-4b40-b2b6-5f5b3c80b444" containerID="5f5edadc2388aa499ca1f932eb674c825034c06a4fa22a6177b74622eca1545f" exitCode=0 Jul 23 21:56:39.492475 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:56:39.492349 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gqvlh" event={"ID":"f6999e23-3782-4b40-b2b6-5f5b3c80b444","Type":"ContainerDied","Data":"5f5edadc2388aa499ca1f932eb674c825034c06a4fa22a6177b74622eca1545f"} Jul 23 21:56:39.493329 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:56:39.493313 2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 23 21:56:46.522843 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:56:46.522804 2578 generic.go:356] "Generic (PLEG): container finished" podID="f6999e23-3782-4b40-b2b6-5f5b3c80b444" containerID="9bbcc42ceee9f6487d293f0086c5fdc1ec801e5b0390617cbf086c5fccc3fb4b" exitCode=0 Jul 23 21:56:46.523246 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:56:46.522884 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gqvlh" event={"ID":"f6999e23-3782-4b40-b2b6-5f5b3c80b444","Type":"ContainerDied","Data":"9bbcc42ceee9f6487d293f0086c5fdc1ec801e5b0390617cbf086c5fccc3fb4b"} Jul 23 21:56:47.654573 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:56:47.654548 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gqvlh" Jul 23 21:56:47.804777 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:56:47.804751 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f6999e23-3782-4b40-b2b6-5f5b3c80b444-util\") pod \"f6999e23-3782-4b40-b2b6-5f5b3c80b444\" (UID: \"f6999e23-3782-4b40-b2b6-5f5b3c80b444\") " Jul 23 21:56:47.804948 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:56:47.804808 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f6999e23-3782-4b40-b2b6-5f5b3c80b444-bundle\") pod \"f6999e23-3782-4b40-b2b6-5f5b3c80b444\" (UID: \"f6999e23-3782-4b40-b2b6-5f5b3c80b444\") " Jul 23 21:56:47.804948 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:56:47.804848 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ghg5q\" (UniqueName: \"kubernetes.io/projected/f6999e23-3782-4b40-b2b6-5f5b3c80b444-kube-api-access-ghg5q\") pod \"f6999e23-3782-4b40-b2b6-5f5b3c80b444\" (UID: \"f6999e23-3782-4b40-b2b6-5f5b3c80b444\") " Jul 23 21:56:47.805590 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:56:47.805443 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6999e23-3782-4b40-b2b6-5f5b3c80b444-bundle" (OuterVolumeSpecName: "bundle") pod "f6999e23-3782-4b40-b2b6-5f5b3c80b444" (UID: "f6999e23-3782-4b40-b2b6-5f5b3c80b444"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 21:56:47.807131 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:56:47.807109 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f6999e23-3782-4b40-b2b6-5f5b3c80b444-kube-api-access-ghg5q" (OuterVolumeSpecName: "kube-api-access-ghg5q") pod "f6999e23-3782-4b40-b2b6-5f5b3c80b444" (UID: "f6999e23-3782-4b40-b2b6-5f5b3c80b444"). InnerVolumeSpecName "kube-api-access-ghg5q". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 21:56:47.809974 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:56:47.809949 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6999e23-3782-4b40-b2b6-5f5b3c80b444-util" (OuterVolumeSpecName: "util") pod "f6999e23-3782-4b40-b2b6-5f5b3c80b444" (UID: "f6999e23-3782-4b40-b2b6-5f5b3c80b444"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 21:56:47.905418 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:56:47.905373 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ghg5q\" (UniqueName: \"kubernetes.io/projected/f6999e23-3782-4b40-b2b6-5f5b3c80b444-kube-api-access-ghg5q\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\"" Jul 23 21:56:47.905418 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:56:47.905412 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f6999e23-3782-4b40-b2b6-5f5b3c80b444-util\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\"" Jul 23 21:56:47.905418 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:56:47.905423 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f6999e23-3782-4b40-b2b6-5f5b3c80b444-bundle\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\"" Jul 23 21:56:48.532430 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:56:48.532352 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gqvlh" Jul 23 21:56:48.532615 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:56:48.532349 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540gqvlh" event={"ID":"f6999e23-3782-4b40-b2b6-5f5b3c80b444","Type":"ContainerDied","Data":"61aeac3a8b01cd595dbb7eaccc583675c67cf3d0de1166c1fc472bebf3318ad1"} Jul 23 21:56:48.532615 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:56:48.532466 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="61aeac3a8b01cd595dbb7eaccc583675c67cf3d0de1166c1fc472bebf3318ad1" Jul 23 21:56:56.626513 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:56:56.626476 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-rjcr7"] Jul 23 21:56:56.626897 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:56:56.626775 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f6999e23-3782-4b40-b2b6-5f5b3c80b444" containerName="pull" Jul 23 21:56:56.626897 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:56:56.626786 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="f6999e23-3782-4b40-b2b6-5f5b3c80b444" containerName="pull" Jul 23 21:56:56.626897 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:56:56.626796 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f6999e23-3782-4b40-b2b6-5f5b3c80b444" containerName="extract" Jul 23 21:56:56.626897 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:56:56.626801 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="f6999e23-3782-4b40-b2b6-5f5b3c80b444" containerName="extract" Jul 23 21:56:56.626897 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:56:56.626818 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f6999e23-3782-4b40-b2b6-5f5b3c80b444" containerName="util" Jul 23 21:56:56.626897 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:56:56.626826 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="f6999e23-3782-4b40-b2b6-5f5b3c80b444" containerName="util" Jul 23 21:56:56.626897 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:56:56.626877 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="f6999e23-3782-4b40-b2b6-5f5b3c80b444" containerName="extract" Jul 23 21:56:56.628817 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:56:56.628802 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-rjcr7" Jul 23 21:56:56.632534 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:56:56.632515 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 23 21:56:56.633439 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:56:56.633420 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 23 21:56:56.633570 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:56:56.633424 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-2qtd9\"" Jul 23 21:56:56.651679 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:56:56.651648 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-rjcr7"] Jul 23 21:56:56.671507 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:56:56.671479 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mxqz2\" (UniqueName: \"kubernetes.io/projected/2ec8119c-3c4e-4b2c-9320-0f783b9ec3f5-kube-api-access-mxqz2\") pod \"cert-manager-webhook-6486ffd997-rjcr7\" (UID: \"2ec8119c-3c4e-4b2c-9320-0f783b9ec3f5\") " pod="cert-manager/cert-manager-webhook-6486ffd997-rjcr7" Jul 23 21:56:56.671641 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:56:56.671516 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2ec8119c-3c4e-4b2c-9320-0f783b9ec3f5-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-rjcr7\" (UID: \"2ec8119c-3c4e-4b2c-9320-0f783b9ec3f5\") " pod="cert-manager/cert-manager-webhook-6486ffd997-rjcr7" Jul 23 21:56:56.772678 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:56:56.772649 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mxqz2\" (UniqueName: \"kubernetes.io/projected/2ec8119c-3c4e-4b2c-9320-0f783b9ec3f5-kube-api-access-mxqz2\") pod \"cert-manager-webhook-6486ffd997-rjcr7\" (UID: \"2ec8119c-3c4e-4b2c-9320-0f783b9ec3f5\") " pod="cert-manager/cert-manager-webhook-6486ffd997-rjcr7" Jul 23 21:56:56.772821 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:56:56.772685 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2ec8119c-3c4e-4b2c-9320-0f783b9ec3f5-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-rjcr7\" (UID: \"2ec8119c-3c4e-4b2c-9320-0f783b9ec3f5\") " pod="cert-manager/cert-manager-webhook-6486ffd997-rjcr7" Jul 23 21:56:56.800934 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:56:56.800907 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2ec8119c-3c4e-4b2c-9320-0f783b9ec3f5-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-rjcr7\" (UID: \"2ec8119c-3c4e-4b2c-9320-0f783b9ec3f5\") " pod="cert-manager/cert-manager-webhook-6486ffd997-rjcr7" Jul 23 21:56:56.814749 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:56:56.814721 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mxqz2\" (UniqueName: \"kubernetes.io/projected/2ec8119c-3c4e-4b2c-9320-0f783b9ec3f5-kube-api-access-mxqz2\") pod \"cert-manager-webhook-6486ffd997-rjcr7\" (UID: \"2ec8119c-3c4e-4b2c-9320-0f783b9ec3f5\") " pod="cert-manager/cert-manager-webhook-6486ffd997-rjcr7" Jul 23 21:56:56.938114 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:56:56.938035 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-rjcr7" Jul 23 21:56:57.050939 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:56:57.050908 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhdlb8"] Jul 23 21:56:57.054044 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:56:57.054020 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhdlb8" Jul 23 21:56:57.056456 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:56:57.056431 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 23 21:56:57.057334 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:56:57.057318 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-mwnm7\"" Jul 23 21:56:57.057334 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:56:57.057326 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 23 21:56:57.068062 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:56:57.068038 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhdlb8"] Jul 23 21:56:57.072183 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:56:57.072162 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-rjcr7"] Jul 23 21:56:57.074819 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:56:57.074791 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2ec8119c_3c4e_4b2c_9320_0f783b9ec3f5.slice/crio-e786df5de3351529e8bd16bd2ecbe960d5ea82bbb94aa435df771b150a65ee0b WatchSource:0}: Error finding container e786df5de3351529e8bd16bd2ecbe960d5ea82bbb94aa435df771b150a65ee0b: Status 404 returned error can't find the container with id e786df5de3351529e8bd16bd2ecbe960d5ea82bbb94aa435df771b150a65ee0b Jul 23 21:56:57.075662 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:56:57.075637 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e0c90fe0-9a3e-40e1-99af-96dad0f84771-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhdlb8\" (UID: \"e0c90fe0-9a3e-40e1-99af-96dad0f84771\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhdlb8" Jul 23 21:56:57.075776 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:56:57.075694 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e0c90fe0-9a3e-40e1-99af-96dad0f84771-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhdlb8\" (UID: \"e0c90fe0-9a3e-40e1-99af-96dad0f84771\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhdlb8" Jul 23 21:56:57.075776 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:56:57.075727 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n68dq\" (UniqueName: \"kubernetes.io/projected/e0c90fe0-9a3e-40e1-99af-96dad0f84771-kube-api-access-n68dq\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhdlb8\" (UID: \"e0c90fe0-9a3e-40e1-99af-96dad0f84771\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhdlb8" Jul 23 21:56:57.176318 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:56:57.176275 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e0c90fe0-9a3e-40e1-99af-96dad0f84771-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhdlb8\" (UID: \"e0c90fe0-9a3e-40e1-99af-96dad0f84771\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhdlb8" Jul 23 21:56:57.176478 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:56:57.176326 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e0c90fe0-9a3e-40e1-99af-96dad0f84771-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhdlb8\" (UID: \"e0c90fe0-9a3e-40e1-99af-96dad0f84771\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhdlb8" Jul 23 21:56:57.176478 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:56:57.176346 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n68dq\" (UniqueName: \"kubernetes.io/projected/e0c90fe0-9a3e-40e1-99af-96dad0f84771-kube-api-access-n68dq\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhdlb8\" (UID: \"e0c90fe0-9a3e-40e1-99af-96dad0f84771\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhdlb8" Jul 23 21:56:57.176679 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:56:57.176659 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e0c90fe0-9a3e-40e1-99af-96dad0f84771-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhdlb8\" (UID: \"e0c90fe0-9a3e-40e1-99af-96dad0f84771\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhdlb8" Jul 23 21:56:57.176722 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:56:57.176694 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e0c90fe0-9a3e-40e1-99af-96dad0f84771-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhdlb8\" (UID: \"e0c90fe0-9a3e-40e1-99af-96dad0f84771\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhdlb8" Jul 23 21:56:57.187121 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:56:57.187095 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n68dq\" (UniqueName: \"kubernetes.io/projected/e0c90fe0-9a3e-40e1-99af-96dad0f84771-kube-api-access-n68dq\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhdlb8\" (UID: \"e0c90fe0-9a3e-40e1-99af-96dad0f84771\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhdlb8" Jul 23 21:56:57.363071 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:56:57.363040 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhdlb8" Jul 23 21:56:57.489630 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:56:57.489599 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhdlb8"] Jul 23 21:56:57.491888 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:56:57.491864 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0c90fe0_9a3e_40e1_99af_96dad0f84771.slice/crio-7c2db847c594485f3324e29f807a37821cbdd68d65839a4acbeb06423c085f85 WatchSource:0}: Error finding container 7c2db847c594485f3324e29f807a37821cbdd68d65839a4acbeb06423c085f85: Status 404 returned error can't find the container with id 7c2db847c594485f3324e29f807a37821cbdd68d65839a4acbeb06423c085f85 Jul 23 21:56:57.564840 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:56:57.564814 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-rjcr7" event={"ID":"2ec8119c-3c4e-4b2c-9320-0f783b9ec3f5","Type":"ContainerStarted","Data":"e786df5de3351529e8bd16bd2ecbe960d5ea82bbb94aa435df771b150a65ee0b"} Jul 23 21:56:57.566334 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:56:57.566308 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhdlb8" event={"ID":"e0c90fe0-9a3e-40e1-99af-96dad0f84771","Type":"ContainerStarted","Data":"e90bf846b4263367abcbb70d8beca69c0dc4507422b6c1c73a36ab3db32393c2"} Jul 23 21:56:57.566437 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:56:57.566357 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhdlb8" event={"ID":"e0c90fe0-9a3e-40e1-99af-96dad0f84771","Type":"ContainerStarted","Data":"7c2db847c594485f3324e29f807a37821cbdd68d65839a4acbeb06423c085f85"} Jul 23 21:56:58.572598 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:56:58.572559 2578 generic.go:356] "Generic (PLEG): container finished" podID="e0c90fe0-9a3e-40e1-99af-96dad0f84771" containerID="e90bf846b4263367abcbb70d8beca69c0dc4507422b6c1c73a36ab3db32393c2" exitCode=0 Jul 23 21:56:58.573168 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:56:58.572629 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhdlb8" event={"ID":"e0c90fe0-9a3e-40e1-99af-96dad0f84771","Type":"ContainerDied","Data":"e90bf846b4263367abcbb70d8beca69c0dc4507422b6c1c73a36ab3db32393c2"} Jul 23 21:57:00.583682 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:00.583585 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-rjcr7" event={"ID":"2ec8119c-3c4e-4b2c-9320-0f783b9ec3f5","Type":"ContainerStarted","Data":"366da84b6b9fcc77ace3d2bb776120e48a754cc420085a59a93337d50e98088a"} Jul 23 21:57:00.584089 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:00.583702 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-rjcr7" Jul 23 21:57:00.601204 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:00.601118 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-rjcr7" podStartSLOduration=1.951236457 podStartE2EDuration="4.601101296s" totalImagesPullingTime="2.649864839s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:56:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:57:00.600029849 +0000 UTC m=+322.841937722" watchObservedRunningTime="2026-07-23 21:57:00.601101296 +0000 UTC m=+322.843009168" Jul 23 21:57:01.589025 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:01.588990 2578 generic.go:356] "Generic (PLEG): container finished" podID="e0c90fe0-9a3e-40e1-99af-96dad0f84771" containerID="889507799db42766cbd1cde281a7b450d5e9464c21bfd9bbe4ea88847384846a" exitCode=0 Jul 23 21:57:01.589415 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:01.589068 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhdlb8" event={"ID":"e0c90fe0-9a3e-40e1-99af-96dad0f84771","Type":"ContainerDied","Data":"889507799db42766cbd1cde281a7b450d5e9464c21bfd9bbe4ea88847384846a"} Jul 23 21:57:02.594310 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:02.594273 2578 generic.go:356] "Generic (PLEG): container finished" podID="e0c90fe0-9a3e-40e1-99af-96dad0f84771" containerID="5934be0e2f1ec0aff029a26c76bb2ebdf146eacbf807451d50aa9ee903726da5" exitCode=0 Jul 23 21:57:02.594702 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:02.594329 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhdlb8" event={"ID":"e0c90fe0-9a3e-40e1-99af-96dad0f84771","Type":"ContainerDied","Data":"5934be0e2f1ec0aff029a26c76bb2ebdf146eacbf807451d50aa9ee903726da5"} Jul 23 21:57:03.719450 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:03.719428 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhdlb8" Jul 23 21:57:03.835652 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:03.835615 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e0c90fe0-9a3e-40e1-99af-96dad0f84771-bundle\") pod \"e0c90fe0-9a3e-40e1-99af-96dad0f84771\" (UID: \"e0c90fe0-9a3e-40e1-99af-96dad0f84771\") " Jul 23 21:57:03.835652 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:03.835656 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e0c90fe0-9a3e-40e1-99af-96dad0f84771-util\") pod \"e0c90fe0-9a3e-40e1-99af-96dad0f84771\" (UID: \"e0c90fe0-9a3e-40e1-99af-96dad0f84771\") " Jul 23 21:57:03.835844 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:03.835717 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n68dq\" (UniqueName: \"kubernetes.io/projected/e0c90fe0-9a3e-40e1-99af-96dad0f84771-kube-api-access-n68dq\") pod \"e0c90fe0-9a3e-40e1-99af-96dad0f84771\" (UID: \"e0c90fe0-9a3e-40e1-99af-96dad0f84771\") " Jul 23 21:57:03.836081 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:03.836056 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0c90fe0-9a3e-40e1-99af-96dad0f84771-bundle" (OuterVolumeSpecName: "bundle") pod "e0c90fe0-9a3e-40e1-99af-96dad0f84771" (UID: "e0c90fe0-9a3e-40e1-99af-96dad0f84771"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 21:57:03.837969 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:03.837950 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e0c90fe0-9a3e-40e1-99af-96dad0f84771-kube-api-access-n68dq" (OuterVolumeSpecName: "kube-api-access-n68dq") pod "e0c90fe0-9a3e-40e1-99af-96dad0f84771" (UID: "e0c90fe0-9a3e-40e1-99af-96dad0f84771"). InnerVolumeSpecName "kube-api-access-n68dq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 21:57:03.839700 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:03.839679 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e0c90fe0-9a3e-40e1-99af-96dad0f84771-util" (OuterVolumeSpecName: "util") pod "e0c90fe0-9a3e-40e1-99af-96dad0f84771" (UID: "e0c90fe0-9a3e-40e1-99af-96dad0f84771"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 21:57:03.937213 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:03.937138 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e0c90fe0-9a3e-40e1-99af-96dad0f84771-bundle\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\"" Jul 23 21:57:03.937213 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:03.937165 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e0c90fe0-9a3e-40e1-99af-96dad0f84771-util\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\"" Jul 23 21:57:03.937213 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:03.937175 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n68dq\" (UniqueName: \"kubernetes.io/projected/e0c90fe0-9a3e-40e1-99af-96dad0f84771-kube-api-access-n68dq\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\"" Jul 23 21:57:04.608140 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:04.608112 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhdlb8" Jul 23 21:57:04.608281 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:04.608114 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fhdlb8" event={"ID":"e0c90fe0-9a3e-40e1-99af-96dad0f84771","Type":"ContainerDied","Data":"7c2db847c594485f3324e29f807a37821cbdd68d65839a4acbeb06423c085f85"} Jul 23 21:57:04.608281 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:04.608250 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7c2db847c594485f3324e29f807a37821cbdd68d65839a4acbeb06423c085f85" Jul 23 21:57:06.591281 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:06.591248 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-rjcr7" Jul 23 21:57:08.126492 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:08.126459 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-x9n8f"] Jul 23 21:57:08.126910 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:08.126746 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e0c90fe0-9a3e-40e1-99af-96dad0f84771" containerName="util" Jul 23 21:57:08.126910 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:08.126758 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e0c90fe0-9a3e-40e1-99af-96dad0f84771" containerName="util" Jul 23 21:57:08.126910 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:08.126772 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e0c90fe0-9a3e-40e1-99af-96dad0f84771" containerName="pull" Jul 23 21:57:08.126910 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:08.126778 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e0c90fe0-9a3e-40e1-99af-96dad0f84771" containerName="pull" Jul 23 21:57:08.126910 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:08.126788 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e0c90fe0-9a3e-40e1-99af-96dad0f84771" containerName="extract" Jul 23 21:57:08.126910 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:08.126794 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e0c90fe0-9a3e-40e1-99af-96dad0f84771" containerName="extract" Jul 23 21:57:08.126910 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:08.126839 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="e0c90fe0-9a3e-40e1-99af-96dad0f84771" containerName="extract" Jul 23 21:57:08.129611 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:08.129595 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-x9n8f" Jul 23 21:57:08.131898 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:08.131882 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-hlnmp\"" Jul 23 21:57:08.139326 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:08.139307 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-x9n8f"] Jul 23 21:57:08.168080 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:08.168055 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vgqjp\" (UniqueName: \"kubernetes.io/projected/123230ba-9892-43ae-9fd8-d274c553157a-kube-api-access-vgqjp\") pod \"cert-manager-55b4d954d5-x9n8f\" (UID: \"123230ba-9892-43ae-9fd8-d274c553157a\") " pod="cert-manager/cert-manager-55b4d954d5-x9n8f" Jul 23 21:57:08.168209 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:08.168095 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/123230ba-9892-43ae-9fd8-d274c553157a-bound-sa-token\") pod \"cert-manager-55b4d954d5-x9n8f\" (UID: \"123230ba-9892-43ae-9fd8-d274c553157a\") " pod="cert-manager/cert-manager-55b4d954d5-x9n8f" Jul 23 21:57:08.268676 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:08.268652 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vgqjp\" (UniqueName: \"kubernetes.io/projected/123230ba-9892-43ae-9fd8-d274c553157a-kube-api-access-vgqjp\") pod \"cert-manager-55b4d954d5-x9n8f\" (UID: \"123230ba-9892-43ae-9fd8-d274c553157a\") " pod="cert-manager/cert-manager-55b4d954d5-x9n8f" Jul 23 21:57:08.268815 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:08.268692 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/123230ba-9892-43ae-9fd8-d274c553157a-bound-sa-token\") pod \"cert-manager-55b4d954d5-x9n8f\" (UID: \"123230ba-9892-43ae-9fd8-d274c553157a\") " pod="cert-manager/cert-manager-55b4d954d5-x9n8f" Jul 23 21:57:08.276648 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:08.276623 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/123230ba-9892-43ae-9fd8-d274c553157a-bound-sa-token\") pod \"cert-manager-55b4d954d5-x9n8f\" (UID: \"123230ba-9892-43ae-9fd8-d274c553157a\") " pod="cert-manager/cert-manager-55b4d954d5-x9n8f" Jul 23 21:57:08.276783 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:08.276726 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vgqjp\" (UniqueName: \"kubernetes.io/projected/123230ba-9892-43ae-9fd8-d274c553157a-kube-api-access-vgqjp\") pod \"cert-manager-55b4d954d5-x9n8f\" (UID: \"123230ba-9892-43ae-9fd8-d274c553157a\") " pod="cert-manager/cert-manager-55b4d954d5-x9n8f" Jul 23 21:57:08.438568 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:08.438479 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-x9n8f" Jul 23 21:57:08.566624 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:08.566584 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-x9n8f"] Jul 23 21:57:08.568924 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:57:08.568897 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod123230ba_9892_43ae_9fd8_d274c553157a.slice/crio-e1fcb137000c44472ecc4ad6f9514fcce8794556c72bff0c6c5fd346d7c67361 WatchSource:0}: Error finding container e1fcb137000c44472ecc4ad6f9514fcce8794556c72bff0c6c5fd346d7c67361: Status 404 returned error can't find the container with id e1fcb137000c44472ecc4ad6f9514fcce8794556c72bff0c6c5fd346d7c67361 Jul 23 21:57:08.624451 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:08.624421 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-x9n8f" event={"ID":"123230ba-9892-43ae-9fd8-d274c553157a","Type":"ContainerStarted","Data":"e1fcb137000c44472ecc4ad6f9514fcce8794556c72bff0c6c5fd346d7c67361"} Jul 23 21:57:09.630008 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:09.629971 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-x9n8f" event={"ID":"123230ba-9892-43ae-9fd8-d274c553157a","Type":"ContainerStarted","Data":"b0b7e88a6a37ec539391cb66a1568007175c213353f77b53117dc101468caeb5"} Jul 23 21:57:09.659904 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:09.659855 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-x9n8f" podStartSLOduration=1.659840252 podStartE2EDuration="1.659840252s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:57:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:57:09.658291972 +0000 UTC m=+331.900199856" watchObservedRunningTime="2026-07-23 21:57:09.659840252 +0000 UTC m=+331.901748125" Jul 23 21:57:21.584679 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:21.584643 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afs58th"] Jul 23 21:57:21.591084 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:21.591064 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afs58th" Jul 23 21:57:21.593966 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:21.593948 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 23 21:57:21.594963 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:21.594945 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 23 21:57:21.595507 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:21.595490 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-mwnm7\"" Jul 23 21:57:21.607701 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:21.607680 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afs58th"] Jul 23 21:57:21.680278 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:21.680246 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/536f6328-45cd-4376-9c05-96f8a78d9a87-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afs58th\" (UID: \"536f6328-45cd-4376-9c05-96f8a78d9a87\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afs58th" Jul 23 21:57:21.680444 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:21.680288 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kzswd\" (UniqueName: \"kubernetes.io/projected/536f6328-45cd-4376-9c05-96f8a78d9a87-kube-api-access-kzswd\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afs58th\" (UID: \"536f6328-45cd-4376-9c05-96f8a78d9a87\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afs58th" Jul 23 21:57:21.680444 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:21.680311 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/536f6328-45cd-4376-9c05-96f8a78d9a87-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afs58th\" (UID: \"536f6328-45cd-4376-9c05-96f8a78d9a87\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afs58th" Jul 23 21:57:21.781719 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:21.781683 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/536f6328-45cd-4376-9c05-96f8a78d9a87-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afs58th\" (UID: \"536f6328-45cd-4376-9c05-96f8a78d9a87\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afs58th" Jul 23 21:57:21.781870 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:21.781759 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/536f6328-45cd-4376-9c05-96f8a78d9a87-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afs58th\" (UID: \"536f6328-45cd-4376-9c05-96f8a78d9a87\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afs58th" Jul 23 21:57:21.781870 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:21.781807 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kzswd\" (UniqueName: \"kubernetes.io/projected/536f6328-45cd-4376-9c05-96f8a78d9a87-kube-api-access-kzswd\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afs58th\" (UID: \"536f6328-45cd-4376-9c05-96f8a78d9a87\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afs58th" Jul 23 21:57:21.782133 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:21.782111 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/536f6328-45cd-4376-9c05-96f8a78d9a87-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afs58th\" (UID: \"536f6328-45cd-4376-9c05-96f8a78d9a87\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afs58th" Jul 23 21:57:21.782175 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:21.782128 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/536f6328-45cd-4376-9c05-96f8a78d9a87-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afs58th\" (UID: \"536f6328-45cd-4376-9c05-96f8a78d9a87\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afs58th" Jul 23 21:57:21.790414 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:21.790388 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kzswd\" (UniqueName: \"kubernetes.io/projected/536f6328-45cd-4376-9c05-96f8a78d9a87-kube-api-access-kzswd\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afs58th\" (UID: \"536f6328-45cd-4376-9c05-96f8a78d9a87\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afs58th" Jul 23 21:57:21.900784 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:21.900704 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afs58th" Jul 23 21:57:22.029336 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:22.029313 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afs58th"] Jul 23 21:57:22.031813 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:57:22.031782 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod536f6328_45cd_4376_9c05_96f8a78d9a87.slice/crio-27a6efcdc6b37ee508eacec0d99f81610c74f599655d2c4b4bb47bb3db8d5254 WatchSource:0}: Error finding container 27a6efcdc6b37ee508eacec0d99f81610c74f599655d2c4b4bb47bb3db8d5254: Status 404 returned error can't find the container with id 27a6efcdc6b37ee508eacec0d99f81610c74f599655d2c4b4bb47bb3db8d5254 Jul 23 21:57:22.676795 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:22.676756 2578 generic.go:356] "Generic (PLEG): container finished" podID="536f6328-45cd-4376-9c05-96f8a78d9a87" containerID="7fddb65aaf40bc6e9f83962d3c41fed0a752b4d0b3b18d63166d2f4fa3bdf80d" exitCode=0 Jul 23 21:57:22.677175 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:22.676800 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afs58th" event={"ID":"536f6328-45cd-4376-9c05-96f8a78d9a87","Type":"ContainerDied","Data":"7fddb65aaf40bc6e9f83962d3c41fed0a752b4d0b3b18d63166d2f4fa3bdf80d"} Jul 23 21:57:22.677175 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:22.676845 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afs58th" event={"ID":"536f6328-45cd-4376-9c05-96f8a78d9a87","Type":"ContainerStarted","Data":"27a6efcdc6b37ee508eacec0d99f81610c74f599655d2c4b4bb47bb3db8d5254"} Jul 23 21:57:23.682316 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:23.682282 2578 generic.go:356] "Generic (PLEG): container finished" podID="536f6328-45cd-4376-9c05-96f8a78d9a87" containerID="786d76b8863b54eee9683ce24ffa4f82a4d1d9ea003df9927f71b2dc2c89402f" exitCode=0 Jul 23 21:57:23.682707 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:23.682366 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afs58th" event={"ID":"536f6328-45cd-4376-9c05-96f8a78d9a87","Type":"ContainerDied","Data":"786d76b8863b54eee9683ce24ffa4f82a4d1d9ea003df9927f71b2dc2c89402f"} Jul 23 21:57:24.688242 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:24.688206 2578 generic.go:356] "Generic (PLEG): container finished" podID="536f6328-45cd-4376-9c05-96f8a78d9a87" containerID="f2a865383ba9fe5085697b06771dc2f72b9dbc69ed24ad8a44ceb48f01b65445" exitCode=0 Jul 23 21:57:24.688635 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:24.688290 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afs58th" event={"ID":"536f6328-45cd-4376-9c05-96f8a78d9a87","Type":"ContainerDied","Data":"f2a865383ba9fe5085697b06771dc2f72b9dbc69ed24ad8a44ceb48f01b65445"} Jul 23 21:57:25.819133 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:25.819109 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afs58th" Jul 23 21:57:25.918513 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:25.918482 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/536f6328-45cd-4376-9c05-96f8a78d9a87-bundle\") pod \"536f6328-45cd-4376-9c05-96f8a78d9a87\" (UID: \"536f6328-45cd-4376-9c05-96f8a78d9a87\") " Jul 23 21:57:25.918704 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:25.918522 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kzswd\" (UniqueName: \"kubernetes.io/projected/536f6328-45cd-4376-9c05-96f8a78d9a87-kube-api-access-kzswd\") pod \"536f6328-45cd-4376-9c05-96f8a78d9a87\" (UID: \"536f6328-45cd-4376-9c05-96f8a78d9a87\") " Jul 23 21:57:25.918704 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:25.918628 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/536f6328-45cd-4376-9c05-96f8a78d9a87-util\") pod \"536f6328-45cd-4376-9c05-96f8a78d9a87\" (UID: \"536f6328-45cd-4376-9c05-96f8a78d9a87\") " Jul 23 21:57:25.919314 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:25.919288 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/536f6328-45cd-4376-9c05-96f8a78d9a87-bundle" (OuterVolumeSpecName: "bundle") pod "536f6328-45cd-4376-9c05-96f8a78d9a87" (UID: "536f6328-45cd-4376-9c05-96f8a78d9a87"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 21:57:25.920728 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:25.920706 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/536f6328-45cd-4376-9c05-96f8a78d9a87-kube-api-access-kzswd" (OuterVolumeSpecName: "kube-api-access-kzswd") pod "536f6328-45cd-4376-9c05-96f8a78d9a87" (UID: "536f6328-45cd-4376-9c05-96f8a78d9a87"). InnerVolumeSpecName "kube-api-access-kzswd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 21:57:25.926511 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:25.926480 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/536f6328-45cd-4376-9c05-96f8a78d9a87-util" (OuterVolumeSpecName: "util") pod "536f6328-45cd-4376-9c05-96f8a78d9a87" (UID: "536f6328-45cd-4376-9c05-96f8a78d9a87"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 21:57:26.019297 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:26.019211 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/536f6328-45cd-4376-9c05-96f8a78d9a87-util\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\"" Jul 23 21:57:26.019297 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:26.019264 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/536f6328-45cd-4376-9c05-96f8a78d9a87-bundle\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\"" Jul 23 21:57:26.019297 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:26.019275 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kzswd\" (UniqueName: \"kubernetes.io/projected/536f6328-45cd-4376-9c05-96f8a78d9a87-kube-api-access-kzswd\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\"" Jul 23 21:57:26.697386 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:26.697363 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afs58th" Jul 23 21:57:26.697582 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:26.697363 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afs58th" event={"ID":"536f6328-45cd-4376-9c05-96f8a78d9a87","Type":"ContainerDied","Data":"27a6efcdc6b37ee508eacec0d99f81610c74f599655d2c4b4bb47bb3db8d5254"} Jul 23 21:57:26.697582 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:26.697469 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="27a6efcdc6b37ee508eacec0d99f81610c74f599655d2c4b4bb47bb3db8d5254" Jul 23 21:57:29.084841 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:29.084812 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-7575c4d6b8-2z7qz"] Jul 23 21:57:29.085196 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:29.085080 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="536f6328-45cd-4376-9c05-96f8a78d9a87" containerName="pull" Jul 23 21:57:29.085196 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:29.085091 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="536f6328-45cd-4376-9c05-96f8a78d9a87" containerName="pull" Jul 23 21:57:29.085196 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:29.085101 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="536f6328-45cd-4376-9c05-96f8a78d9a87" containerName="util" Jul 23 21:57:29.085196 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:29.085106 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="536f6328-45cd-4376-9c05-96f8a78d9a87" containerName="util" Jul 23 21:57:29.085196 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:29.085111 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="536f6328-45cd-4376-9c05-96f8a78d9a87" containerName="extract" Jul 23 21:57:29.085196 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:29.085116 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="536f6328-45cd-4376-9c05-96f8a78d9a87" containerName="extract" Jul 23 21:57:29.085196 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:29.085165 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="536f6328-45cd-4376-9c05-96f8a78d9a87" containerName="extract" Jul 23 21:57:29.086971 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:29.086950 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7575c4d6b8-2z7qz" Jul 23 21:57:29.091576 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:29.090393 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 23 21:57:29.091576 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:29.090983 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-68fvx\"" Jul 23 21:57:29.091576 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:29.091018 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 23 21:57:29.091576 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:29.091440 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 23 21:57:29.091576 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:29.091511 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 23 21:57:29.091914 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:29.091744 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 23 21:57:29.098156 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:29.098132 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7575c4d6b8-2z7qz"] Jul 23 21:57:29.245239 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:29.245164 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/56562de7-fa5d-4b31-974d-1bb76ab0927f-metrics-cert\") pod \"lws-controller-manager-7575c4d6b8-2z7qz\" (UID: \"56562de7-fa5d-4b31-974d-1bb76ab0927f\") " pod="openshift-lws-operator/lws-controller-manager-7575c4d6b8-2z7qz" Jul 23 21:57:29.245239 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:29.245207 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d7kmt\" (UniqueName: \"kubernetes.io/projected/56562de7-fa5d-4b31-974d-1bb76ab0927f-kube-api-access-d7kmt\") pod \"lws-controller-manager-7575c4d6b8-2z7qz\" (UID: \"56562de7-fa5d-4b31-974d-1bb76ab0927f\") " pod="openshift-lws-operator/lws-controller-manager-7575c4d6b8-2z7qz" Jul 23 21:57:29.245471 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:29.245347 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/56562de7-fa5d-4b31-974d-1bb76ab0927f-manager-config\") pod \"lws-controller-manager-7575c4d6b8-2z7qz\" (UID: \"56562de7-fa5d-4b31-974d-1bb76ab0927f\") " pod="openshift-lws-operator/lws-controller-manager-7575c4d6b8-2z7qz" Jul 23 21:57:29.245471 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:29.245396 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/56562de7-fa5d-4b31-974d-1bb76ab0927f-cert\") pod \"lws-controller-manager-7575c4d6b8-2z7qz\" (UID: \"56562de7-fa5d-4b31-974d-1bb76ab0927f\") " pod="openshift-lws-operator/lws-controller-manager-7575c4d6b8-2z7qz" Jul 23 21:57:29.346110 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:29.346025 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d7kmt\" (UniqueName: \"kubernetes.io/projected/56562de7-fa5d-4b31-974d-1bb76ab0927f-kube-api-access-d7kmt\") pod \"lws-controller-manager-7575c4d6b8-2z7qz\" (UID: \"56562de7-fa5d-4b31-974d-1bb76ab0927f\") " pod="openshift-lws-operator/lws-controller-manager-7575c4d6b8-2z7qz" Jul 23 21:57:29.346110 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:29.346092 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/56562de7-fa5d-4b31-974d-1bb76ab0927f-manager-config\") pod \"lws-controller-manager-7575c4d6b8-2z7qz\" (UID: \"56562de7-fa5d-4b31-974d-1bb76ab0927f\") " pod="openshift-lws-operator/lws-controller-manager-7575c4d6b8-2z7qz" Jul 23 21:57:29.346320 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:29.346117 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/56562de7-fa5d-4b31-974d-1bb76ab0927f-cert\") pod \"lws-controller-manager-7575c4d6b8-2z7qz\" (UID: \"56562de7-fa5d-4b31-974d-1bb76ab0927f\") " pod="openshift-lws-operator/lws-controller-manager-7575c4d6b8-2z7qz" Jul 23 21:57:29.346320 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:29.346138 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/56562de7-fa5d-4b31-974d-1bb76ab0927f-metrics-cert\") pod \"lws-controller-manager-7575c4d6b8-2z7qz\" (UID: \"56562de7-fa5d-4b31-974d-1bb76ab0927f\") " pod="openshift-lws-operator/lws-controller-manager-7575c4d6b8-2z7qz" Jul 23 21:57:29.346806 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:29.346780 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/56562de7-fa5d-4b31-974d-1bb76ab0927f-manager-config\") pod \"lws-controller-manager-7575c4d6b8-2z7qz\" (UID: \"56562de7-fa5d-4b31-974d-1bb76ab0927f\") " pod="openshift-lws-operator/lws-controller-manager-7575c4d6b8-2z7qz" Jul 23 21:57:29.348731 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:29.348708 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/56562de7-fa5d-4b31-974d-1bb76ab0927f-metrics-cert\") pod \"lws-controller-manager-7575c4d6b8-2z7qz\" (UID: \"56562de7-fa5d-4b31-974d-1bb76ab0927f\") " pod="openshift-lws-operator/lws-controller-manager-7575c4d6b8-2z7qz" Jul 23 21:57:29.348813 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:29.348738 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/56562de7-fa5d-4b31-974d-1bb76ab0927f-cert\") pod \"lws-controller-manager-7575c4d6b8-2z7qz\" (UID: \"56562de7-fa5d-4b31-974d-1bb76ab0927f\") " pod="openshift-lws-operator/lws-controller-manager-7575c4d6b8-2z7qz" Jul 23 21:57:29.359370 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:29.359348 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d7kmt\" (UniqueName: \"kubernetes.io/projected/56562de7-fa5d-4b31-974d-1bb76ab0927f-kube-api-access-d7kmt\") pod \"lws-controller-manager-7575c4d6b8-2z7qz\" (UID: \"56562de7-fa5d-4b31-974d-1bb76ab0927f\") " pod="openshift-lws-operator/lws-controller-manager-7575c4d6b8-2z7qz" Jul 23 21:57:29.398977 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:29.398948 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7575c4d6b8-2z7qz" Jul 23 21:57:29.528472 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:29.528445 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7575c4d6b8-2z7qz"] Jul 23 21:57:29.530894 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:57:29.530869 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56562de7_fa5d_4b31_974d_1bb76ab0927f.slice/crio-71497b9c579fd63cb3d92e932a9658573090796fbe5032454afb1a55eeac47ff WatchSource:0}: Error finding container 71497b9c579fd63cb3d92e932a9658573090796fbe5032454afb1a55eeac47ff: Status 404 returned error can't find the container with id 71497b9c579fd63cb3d92e932a9658573090796fbe5032454afb1a55eeac47ff Jul 23 21:57:29.708944 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:29.708849 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7575c4d6b8-2z7qz" event={"ID":"56562de7-fa5d-4b31-974d-1bb76ab0927f","Type":"ContainerStarted","Data":"71497b9c579fd63cb3d92e932a9658573090796fbe5032454afb1a55eeac47ff"} Jul 23 21:57:33.889518 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:33.889485 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6864c64679-mgzwp"] Jul 23 21:57:33.892247 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:33.892232 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6864c64679-mgzwp" Jul 23 21:57:33.895068 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:33.895047 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 23 21:57:33.896451 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:33.896437 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 23 21:57:33.896921 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:33.896908 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 23 21:57:33.897250 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:33.897217 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 23 21:57:33.897564 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:33.897528 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-qjdhj\"" Jul 23 21:57:33.935996 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:33.935964 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6864c64679-mgzwp"] Jul 23 21:57:33.982581 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:33.982525 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/1b35faf5-1558-4b78-8af3-9b8ab44205a1-manifests\") pod \"opendatahub-operator-controller-manager-6864c64679-mgzwp\" (UID: \"1b35faf5-1558-4b78-8af3-9b8ab44205a1\") " pod="opendatahub/opendatahub-operator-controller-manager-6864c64679-mgzwp" Jul 23 21:57:33.982722 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:33.982586 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/1b35faf5-1558-4b78-8af3-9b8ab44205a1-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6864c64679-mgzwp\" (UID: \"1b35faf5-1558-4b78-8af3-9b8ab44205a1\") " pod="opendatahub/opendatahub-operator-controller-manager-6864c64679-mgzwp" Jul 23 21:57:33.982722 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:33.982634 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1b35faf5-1558-4b78-8af3-9b8ab44205a1-webhook-cert\") pod \"opendatahub-operator-controller-manager-6864c64679-mgzwp\" (UID: \"1b35faf5-1558-4b78-8af3-9b8ab44205a1\") " pod="opendatahub/opendatahub-operator-controller-manager-6864c64679-mgzwp" Jul 23 21:57:33.982722 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:33.982657 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rw5kg\" (UniqueName: \"kubernetes.io/projected/1b35faf5-1558-4b78-8af3-9b8ab44205a1-kube-api-access-rw5kg\") pod \"opendatahub-operator-controller-manager-6864c64679-mgzwp\" (UID: \"1b35faf5-1558-4b78-8af3-9b8ab44205a1\") " pod="opendatahub/opendatahub-operator-controller-manager-6864c64679-mgzwp" Jul 23 21:57:33.982722 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:33.982720 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1b35faf5-1558-4b78-8af3-9b8ab44205a1-tmp\") pod \"opendatahub-operator-controller-manager-6864c64679-mgzwp\" (UID: \"1b35faf5-1558-4b78-8af3-9b8ab44205a1\") " pod="opendatahub/opendatahub-operator-controller-manager-6864c64679-mgzwp" Jul 23 21:57:34.083095 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:34.083055 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1b35faf5-1558-4b78-8af3-9b8ab44205a1-webhook-cert\") pod \"opendatahub-operator-controller-manager-6864c64679-mgzwp\" (UID: \"1b35faf5-1558-4b78-8af3-9b8ab44205a1\") " pod="opendatahub/opendatahub-operator-controller-manager-6864c64679-mgzwp" Jul 23 21:57:34.083254 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:34.083101 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rw5kg\" (UniqueName: \"kubernetes.io/projected/1b35faf5-1558-4b78-8af3-9b8ab44205a1-kube-api-access-rw5kg\") pod \"opendatahub-operator-controller-manager-6864c64679-mgzwp\" (UID: \"1b35faf5-1558-4b78-8af3-9b8ab44205a1\") " pod="opendatahub/opendatahub-operator-controller-manager-6864c64679-mgzwp" Jul 23 21:57:34.083296 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:34.083269 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1b35faf5-1558-4b78-8af3-9b8ab44205a1-tmp\") pod \"opendatahub-operator-controller-manager-6864c64679-mgzwp\" (UID: \"1b35faf5-1558-4b78-8af3-9b8ab44205a1\") " pod="opendatahub/opendatahub-operator-controller-manager-6864c64679-mgzwp" Jul 23 21:57:34.083344 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:34.083329 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/1b35faf5-1558-4b78-8af3-9b8ab44205a1-manifests\") pod \"opendatahub-operator-controller-manager-6864c64679-mgzwp\" (UID: \"1b35faf5-1558-4b78-8af3-9b8ab44205a1\") " pod="opendatahub/opendatahub-operator-controller-manager-6864c64679-mgzwp" Jul 23 21:57:34.083383 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:34.083364 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/1b35faf5-1558-4b78-8af3-9b8ab44205a1-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6864c64679-mgzwp\" (UID: \"1b35faf5-1558-4b78-8af3-9b8ab44205a1\") " pod="opendatahub/opendatahub-operator-controller-manager-6864c64679-mgzwp" Jul 23 21:57:34.083783 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:34.083736 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/1b35faf5-1558-4b78-8af3-9b8ab44205a1-manifests\") pod \"opendatahub-operator-controller-manager-6864c64679-mgzwp\" (UID: \"1b35faf5-1558-4b78-8af3-9b8ab44205a1\") " pod="opendatahub/opendatahub-operator-controller-manager-6864c64679-mgzwp" Jul 23 21:57:34.085530 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:34.085511 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1b35faf5-1558-4b78-8af3-9b8ab44205a1-tmp\") pod \"opendatahub-operator-controller-manager-6864c64679-mgzwp\" (UID: \"1b35faf5-1558-4b78-8af3-9b8ab44205a1\") " pod="opendatahub/opendatahub-operator-controller-manager-6864c64679-mgzwp" Jul 23 21:57:34.085627 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:34.085558 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1b35faf5-1558-4b78-8af3-9b8ab44205a1-webhook-cert\") pod \"opendatahub-operator-controller-manager-6864c64679-mgzwp\" (UID: \"1b35faf5-1558-4b78-8af3-9b8ab44205a1\") " pod="opendatahub/opendatahub-operator-controller-manager-6864c64679-mgzwp" Jul 23 21:57:34.085794 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:34.085778 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/1b35faf5-1558-4b78-8af3-9b8ab44205a1-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6864c64679-mgzwp\" (UID: \"1b35faf5-1558-4b78-8af3-9b8ab44205a1\") " pod="opendatahub/opendatahub-operator-controller-manager-6864c64679-mgzwp" Jul 23 21:57:34.091046 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:34.091027 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rw5kg\" (UniqueName: \"kubernetes.io/projected/1b35faf5-1558-4b78-8af3-9b8ab44205a1-kube-api-access-rw5kg\") pod \"opendatahub-operator-controller-manager-6864c64679-mgzwp\" (UID: \"1b35faf5-1558-4b78-8af3-9b8ab44205a1\") " pod="opendatahub/opendatahub-operator-controller-manager-6864c64679-mgzwp" Jul 23 21:57:34.202771 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:34.202691 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6864c64679-mgzwp" Jul 23 21:57:34.340629 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:34.340478 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6864c64679-mgzwp"] Jul 23 21:57:34.342828 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:57:34.342799 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b35faf5_1558_4b78_8af3_9b8ab44205a1.slice/crio-60ca769b55fa6abb7e422520710de3d1808db09e7dccfd9c3c767da678ead94d WatchSource:0}: Error finding container 60ca769b55fa6abb7e422520710de3d1808db09e7dccfd9c3c767da678ead94d: Status 404 returned error can't find the container with id 60ca769b55fa6abb7e422520710de3d1808db09e7dccfd9c3c767da678ead94d Jul 23 21:57:34.728718 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:34.728682 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6864c64679-mgzwp" event={"ID":"1b35faf5-1558-4b78-8af3-9b8ab44205a1","Type":"ContainerStarted","Data":"60ca769b55fa6abb7e422520710de3d1808db09e7dccfd9c3c767da678ead94d"} Jul 23 21:57:37.747307 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:37.747266 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6864c64679-mgzwp" event={"ID":"1b35faf5-1558-4b78-8af3-9b8ab44205a1","Type":"ContainerStarted","Data":"633ad94bd9efc7d19ba3649548d1d2ec724043e655bc3d267034d3b5ffd9edbf"} Jul 23 21:57:38.753324 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:38.753229 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7575c4d6b8-2z7qz" event={"ID":"56562de7-fa5d-4b31-974d-1bb76ab0927f","Type":"ContainerStarted","Data":"db27d6eb0349b261afcff41d2bd3ec52b40291f4cf5fc63f65a1c3c19dab3e44"} Jul 23 21:57:38.753324 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:38.753302 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-7575c4d6b8-2z7qz" Jul 23 21:57:38.754861 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:38.754832 2578 generic.go:356] "Generic (PLEG): container finished" podID="1b35faf5-1558-4b78-8af3-9b8ab44205a1" containerID="633ad94bd9efc7d19ba3649548d1d2ec724043e655bc3d267034d3b5ffd9edbf" exitCode=0 Jul 23 21:57:38.754981 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:38.754900 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6864c64679-mgzwp" event={"ID":"1b35faf5-1558-4b78-8af3-9b8ab44205a1","Type":"ContainerDied","Data":"633ad94bd9efc7d19ba3649548d1d2ec724043e655bc3d267034d3b5ffd9edbf"} Jul 23 21:57:38.772035 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:38.771988 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-7575c4d6b8-2z7qz" podStartSLOduration=1.402131698 podStartE2EDuration="9.771972351s" totalImagesPullingTime="8.369840653s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:57:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:57:38.770740237 +0000 UTC m=+361.012648110" watchObservedRunningTime="2026-07-23 21:57:38.771972351 +0000 UTC m=+361.013880225" Jul 23 21:57:39.761359 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:39.761309 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6864c64679-mgzwp" event={"ID":"1b35faf5-1558-4b78-8af3-9b8ab44205a1","Type":"ContainerStarted","Data":"beedde544f100d18b3c9669ffdbd427bc183c3a274bf01980ad626881c8ac770"} Jul 23 21:57:39.783204 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:39.783157 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6864c64679-mgzwp" podStartSLOduration=2.593439318 podStartE2EDuration="6.783143814s" totalImagesPullingTime="3.028435605s" totalInitContainerRuntime="1.161268891s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:57:33 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:57:39.782180026 +0000 UTC m=+362.024087900" watchObservedRunningTime="2026-07-23 21:57:39.783143814 +0000 UTC m=+362.025051688" Jul 23 21:57:40.765718 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:40.765689 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6864c64679-mgzwp" Jul 23 21:57:49.763334 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:49.763304 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-7575c4d6b8-2z7qz" Jul 23 21:57:51.772688 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:57:51.772655 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6864c64679-mgzwp" Jul 23 21:58:07.478022 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:58:07.477983 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4svqcn"] Jul 23 21:58:07.481366 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:58:07.481343 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4svqcn" Jul 23 21:58:07.495695 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:58:07.495678 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-4qr62\"" Jul 23 21:58:07.496577 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:58:07.496526 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 23 21:58:07.496830 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:58:07.496816 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 23 21:58:07.497741 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:58:07.497723 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 23 21:58:07.520914 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:58:07.520889 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4svqcn"] Jul 23 21:58:07.555518 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:58:07.555490 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/91e17fed-a12b-4a8f-9c0e-6f84e3776d1f-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4svqcn\" (UID: \"91e17fed-a12b-4a8f-9c0e-6f84e3776d1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4svqcn" Jul 23 21:58:07.555674 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:58:07.555525 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fp7hv\" (UniqueName: \"kubernetes.io/projected/91e17fed-a12b-4a8f-9c0e-6f84e3776d1f-kube-api-access-fp7hv\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4svqcn\" (UID: \"91e17fed-a12b-4a8f-9c0e-6f84e3776d1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4svqcn" Jul 23 21:58:07.555674 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:58:07.555572 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/91e17fed-a12b-4a8f-9c0e-6f84e3776d1f-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4svqcn\" (UID: \"91e17fed-a12b-4a8f-9c0e-6f84e3776d1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4svqcn" Jul 23 21:58:07.555674 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:58:07.555663 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/91e17fed-a12b-4a8f-9c0e-6f84e3776d1f-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4svqcn\" (UID: \"91e17fed-a12b-4a8f-9c0e-6f84e3776d1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4svqcn" Jul 23 21:58:07.555815 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:58:07.555701 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/91e17fed-a12b-4a8f-9c0e-6f84e3776d1f-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4svqcn\" (UID: \"91e17fed-a12b-4a8f-9c0e-6f84e3776d1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4svqcn" Jul 23 21:58:07.555815 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:58:07.555754 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/91e17fed-a12b-4a8f-9c0e-6f84e3776d1f-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4svqcn\" (UID: \"91e17fed-a12b-4a8f-9c0e-6f84e3776d1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4svqcn" Jul 23 21:58:07.555815 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:58:07.555796 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/91e17fed-a12b-4a8f-9c0e-6f84e3776d1f-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4svqcn\" (UID: \"91e17fed-a12b-4a8f-9c0e-6f84e3776d1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4svqcn" Jul 23 21:58:07.555931 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:58:07.555839 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/91e17fed-a12b-4a8f-9c0e-6f84e3776d1f-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4svqcn\" (UID: \"91e17fed-a12b-4a8f-9c0e-6f84e3776d1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4svqcn" Jul 23 21:58:07.555931 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:58:07.555867 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/91e17fed-a12b-4a8f-9c0e-6f84e3776d1f-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4svqcn\" (UID: \"91e17fed-a12b-4a8f-9c0e-6f84e3776d1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4svqcn" Jul 23 21:58:07.656644 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:58:07.656557 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/91e17fed-a12b-4a8f-9c0e-6f84e3776d1f-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4svqcn\" (UID: \"91e17fed-a12b-4a8f-9c0e-6f84e3776d1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4svqcn" Jul 23 21:58:07.656819 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:58:07.656661 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/91e17fed-a12b-4a8f-9c0e-6f84e3776d1f-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4svqcn\" (UID: \"91e17fed-a12b-4a8f-9c0e-6f84e3776d1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4svqcn" Jul 23 21:58:07.656819 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:58:07.656695 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/91e17fed-a12b-4a8f-9c0e-6f84e3776d1f-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4svqcn\" (UID: \"91e17fed-a12b-4a8f-9c0e-6f84e3776d1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4svqcn" Jul 23 21:58:07.656819 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:58:07.656722 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/91e17fed-a12b-4a8f-9c0e-6f84e3776d1f-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4svqcn\" (UID: \"91e17fed-a12b-4a8f-9c0e-6f84e3776d1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4svqcn" Jul 23 21:58:07.656819 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:58:07.656758 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/91e17fed-a12b-4a8f-9c0e-6f84e3776d1f-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4svqcn\" (UID: \"91e17fed-a12b-4a8f-9c0e-6f84e3776d1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4svqcn" Jul 23 21:58:07.656819 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:58:07.656783 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/91e17fed-a12b-4a8f-9c0e-6f84e3776d1f-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4svqcn\" (UID: \"91e17fed-a12b-4a8f-9c0e-6f84e3776d1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4svqcn" Jul 23 21:58:07.657068 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:58:07.656864 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/91e17fed-a12b-4a8f-9c0e-6f84e3776d1f-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4svqcn\" (UID: \"91e17fed-a12b-4a8f-9c0e-6f84e3776d1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4svqcn" Jul 23 21:58:07.657068 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:58:07.656897 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fp7hv\" (UniqueName: \"kubernetes.io/projected/91e17fed-a12b-4a8f-9c0e-6f84e3776d1f-kube-api-access-fp7hv\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4svqcn\" (UID: \"91e17fed-a12b-4a8f-9c0e-6f84e3776d1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4svqcn" Jul 23 21:58:07.657068 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:58:07.656928 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/91e17fed-a12b-4a8f-9c0e-6f84e3776d1f-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4svqcn\" (UID: \"91e17fed-a12b-4a8f-9c0e-6f84e3776d1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4svqcn" Jul 23 21:58:07.657313 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:58:07.657284 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/91e17fed-a12b-4a8f-9c0e-6f84e3776d1f-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4svqcn\" (UID: \"91e17fed-a12b-4a8f-9c0e-6f84e3776d1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4svqcn" Jul 23 21:58:07.657392 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:58:07.657300 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/91e17fed-a12b-4a8f-9c0e-6f84e3776d1f-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4svqcn\" (UID: \"91e17fed-a12b-4a8f-9c0e-6f84e3776d1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4svqcn" Jul 23 21:58:07.657392 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:58:07.657373 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/91e17fed-a12b-4a8f-9c0e-6f84e3776d1f-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4svqcn\" (UID: \"91e17fed-a12b-4a8f-9c0e-6f84e3776d1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4svqcn" Jul 23 21:58:07.657597 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:58:07.657573 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/91e17fed-a12b-4a8f-9c0e-6f84e3776d1f-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4svqcn\" (UID: \"91e17fed-a12b-4a8f-9c0e-6f84e3776d1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4svqcn" Jul 23 21:58:07.657836 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:58:07.657816 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/91e17fed-a12b-4a8f-9c0e-6f84e3776d1f-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4svqcn\" (UID: \"91e17fed-a12b-4a8f-9c0e-6f84e3776d1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4svqcn" Jul 23 21:58:07.659252 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:58:07.659233 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/91e17fed-a12b-4a8f-9c0e-6f84e3776d1f-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4svqcn\" (UID: \"91e17fed-a12b-4a8f-9c0e-6f84e3776d1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4svqcn" Jul 23 21:58:07.659390 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:58:07.659370 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/91e17fed-a12b-4a8f-9c0e-6f84e3776d1f-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4svqcn\" (UID: \"91e17fed-a12b-4a8f-9c0e-6f84e3776d1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4svqcn" Jul 23 21:58:07.666061 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:58:07.666044 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fp7hv\" (UniqueName: \"kubernetes.io/projected/91e17fed-a12b-4a8f-9c0e-6f84e3776d1f-kube-api-access-fp7hv\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4svqcn\" (UID: \"91e17fed-a12b-4a8f-9c0e-6f84e3776d1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4svqcn" Jul 23 21:58:07.669994 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:58:07.669967 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/91e17fed-a12b-4a8f-9c0e-6f84e3776d1f-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4svqcn\" (UID: \"91e17fed-a12b-4a8f-9c0e-6f84e3776d1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4svqcn" Jul 23 21:58:07.791471 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:58:07.791397 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4svqcn" Jul 23 21:58:07.938121 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:58:07.938098 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4svqcn"] Jul 23 21:58:07.940633 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:58:07.940598 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91e17fed_a12b_4a8f_9c0e_6f84e3776d1f.slice/crio-2cd39b6210ab15075c41f228d453f3ed4ddb33ba1c921b3948d88d339a44a530 WatchSource:0}: Error finding container 2cd39b6210ab15075c41f228d453f3ed4ddb33ba1c921b3948d88d339a44a530: Status 404 returned error can't find the container with id 2cd39b6210ab15075c41f228d453f3ed4ddb33ba1c921b3948d88d339a44a530 Jul 23 21:58:08.057085 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:58:08.057052 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9pzmh"] Jul 23 21:58:08.061671 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:58:08.061655 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9pzmh" Jul 23 21:58:08.067254 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:58:08.067233 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 23 21:58:08.067386 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:58:08.067235 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 23 21:58:08.067386 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:58:08.067233 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 23 21:58:08.069158 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:58:08.069140 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-jth9l\"" Jul 23 21:58:08.075319 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:58:08.075299 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9pzmh"] Jul 23 21:58:08.161838 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:58:08.161795 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/a7683500-183f-4e5c-bd8d-756d50a6f5e0-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-9pzmh\" (UID: \"a7683500-183f-4e5c-bd8d-756d50a6f5e0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9pzmh" Jul 23 21:58:08.162056 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:58:08.161847 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/a7683500-183f-4e5c-bd8d-756d50a6f5e0-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-9pzmh\" (UID: \"a7683500-183f-4e5c-bd8d-756d50a6f5e0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9pzmh" Jul 23 21:58:08.162056 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:58:08.161876 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z69sq\" (UniqueName: \"kubernetes.io/projected/a7683500-183f-4e5c-bd8d-756d50a6f5e0-kube-api-access-z69sq\") pod \"istiod-openshift-gateway-94bb8fbfd-9pzmh\" (UID: \"a7683500-183f-4e5c-bd8d-756d50a6f5e0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9pzmh" Jul 23 21:58:08.162056 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:58:08.161900 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/a7683500-183f-4e5c-bd8d-756d50a6f5e0-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-9pzmh\" (UID: \"a7683500-183f-4e5c-bd8d-756d50a6f5e0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9pzmh" Jul 23 21:58:08.162056 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:58:08.161917 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/a7683500-183f-4e5c-bd8d-756d50a6f5e0-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-9pzmh\" (UID: \"a7683500-183f-4e5c-bd8d-756d50a6f5e0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9pzmh" Jul 23 21:58:08.162056 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:58:08.161936 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a7683500-183f-4e5c-bd8d-756d50a6f5e0-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-9pzmh\" (UID: \"a7683500-183f-4e5c-bd8d-756d50a6f5e0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9pzmh" Jul 23 21:58:08.162056 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:58:08.161985 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a7683500-183f-4e5c-bd8d-756d50a6f5e0-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-9pzmh\" (UID: \"a7683500-183f-4e5c-bd8d-756d50a6f5e0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9pzmh" Jul 23 21:58:08.262375 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:58:08.262336 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a7683500-183f-4e5c-bd8d-756d50a6f5e0-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-9pzmh\" (UID: \"a7683500-183f-4e5c-bd8d-756d50a6f5e0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9pzmh" Jul 23 21:58:08.262589 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:58:08.262391 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/a7683500-183f-4e5c-bd8d-756d50a6f5e0-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-9pzmh\" (UID: \"a7683500-183f-4e5c-bd8d-756d50a6f5e0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9pzmh" Jul 23 21:58:08.262589 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:58:08.262437 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/a7683500-183f-4e5c-bd8d-756d50a6f5e0-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-9pzmh\" (UID: \"a7683500-183f-4e5c-bd8d-756d50a6f5e0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9pzmh" Jul 23 21:58:08.262589 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:58:08.262464 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z69sq\" (UniqueName: \"kubernetes.io/projected/a7683500-183f-4e5c-bd8d-756d50a6f5e0-kube-api-access-z69sq\") pod \"istiod-openshift-gateway-94bb8fbfd-9pzmh\" (UID: \"a7683500-183f-4e5c-bd8d-756d50a6f5e0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9pzmh" Jul 23 21:58:08.262589 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:58:08.262490 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/a7683500-183f-4e5c-bd8d-756d50a6f5e0-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-9pzmh\" (UID: \"a7683500-183f-4e5c-bd8d-756d50a6f5e0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9pzmh" Jul 23 21:58:08.262589 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:58:08.262513 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/a7683500-183f-4e5c-bd8d-756d50a6f5e0-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-9pzmh\" (UID: \"a7683500-183f-4e5c-bd8d-756d50a6f5e0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9pzmh" Jul 23 21:58:08.262589 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:58:08.262556 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a7683500-183f-4e5c-bd8d-756d50a6f5e0-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-9pzmh\" (UID: \"a7683500-183f-4e5c-bd8d-756d50a6f5e0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9pzmh" Jul 23 21:58:08.263785 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:58:08.263746 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/a7683500-183f-4e5c-bd8d-756d50a6f5e0-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-9pzmh\" (UID: \"a7683500-183f-4e5c-bd8d-756d50a6f5e0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9pzmh" Jul 23 21:58:08.265290 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:58:08.265269 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/a7683500-183f-4e5c-bd8d-756d50a6f5e0-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-9pzmh\" (UID: \"a7683500-183f-4e5c-bd8d-756d50a6f5e0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9pzmh" Jul 23 21:58:08.265388 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:58:08.265299 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/a7683500-183f-4e5c-bd8d-756d50a6f5e0-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-9pzmh\" (UID: \"a7683500-183f-4e5c-bd8d-756d50a6f5e0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9pzmh" Jul 23 21:58:08.265388 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:58:08.265325 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/a7683500-183f-4e5c-bd8d-756d50a6f5e0-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-9pzmh\" (UID: \"a7683500-183f-4e5c-bd8d-756d50a6f5e0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9pzmh" Jul 23 21:58:08.265467 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:58:08.265433 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a7683500-183f-4e5c-bd8d-756d50a6f5e0-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-9pzmh\" (UID: \"a7683500-183f-4e5c-bd8d-756d50a6f5e0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9pzmh" Jul 23 21:58:08.279279 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:58:08.279245 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z69sq\" (UniqueName: \"kubernetes.io/projected/a7683500-183f-4e5c-bd8d-756d50a6f5e0-kube-api-access-z69sq\") pod \"istiod-openshift-gateway-94bb8fbfd-9pzmh\" (UID: \"a7683500-183f-4e5c-bd8d-756d50a6f5e0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9pzmh" Jul 23 21:58:08.279843 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:58:08.279820 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a7683500-183f-4e5c-bd8d-756d50a6f5e0-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-9pzmh\" (UID: \"a7683500-183f-4e5c-bd8d-756d50a6f5e0\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9pzmh" Jul 23 21:58:08.371217 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:58:08.371119 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9pzmh" Jul 23 21:58:08.544861 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:58:08.544709 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9pzmh"] Jul 23 21:58:08.547213 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:58:08.547188 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7683500_183f_4e5c_bd8d_756d50a6f5e0.slice/crio-141a29a27fc364db9eae09d1862b7bcfa62c0e9eea1d619ee7d16c5328040434 WatchSource:0}: Error finding container 141a29a27fc364db9eae09d1862b7bcfa62c0e9eea1d619ee7d16c5328040434: Status 404 returned error can't find the container with id 141a29a27fc364db9eae09d1862b7bcfa62c0e9eea1d619ee7d16c5328040434 Jul 23 21:58:08.887232 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:58:08.887186 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4svqcn" event={"ID":"91e17fed-a12b-4a8f-9c0e-6f84e3776d1f","Type":"ContainerStarted","Data":"2cd39b6210ab15075c41f228d453f3ed4ddb33ba1c921b3948d88d339a44a530"} Jul 23 21:58:08.888694 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:58:08.888652 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9pzmh" event={"ID":"a7683500-183f-4e5c-bd8d-756d50a6f5e0","Type":"ContainerStarted","Data":"141a29a27fc364db9eae09d1862b7bcfa62c0e9eea1d619ee7d16c5328040434"} Jul 23 21:58:11.591370 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:58:11.591332 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 23 21:58:11.591644 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:58:11.591412 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 23 21:58:11.591644 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:58:11.591442 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 23 21:58:11.592628 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:58:11.592606 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 23 21:58:11.592699 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:58:11.592653 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 23 21:58:11.908930 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:58:11.908822 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4svqcn" event={"ID":"91e17fed-a12b-4a8f-9c0e-6f84e3776d1f","Type":"ContainerStarted","Data":"2fb5679778dd6af6a3ccfd80d1a439cbf113249201f44d46a6f0433a5efedf71"} Jul 23 21:58:11.910465 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:58:11.910437 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9pzmh" event={"ID":"a7683500-183f-4e5c-bd8d-756d50a6f5e0","Type":"ContainerStarted","Data":"635bdf7535bac15c1d9a3b5065297907e2b2dd95bd6028178ee54b937cbb5756"} Jul 23 21:58:11.910608 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:58:11.910489 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9pzmh" Jul 23 21:58:11.931837 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:58:11.931787 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4svqcn" podStartSLOduration=1.28329469 podStartE2EDuration="4.931772094s" totalImagesPullingTime="3.648477404s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:58:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:58:11.929715669 +0000 UTC m=+394.171623543" watchObservedRunningTime="2026-07-23 21:58:11.931772094 +0000 UTC m=+394.173679967" Jul 23 21:58:11.953726 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:58:11.953669 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9pzmh" podStartSLOduration=0.910488129 podStartE2EDuration="3.953650385s" totalImagesPullingTime="3.043162256s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 21:58:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 21:58:11.951350165 +0000 UTC m=+394.193258038" watchObservedRunningTime="2026-07-23 21:58:11.953650385 +0000 UTC m=+394.195558260" Jul 23 21:58:12.791898 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:58:12.791859 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4svqcn" Jul 23 21:58:12.796937 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:58:12.796912 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4svqcn" Jul 23 21:58:12.915510 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:58:12.915476 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4svqcn" Jul 23 21:58:12.916516 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:58:12.916497 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4svqcn" Jul 23 21:58:12.917276 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:58:12.917259 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-9pzmh" Jul 23 21:59:10.387965 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:10.387929 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732886f"] Jul 23 21:59:10.390313 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:10.390298 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732886f" Jul 23 21:59:10.392629 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:10.392604 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 23 21:59:10.392754 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:10.392668 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-zk25n\"" Jul 23 21:59:10.393459 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:10.393445 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 23 21:59:10.403885 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:10.403865 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732886f"] Jul 23 21:59:10.501683 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:10.501645 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bzv66\" (UniqueName: \"kubernetes.io/projected/2c72f0f0-99ff-4f6e-a4bc-8e630d97f177-kube-api-access-bzv66\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732886f\" (UID: \"2c72f0f0-99ff-4f6e-a4bc-8e630d97f177\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732886f" Jul 23 21:59:10.501853 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:10.501690 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2c72f0f0-99ff-4f6e-a4bc-8e630d97f177-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732886f\" (UID: \"2c72f0f0-99ff-4f6e-a4bc-8e630d97f177\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732886f" Jul 23 21:59:10.501853 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:10.501738 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2c72f0f0-99ff-4f6e-a4bc-8e630d97f177-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732886f\" (UID: \"2c72f0f0-99ff-4f6e-a4bc-8e630d97f177\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732886f" Jul 23 21:59:10.602965 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:10.602932 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bzv66\" (UniqueName: \"kubernetes.io/projected/2c72f0f0-99ff-4f6e-a4bc-8e630d97f177-kube-api-access-bzv66\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732886f\" (UID: \"2c72f0f0-99ff-4f6e-a4bc-8e630d97f177\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732886f" Jul 23 21:59:10.603129 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:10.602987 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2c72f0f0-99ff-4f6e-a4bc-8e630d97f177-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732886f\" (UID: \"2c72f0f0-99ff-4f6e-a4bc-8e630d97f177\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732886f" Jul 23 21:59:10.603129 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:10.603031 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2c72f0f0-99ff-4f6e-a4bc-8e630d97f177-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732886f\" (UID: \"2c72f0f0-99ff-4f6e-a4bc-8e630d97f177\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732886f" Jul 23 21:59:10.603386 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:10.603367 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2c72f0f0-99ff-4f6e-a4bc-8e630d97f177-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732886f\" (UID: \"2c72f0f0-99ff-4f6e-a4bc-8e630d97f177\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732886f" Jul 23 21:59:10.603422 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:10.603385 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2c72f0f0-99ff-4f6e-a4bc-8e630d97f177-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732886f\" (UID: \"2c72f0f0-99ff-4f6e-a4bc-8e630d97f177\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732886f" Jul 23 21:59:10.612005 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:10.611979 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bzv66\" (UniqueName: \"kubernetes.io/projected/2c72f0f0-99ff-4f6e-a4bc-8e630d97f177-kube-api-access-bzv66\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732886f\" (UID: \"2c72f0f0-99ff-4f6e-a4bc-8e630d97f177\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732886f" Jul 23 21:59:10.699079 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:10.698986 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732886f" Jul 23 21:59:10.825399 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:10.825376 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732886f"] Jul 23 21:59:10.827492 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:59:10.827460 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c72f0f0_99ff_4f6e_a4bc_8e630d97f177.slice/crio-80db03393b7675e82879e211b1098ad64e33d435f062c37612a018d4f9452200 WatchSource:0}: Error finding container 80db03393b7675e82879e211b1098ad64e33d435f062c37612a018d4f9452200: Status 404 returned error can't find the container with id 80db03393b7675e82879e211b1098ad64e33d435f062c37612a018d4f9452200 Jul 23 21:59:11.171110 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:11.171081 2578 generic.go:356] "Generic (PLEG): container finished" podID="2c72f0f0-99ff-4f6e-a4bc-8e630d97f177" containerID="284dfac95a5a6d7f75825acc06839a774a153964898d4270ec75e950bd74cfe9" exitCode=0 Jul 23 21:59:11.171286 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:11.171172 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732886f" event={"ID":"2c72f0f0-99ff-4f6e-a4bc-8e630d97f177","Type":"ContainerDied","Data":"284dfac95a5a6d7f75825acc06839a774a153964898d4270ec75e950bd74cfe9"} Jul 23 21:59:11.171286 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:11.171231 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732886f" event={"ID":"2c72f0f0-99ff-4f6e-a4bc-8e630d97f177","Type":"ContainerStarted","Data":"80db03393b7675e82879e211b1098ad64e33d435f062c37612a018d4f9452200"} Jul 23 21:59:11.183418 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:11.183395 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jcc56"] Jul 23 21:59:11.185811 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:11.185798 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jcc56" Jul 23 21:59:11.195367 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:11.195346 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jcc56"] Jul 23 21:59:11.308473 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:11.308443 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1c0bc115-5e71-4a61-b2a9-8cee460568ce-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jcc56\" (UID: \"1c0bc115-5e71-4a61-b2a9-8cee460568ce\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jcc56" Jul 23 21:59:11.308669 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:11.308490 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q7q2c\" (UniqueName: \"kubernetes.io/projected/1c0bc115-5e71-4a61-b2a9-8cee460568ce-kube-api-access-q7q2c\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jcc56\" (UID: \"1c0bc115-5e71-4a61-b2a9-8cee460568ce\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jcc56" Jul 23 21:59:11.308669 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:11.308650 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1c0bc115-5e71-4a61-b2a9-8cee460568ce-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jcc56\" (UID: \"1c0bc115-5e71-4a61-b2a9-8cee460568ce\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jcc56" Jul 23 21:59:11.409885 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:11.409847 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1c0bc115-5e71-4a61-b2a9-8cee460568ce-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jcc56\" (UID: \"1c0bc115-5e71-4a61-b2a9-8cee460568ce\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jcc56" Jul 23 21:59:11.410307 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:11.409904 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q7q2c\" (UniqueName: \"kubernetes.io/projected/1c0bc115-5e71-4a61-b2a9-8cee460568ce-kube-api-access-q7q2c\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jcc56\" (UID: \"1c0bc115-5e71-4a61-b2a9-8cee460568ce\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jcc56" Jul 23 21:59:11.410307 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:11.409949 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1c0bc115-5e71-4a61-b2a9-8cee460568ce-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jcc56\" (UID: \"1c0bc115-5e71-4a61-b2a9-8cee460568ce\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jcc56" Jul 23 21:59:11.410307 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:11.410293 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1c0bc115-5e71-4a61-b2a9-8cee460568ce-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jcc56\" (UID: \"1c0bc115-5e71-4a61-b2a9-8cee460568ce\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jcc56" Jul 23 21:59:11.410423 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:11.410301 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1c0bc115-5e71-4a61-b2a9-8cee460568ce-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jcc56\" (UID: \"1c0bc115-5e71-4a61-b2a9-8cee460568ce\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jcc56" Jul 23 21:59:11.418859 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:11.418837 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q7q2c\" (UniqueName: \"kubernetes.io/projected/1c0bc115-5e71-4a61-b2a9-8cee460568ce-kube-api-access-q7q2c\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jcc56\" (UID: \"1c0bc115-5e71-4a61-b2a9-8cee460568ce\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jcc56" Jul 23 21:59:11.495837 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:11.495777 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jcc56" Jul 23 21:59:11.579336 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:11.579302 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hppvf"] Jul 23 21:59:11.582123 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:11.582100 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hppvf" Jul 23 21:59:11.592377 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:11.592314 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hppvf"] Jul 23 21:59:11.625330 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:11.625307 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jcc56"] Jul 23 21:59:11.627195 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:59:11.627171 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c0bc115_5e71_4a61_b2a9_8cee460568ce.slice/crio-8f230532bd55084fe90f7f5e9049b1acac35bf12265e0c7393666d66524643c5 WatchSource:0}: Error finding container 8f230532bd55084fe90f7f5e9049b1acac35bf12265e0c7393666d66524643c5: Status 404 returned error can't find the container with id 8f230532bd55084fe90f7f5e9049b1acac35bf12265e0c7393666d66524643c5 Jul 23 21:59:11.712736 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:11.712708 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dfa15e28-ab86-4899-8aa8-c7ccabbc2a12-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hppvf\" (UID: \"dfa15e28-ab86-4899-8aa8-c7ccabbc2a12\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hppvf" Jul 23 21:59:11.712893 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:11.712776 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vzgcm\" (UniqueName: \"kubernetes.io/projected/dfa15e28-ab86-4899-8aa8-c7ccabbc2a12-kube-api-access-vzgcm\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hppvf\" (UID: \"dfa15e28-ab86-4899-8aa8-c7ccabbc2a12\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hppvf" Jul 23 21:59:11.712893 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:11.712796 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dfa15e28-ab86-4899-8aa8-c7ccabbc2a12-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hppvf\" (UID: \"dfa15e28-ab86-4899-8aa8-c7ccabbc2a12\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hppvf" Jul 23 21:59:11.813655 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:11.813626 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vzgcm\" (UniqueName: \"kubernetes.io/projected/dfa15e28-ab86-4899-8aa8-c7ccabbc2a12-kube-api-access-vzgcm\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hppvf\" (UID: \"dfa15e28-ab86-4899-8aa8-c7ccabbc2a12\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hppvf" Jul 23 21:59:11.813767 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:11.813674 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dfa15e28-ab86-4899-8aa8-c7ccabbc2a12-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hppvf\" (UID: \"dfa15e28-ab86-4899-8aa8-c7ccabbc2a12\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hppvf" Jul 23 21:59:11.813767 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:11.813741 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dfa15e28-ab86-4899-8aa8-c7ccabbc2a12-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hppvf\" (UID: \"dfa15e28-ab86-4899-8aa8-c7ccabbc2a12\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hppvf" Jul 23 21:59:11.814021 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:11.814002 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dfa15e28-ab86-4899-8aa8-c7ccabbc2a12-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hppvf\" (UID: \"dfa15e28-ab86-4899-8aa8-c7ccabbc2a12\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hppvf" Jul 23 21:59:11.814072 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:11.814038 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dfa15e28-ab86-4899-8aa8-c7ccabbc2a12-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hppvf\" (UID: \"dfa15e28-ab86-4899-8aa8-c7ccabbc2a12\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hppvf" Jul 23 21:59:11.823160 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:11.823138 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vzgcm\" (UniqueName: \"kubernetes.io/projected/dfa15e28-ab86-4899-8aa8-c7ccabbc2a12-kube-api-access-vzgcm\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hppvf\" (UID: \"dfa15e28-ab86-4899-8aa8-c7ccabbc2a12\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hppvf" Jul 23 21:59:11.893763 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:11.893733 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hppvf" Jul 23 21:59:11.983228 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:11.983197 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759p9kwg"] Jul 23 21:59:11.985987 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:11.985962 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759p9kwg" Jul 23 21:59:11.995682 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:11.995649 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759p9kwg"] Jul 23 21:59:12.025372 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:12.025338 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hppvf"] Jul 23 21:59:12.027390 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:59:12.027361 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddfa15e28_ab86_4899_8aa8_c7ccabbc2a12.slice/crio-262bc4479d76f64a510be04439d160cf5e417f24a9d97aad80836629953d6431 WatchSource:0}: Error finding container 262bc4479d76f64a510be04439d160cf5e417f24a9d97aad80836629953d6431: Status 404 returned error can't find the container with id 262bc4479d76f64a510be04439d160cf5e417f24a9d97aad80836629953d6431 Jul 23 21:59:12.116011 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:12.115979 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/21657d68-6f73-4633-ac53-6ec0f2f84890-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759p9kwg\" (UID: \"21657d68-6f73-4633-ac53-6ec0f2f84890\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759p9kwg" Jul 23 21:59:12.116011 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:12.116014 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/21657d68-6f73-4633-ac53-6ec0f2f84890-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759p9kwg\" (UID: \"21657d68-6f73-4633-ac53-6ec0f2f84890\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759p9kwg" Jul 23 21:59:12.116235 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:12.116055 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dzjp4\" (UniqueName: \"kubernetes.io/projected/21657d68-6f73-4633-ac53-6ec0f2f84890-kube-api-access-dzjp4\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759p9kwg\" (UID: \"21657d68-6f73-4633-ac53-6ec0f2f84890\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759p9kwg" Jul 23 21:59:12.177142 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:12.177108 2578 generic.go:356] "Generic (PLEG): container finished" podID="2c72f0f0-99ff-4f6e-a4bc-8e630d97f177" containerID="47ed8983b9105dfac448ba70f2696a4a3c5306d03d8ae3e48b6de110121fe48c" exitCode=0 Jul 23 21:59:12.177295 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:12.177162 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732886f" event={"ID":"2c72f0f0-99ff-4f6e-a4bc-8e630d97f177","Type":"ContainerDied","Data":"47ed8983b9105dfac448ba70f2696a4a3c5306d03d8ae3e48b6de110121fe48c"} Jul 23 21:59:12.178866 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:12.178842 2578 generic.go:356] "Generic (PLEG): container finished" podID="dfa15e28-ab86-4899-8aa8-c7ccabbc2a12" containerID="c7590d6abaf2c43188c806095c07de11f5ee7f6f6b1879ef7285774d24d5ace6" exitCode=0 Jul 23 21:59:12.178952 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:12.178907 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hppvf" event={"ID":"dfa15e28-ab86-4899-8aa8-c7ccabbc2a12","Type":"ContainerDied","Data":"c7590d6abaf2c43188c806095c07de11f5ee7f6f6b1879ef7285774d24d5ace6"} Jul 23 21:59:12.179009 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:12.178951 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hppvf" event={"ID":"dfa15e28-ab86-4899-8aa8-c7ccabbc2a12","Type":"ContainerStarted","Data":"262bc4479d76f64a510be04439d160cf5e417f24a9d97aad80836629953d6431"} Jul 23 21:59:12.180620 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:12.180589 2578 generic.go:356] "Generic (PLEG): container finished" podID="1c0bc115-5e71-4a61-b2a9-8cee460568ce" containerID="03b01a23c7d772f7ea822df1388e17bd6d08fa4acf19855ecd0f13b910e1d894" exitCode=0 Jul 23 21:59:12.180717 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:12.180632 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jcc56" event={"ID":"1c0bc115-5e71-4a61-b2a9-8cee460568ce","Type":"ContainerDied","Data":"03b01a23c7d772f7ea822df1388e17bd6d08fa4acf19855ecd0f13b910e1d894"} Jul 23 21:59:12.180717 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:12.180674 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jcc56" event={"ID":"1c0bc115-5e71-4a61-b2a9-8cee460568ce","Type":"ContainerStarted","Data":"8f230532bd55084fe90f7f5e9049b1acac35bf12265e0c7393666d66524643c5"} Jul 23 21:59:12.216641 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:12.216617 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/21657d68-6f73-4633-ac53-6ec0f2f84890-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759p9kwg\" (UID: \"21657d68-6f73-4633-ac53-6ec0f2f84890\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759p9kwg" Jul 23 21:59:12.216749 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:12.216647 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/21657d68-6f73-4633-ac53-6ec0f2f84890-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759p9kwg\" (UID: \"21657d68-6f73-4633-ac53-6ec0f2f84890\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759p9kwg" Jul 23 21:59:12.216749 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:12.216684 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dzjp4\" (UniqueName: \"kubernetes.io/projected/21657d68-6f73-4633-ac53-6ec0f2f84890-kube-api-access-dzjp4\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759p9kwg\" (UID: \"21657d68-6f73-4633-ac53-6ec0f2f84890\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759p9kwg" Jul 23 21:59:12.216995 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:12.216972 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/21657d68-6f73-4633-ac53-6ec0f2f84890-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759p9kwg\" (UID: \"21657d68-6f73-4633-ac53-6ec0f2f84890\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759p9kwg" Jul 23 21:59:12.217060 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:12.216977 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/21657d68-6f73-4633-ac53-6ec0f2f84890-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759p9kwg\" (UID: \"21657d68-6f73-4633-ac53-6ec0f2f84890\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759p9kwg" Jul 23 21:59:12.226446 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:12.226424 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dzjp4\" (UniqueName: \"kubernetes.io/projected/21657d68-6f73-4633-ac53-6ec0f2f84890-kube-api-access-dzjp4\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759p9kwg\" (UID: \"21657d68-6f73-4633-ac53-6ec0f2f84890\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759p9kwg" Jul 23 21:59:12.298893 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:12.298866 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759p9kwg" Jul 23 21:59:12.429617 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:12.429591 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759p9kwg"] Jul 23 21:59:12.431553 ip-10-0-136-157 kubenswrapper[2578]: W0723 21:59:12.431510 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21657d68_6f73_4633_ac53_6ec0f2f84890.slice/crio-0bc2dd1664fe19fe00c1a1ef127afdc5673bdb471230a4b1f6879fb1e7674f50 WatchSource:0}: Error finding container 0bc2dd1664fe19fe00c1a1ef127afdc5673bdb471230a4b1f6879fb1e7674f50: Status 404 returned error can't find the container with id 0bc2dd1664fe19fe00c1a1ef127afdc5673bdb471230a4b1f6879fb1e7674f50 Jul 23 21:59:13.188816 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:13.188725 2578 generic.go:356] "Generic (PLEG): container finished" podID="2c72f0f0-99ff-4f6e-a4bc-8e630d97f177" containerID="3c62f0683c67c45a4f80eec1fd32ec9e1e9ef667878cb5466e8d739ba9efe41c" exitCode=0 Jul 23 21:59:13.188978 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:13.188829 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732886f" event={"ID":"2c72f0f0-99ff-4f6e-a4bc-8e630d97f177","Type":"ContainerDied","Data":"3c62f0683c67c45a4f80eec1fd32ec9e1e9ef667878cb5466e8d739ba9efe41c"} Jul 23 21:59:13.190149 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:13.190125 2578 generic.go:356] "Generic (PLEG): container finished" podID="21657d68-6f73-4633-ac53-6ec0f2f84890" containerID="11f6b0d3e41f22a08fa6a1490f6c0061e54e2542bdf445593282fbad0662cc24" exitCode=0 Jul 23 21:59:13.190266 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:13.190236 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759p9kwg" event={"ID":"21657d68-6f73-4633-ac53-6ec0f2f84890","Type":"ContainerDied","Data":"11f6b0d3e41f22a08fa6a1490f6c0061e54e2542bdf445593282fbad0662cc24"} Jul 23 21:59:13.190316 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:13.190274 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759p9kwg" event={"ID":"21657d68-6f73-4633-ac53-6ec0f2f84890","Type":"ContainerStarted","Data":"0bc2dd1664fe19fe00c1a1ef127afdc5673bdb471230a4b1f6879fb1e7674f50"} Jul 23 21:59:13.192016 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:13.191996 2578 generic.go:356] "Generic (PLEG): container finished" podID="dfa15e28-ab86-4899-8aa8-c7ccabbc2a12" containerID="0b78f3e29890ebd7a7eb3394cf421805b0c838cdda2ca2c5d36de6c3de9da9db" exitCode=0 Jul 23 21:59:13.192096 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:13.192072 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hppvf" event={"ID":"dfa15e28-ab86-4899-8aa8-c7ccabbc2a12","Type":"ContainerDied","Data":"0b78f3e29890ebd7a7eb3394cf421805b0c838cdda2ca2c5d36de6c3de9da9db"} Jul 23 21:59:13.194056 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:13.194027 2578 generic.go:356] "Generic (PLEG): container finished" podID="1c0bc115-5e71-4a61-b2a9-8cee460568ce" containerID="d7ba6115900cd3c14bbd9ba2b19fa345022d7beceaa1af083a947a1b245b34e8" exitCode=0 Jul 23 21:59:13.194134 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:13.194055 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jcc56" event={"ID":"1c0bc115-5e71-4a61-b2a9-8cee460568ce","Type":"ContainerDied","Data":"d7ba6115900cd3c14bbd9ba2b19fa345022d7beceaa1af083a947a1b245b34e8"} Jul 23 21:59:14.200946 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:14.200860 2578 generic.go:356] "Generic (PLEG): container finished" podID="1c0bc115-5e71-4a61-b2a9-8cee460568ce" containerID="b4506646a6387d260126ecdaa6b0c25d6dc0d432f42b76e72ff22198082fcfd9" exitCode=0 Jul 23 21:59:14.201327 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:14.200944 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jcc56" event={"ID":"1c0bc115-5e71-4a61-b2a9-8cee460568ce","Type":"ContainerDied","Data":"b4506646a6387d260126ecdaa6b0c25d6dc0d432f42b76e72ff22198082fcfd9"} Jul 23 21:59:14.202668 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:14.202646 2578 generic.go:356] "Generic (PLEG): container finished" podID="21657d68-6f73-4633-ac53-6ec0f2f84890" containerID="3c3b20c9688f369ba9b3e25ee6bd3c33f3391e1f69726395f4bc4e5dcd6d13d8" exitCode=0 Jul 23 21:59:14.202774 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:14.202724 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759p9kwg" event={"ID":"21657d68-6f73-4633-ac53-6ec0f2f84890","Type":"ContainerDied","Data":"3c3b20c9688f369ba9b3e25ee6bd3c33f3391e1f69726395f4bc4e5dcd6d13d8"} Jul 23 21:59:14.204887 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:14.204832 2578 generic.go:356] "Generic (PLEG): container finished" podID="dfa15e28-ab86-4899-8aa8-c7ccabbc2a12" containerID="87cd1f4a33c973d473a7edab6ff8c740d961dc2f5e81149e61fcfe7a6cae2f82" exitCode=0 Jul 23 21:59:14.204963 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:14.204906 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hppvf" event={"ID":"dfa15e28-ab86-4899-8aa8-c7ccabbc2a12","Type":"ContainerDied","Data":"87cd1f4a33c973d473a7edab6ff8c740d961dc2f5e81149e61fcfe7a6cae2f82"} Jul 23 21:59:14.333866 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:14.333842 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732886f" Jul 23 21:59:14.438297 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:14.438269 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2c72f0f0-99ff-4f6e-a4bc-8e630d97f177-bundle\") pod \"2c72f0f0-99ff-4f6e-a4bc-8e630d97f177\" (UID: \"2c72f0f0-99ff-4f6e-a4bc-8e630d97f177\") " Jul 23 21:59:14.438432 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:14.438351 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2c72f0f0-99ff-4f6e-a4bc-8e630d97f177-util\") pod \"2c72f0f0-99ff-4f6e-a4bc-8e630d97f177\" (UID: \"2c72f0f0-99ff-4f6e-a4bc-8e630d97f177\") " Jul 23 21:59:14.438432 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:14.438398 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bzv66\" (UniqueName: \"kubernetes.io/projected/2c72f0f0-99ff-4f6e-a4bc-8e630d97f177-kube-api-access-bzv66\") pod \"2c72f0f0-99ff-4f6e-a4bc-8e630d97f177\" (UID: \"2c72f0f0-99ff-4f6e-a4bc-8e630d97f177\") " Jul 23 21:59:14.438963 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:14.438932 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c72f0f0-99ff-4f6e-a4bc-8e630d97f177-bundle" (OuterVolumeSpecName: "bundle") pod "2c72f0f0-99ff-4f6e-a4bc-8e630d97f177" (UID: "2c72f0f0-99ff-4f6e-a4bc-8e630d97f177"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 21:59:14.440717 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:14.440666 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c72f0f0-99ff-4f6e-a4bc-8e630d97f177-kube-api-access-bzv66" (OuterVolumeSpecName: "kube-api-access-bzv66") pod "2c72f0f0-99ff-4f6e-a4bc-8e630d97f177" (UID: "2c72f0f0-99ff-4f6e-a4bc-8e630d97f177"). InnerVolumeSpecName "kube-api-access-bzv66". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 21:59:14.444299 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:14.444281 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c72f0f0-99ff-4f6e-a4bc-8e630d97f177-util" (OuterVolumeSpecName: "util") pod "2c72f0f0-99ff-4f6e-a4bc-8e630d97f177" (UID: "2c72f0f0-99ff-4f6e-a4bc-8e630d97f177"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 21:59:14.538979 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:14.538912 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2c72f0f0-99ff-4f6e-a4bc-8e630d97f177-util\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\"" Jul 23 21:59:14.538979 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:14.538935 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bzv66\" (UniqueName: \"kubernetes.io/projected/2c72f0f0-99ff-4f6e-a4bc-8e630d97f177-kube-api-access-bzv66\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\"" Jul 23 21:59:14.538979 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:14.538945 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2c72f0f0-99ff-4f6e-a4bc-8e630d97f177-bundle\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\"" Jul 23 21:59:15.211739 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:15.211708 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732886f" Jul 23 21:59:15.211739 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:15.211717 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732886f" event={"ID":"2c72f0f0-99ff-4f6e-a4bc-8e630d97f177","Type":"ContainerDied","Data":"80db03393b7675e82879e211b1098ad64e33d435f062c37612a018d4f9452200"} Jul 23 21:59:15.212266 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:15.211769 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="80db03393b7675e82879e211b1098ad64e33d435f062c37612a018d4f9452200" Jul 23 21:59:15.213850 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:15.213821 2578 generic.go:356] "Generic (PLEG): container finished" podID="21657d68-6f73-4633-ac53-6ec0f2f84890" containerID="6ceb01198f95eab30bf90d6b8f86392f9fb1cda95e244e8c723576dd9458b326" exitCode=0 Jul 23 21:59:15.213958 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:15.213855 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759p9kwg" event={"ID":"21657d68-6f73-4633-ac53-6ec0f2f84890","Type":"ContainerDied","Data":"6ceb01198f95eab30bf90d6b8f86392f9fb1cda95e244e8c723576dd9458b326"} Jul 23 21:59:15.346001 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:15.345977 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jcc56" Jul 23 21:59:15.378872 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:15.378849 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hppvf" Jul 23 21:59:15.446074 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:15.446040 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1c0bc115-5e71-4a61-b2a9-8cee460568ce-util\") pod \"1c0bc115-5e71-4a61-b2a9-8cee460568ce\" (UID: \"1c0bc115-5e71-4a61-b2a9-8cee460568ce\") " Jul 23 21:59:15.446074 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:15.446079 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dfa15e28-ab86-4899-8aa8-c7ccabbc2a12-util\") pod \"dfa15e28-ab86-4899-8aa8-c7ccabbc2a12\" (UID: \"dfa15e28-ab86-4899-8aa8-c7ccabbc2a12\") " Jul 23 21:59:15.446311 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:15.446115 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dfa15e28-ab86-4899-8aa8-c7ccabbc2a12-bundle\") pod \"dfa15e28-ab86-4899-8aa8-c7ccabbc2a12\" (UID: \"dfa15e28-ab86-4899-8aa8-c7ccabbc2a12\") " Jul 23 21:59:15.446311 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:15.446188 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q7q2c\" (UniqueName: \"kubernetes.io/projected/1c0bc115-5e71-4a61-b2a9-8cee460568ce-kube-api-access-q7q2c\") pod \"1c0bc115-5e71-4a61-b2a9-8cee460568ce\" (UID: \"1c0bc115-5e71-4a61-b2a9-8cee460568ce\") " Jul 23 21:59:15.446311 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:15.446212 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1c0bc115-5e71-4a61-b2a9-8cee460568ce-bundle\") pod \"1c0bc115-5e71-4a61-b2a9-8cee460568ce\" (UID: \"1c0bc115-5e71-4a61-b2a9-8cee460568ce\") " Jul 23 21:59:15.446462 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:15.446343 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vzgcm\" (UniqueName: \"kubernetes.io/projected/dfa15e28-ab86-4899-8aa8-c7ccabbc2a12-kube-api-access-vzgcm\") pod \"dfa15e28-ab86-4899-8aa8-c7ccabbc2a12\" (UID: \"dfa15e28-ab86-4899-8aa8-c7ccabbc2a12\") " Jul 23 21:59:15.446741 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:15.446716 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dfa15e28-ab86-4899-8aa8-c7ccabbc2a12-bundle" (OuterVolumeSpecName: "bundle") pod "dfa15e28-ab86-4899-8aa8-c7ccabbc2a12" (UID: "dfa15e28-ab86-4899-8aa8-c7ccabbc2a12"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 21:59:15.446860 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:15.446750 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1c0bc115-5e71-4a61-b2a9-8cee460568ce-bundle" (OuterVolumeSpecName: "bundle") pod "1c0bc115-5e71-4a61-b2a9-8cee460568ce" (UID: "1c0bc115-5e71-4a61-b2a9-8cee460568ce"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 21:59:15.448329 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:15.448310 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1c0bc115-5e71-4a61-b2a9-8cee460568ce-kube-api-access-q7q2c" (OuterVolumeSpecName: "kube-api-access-q7q2c") pod "1c0bc115-5e71-4a61-b2a9-8cee460568ce" (UID: "1c0bc115-5e71-4a61-b2a9-8cee460568ce"). InnerVolumeSpecName "kube-api-access-q7q2c". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 21:59:15.448715 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:15.448691 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dfa15e28-ab86-4899-8aa8-c7ccabbc2a12-kube-api-access-vzgcm" (OuterVolumeSpecName: "kube-api-access-vzgcm") pod "dfa15e28-ab86-4899-8aa8-c7ccabbc2a12" (UID: "dfa15e28-ab86-4899-8aa8-c7ccabbc2a12"). InnerVolumeSpecName "kube-api-access-vzgcm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 21:59:15.452114 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:15.452085 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dfa15e28-ab86-4899-8aa8-c7ccabbc2a12-util" (OuterVolumeSpecName: "util") pod "dfa15e28-ab86-4899-8aa8-c7ccabbc2a12" (UID: "dfa15e28-ab86-4899-8aa8-c7ccabbc2a12"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 21:59:15.452380 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:15.452363 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1c0bc115-5e71-4a61-b2a9-8cee460568ce-util" (OuterVolumeSpecName: "util") pod "1c0bc115-5e71-4a61-b2a9-8cee460568ce" (UID: "1c0bc115-5e71-4a61-b2a9-8cee460568ce"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 21:59:15.547401 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:15.547330 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vzgcm\" (UniqueName: \"kubernetes.io/projected/dfa15e28-ab86-4899-8aa8-c7ccabbc2a12-kube-api-access-vzgcm\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\"" Jul 23 21:59:15.547401 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:15.547359 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1c0bc115-5e71-4a61-b2a9-8cee460568ce-util\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\"" Jul 23 21:59:15.547401 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:15.547368 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/dfa15e28-ab86-4899-8aa8-c7ccabbc2a12-util\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\"" Jul 23 21:59:15.547401 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:15.547375 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/dfa15e28-ab86-4899-8aa8-c7ccabbc2a12-bundle\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\"" Jul 23 21:59:15.547401 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:15.547384 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q7q2c\" (UniqueName: \"kubernetes.io/projected/1c0bc115-5e71-4a61-b2a9-8cee460568ce-kube-api-access-q7q2c\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\"" Jul 23 21:59:15.547401 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:15.547391 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1c0bc115-5e71-4a61-b2a9-8cee460568ce-bundle\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\"" Jul 23 21:59:16.220880 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:16.220852 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hppvf" Jul 23 21:59:16.221332 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:16.220859 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hppvf" event={"ID":"dfa15e28-ab86-4899-8aa8-c7ccabbc2a12","Type":"ContainerDied","Data":"262bc4479d76f64a510be04439d160cf5e417f24a9d97aad80836629953d6431"} Jul 23 21:59:16.221332 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:16.220990 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="262bc4479d76f64a510be04439d160cf5e417f24a9d97aad80836629953d6431" Jul 23 21:59:16.228898 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:16.228864 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jcc56" event={"ID":"1c0bc115-5e71-4a61-b2a9-8cee460568ce","Type":"ContainerDied","Data":"8f230532bd55084fe90f7f5e9049b1acac35bf12265e0c7393666d66524643c5"} Jul 23 21:59:16.229061 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:16.228905 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jcc56" Jul 23 21:59:16.229061 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:16.228948 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8f230532bd55084fe90f7f5e9049b1acac35bf12265e0c7393666d66524643c5" Jul 23 21:59:16.358408 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:16.358384 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759p9kwg" Jul 23 21:59:16.454588 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:16.454556 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/21657d68-6f73-4633-ac53-6ec0f2f84890-bundle\") pod \"21657d68-6f73-4633-ac53-6ec0f2f84890\" (UID: \"21657d68-6f73-4633-ac53-6ec0f2f84890\") " Jul 23 21:59:16.454764 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:16.454606 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/21657d68-6f73-4633-ac53-6ec0f2f84890-util\") pod \"21657d68-6f73-4633-ac53-6ec0f2f84890\" (UID: \"21657d68-6f73-4633-ac53-6ec0f2f84890\") " Jul 23 21:59:16.454764 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:16.454649 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dzjp4\" (UniqueName: \"kubernetes.io/projected/21657d68-6f73-4633-ac53-6ec0f2f84890-kube-api-access-dzjp4\") pod \"21657d68-6f73-4633-ac53-6ec0f2f84890\" (UID: \"21657d68-6f73-4633-ac53-6ec0f2f84890\") " Jul 23 21:59:16.455000 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:16.454977 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/21657d68-6f73-4633-ac53-6ec0f2f84890-bundle" (OuterVolumeSpecName: "bundle") pod "21657d68-6f73-4633-ac53-6ec0f2f84890" (UID: "21657d68-6f73-4633-ac53-6ec0f2f84890"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 21:59:16.456785 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:16.456761 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/21657d68-6f73-4633-ac53-6ec0f2f84890-kube-api-access-dzjp4" (OuterVolumeSpecName: "kube-api-access-dzjp4") pod "21657d68-6f73-4633-ac53-6ec0f2f84890" (UID: "21657d68-6f73-4633-ac53-6ec0f2f84890"). InnerVolumeSpecName "kube-api-access-dzjp4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 21:59:16.459749 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:16.459726 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/21657d68-6f73-4633-ac53-6ec0f2f84890-util" (OuterVolumeSpecName: "util") pod "21657d68-6f73-4633-ac53-6ec0f2f84890" (UID: "21657d68-6f73-4633-ac53-6ec0f2f84890"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 21:59:16.555379 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:16.555354 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/21657d68-6f73-4633-ac53-6ec0f2f84890-bundle\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\"" Jul 23 21:59:16.555379 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:16.555377 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/21657d68-6f73-4633-ac53-6ec0f2f84890-util\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\"" Jul 23 21:59:16.555565 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:16.555386 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dzjp4\" (UniqueName: \"kubernetes.io/projected/21657d68-6f73-4633-ac53-6ec0f2f84890-kube-api-access-dzjp4\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\"" Jul 23 21:59:17.236747 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:17.236718 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759p9kwg" Jul 23 21:59:17.236747 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:17.236733 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759p9kwg" event={"ID":"21657d68-6f73-4633-ac53-6ec0f2f84890","Type":"ContainerDied","Data":"0bc2dd1664fe19fe00c1a1ef127afdc5673bdb471230a4b1f6879fb1e7674f50"} Jul 23 21:59:17.237191 ip-10-0-136-157 kubenswrapper[2578]: I0723 21:59:17.236774 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0bc2dd1664fe19fe00c1a1ef127afdc5673bdb471230a4b1f6879fb1e7674f50" Jul 23 22:00:20.367748 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:00:20.367714 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jnrd4"] Jul 23 22:00:20.368151 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:00:20.368137 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c72f0f0-99ff-4f6e-a4bc-8e630d97f177" containerName="pull" Jul 23 22:00:20.368196 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:00:20.368154 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c72f0f0-99ff-4f6e-a4bc-8e630d97f177" containerName="pull" Jul 23 22:00:20.368196 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:00:20.368169 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="21657d68-6f73-4633-ac53-6ec0f2f84890" containerName="util" Jul 23 22:00:20.368196 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:00:20.368178 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="21657d68-6f73-4633-ac53-6ec0f2f84890" containerName="util" Jul 23 22:00:20.368196 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:00:20.368189 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="21657d68-6f73-4633-ac53-6ec0f2f84890" containerName="pull" Jul 23 22:00:20.368318 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:00:20.368197 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="21657d68-6f73-4633-ac53-6ec0f2f84890" containerName="pull" Jul 23 22:00:20.368318 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:00:20.368214 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c0bc115-5e71-4a61-b2a9-8cee460568ce" containerName="pull" Jul 23 22:00:20.368318 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:00:20.368221 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c0bc115-5e71-4a61-b2a9-8cee460568ce" containerName="pull" Jul 23 22:00:20.368318 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:00:20.368232 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c0bc115-5e71-4a61-b2a9-8cee460568ce" containerName="extract" Jul 23 22:00:20.368318 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:00:20.368240 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c0bc115-5e71-4a61-b2a9-8cee460568ce" containerName="extract" Jul 23 22:00:20.368318 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:00:20.368252 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dfa15e28-ab86-4899-8aa8-c7ccabbc2a12" containerName="extract" Jul 23 22:00:20.368318 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:00:20.368259 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="dfa15e28-ab86-4899-8aa8-c7ccabbc2a12" containerName="extract" Jul 23 22:00:20.368318 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:00:20.368269 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c72f0f0-99ff-4f6e-a4bc-8e630d97f177" containerName="util" Jul 23 22:00:20.368318 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:00:20.368276 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c72f0f0-99ff-4f6e-a4bc-8e630d97f177" containerName="util" Jul 23 22:00:20.368318 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:00:20.368285 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dfa15e28-ab86-4899-8aa8-c7ccabbc2a12" containerName="pull" Jul 23 22:00:20.368318 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:00:20.368293 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="dfa15e28-ab86-4899-8aa8-c7ccabbc2a12" containerName="pull" Jul 23 22:00:20.368318 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:00:20.368301 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="21657d68-6f73-4633-ac53-6ec0f2f84890" containerName="extract" Jul 23 22:00:20.368318 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:00:20.368308 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="21657d68-6f73-4633-ac53-6ec0f2f84890" containerName="extract" Jul 23 22:00:20.368318 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:00:20.368321 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dfa15e28-ab86-4899-8aa8-c7ccabbc2a12" containerName="util" Jul 23 22:00:20.368722 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:00:20.368328 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="dfa15e28-ab86-4899-8aa8-c7ccabbc2a12" containerName="util" Jul 23 22:00:20.368722 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:00:20.368337 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c0bc115-5e71-4a61-b2a9-8cee460568ce" containerName="util" Jul 23 22:00:20.368722 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:00:20.368344 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c0bc115-5e71-4a61-b2a9-8cee460568ce" containerName="util" Jul 23 22:00:20.368722 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:00:20.368353 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c72f0f0-99ff-4f6e-a4bc-8e630d97f177" containerName="extract" Jul 23 22:00:20.368722 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:00:20.368360 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c72f0f0-99ff-4f6e-a4bc-8e630d97f177" containerName="extract" Jul 23 22:00:20.368722 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:00:20.368422 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="21657d68-6f73-4633-ac53-6ec0f2f84890" containerName="extract" Jul 23 22:00:20.368722 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:00:20.368434 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="dfa15e28-ab86-4899-8aa8-c7ccabbc2a12" containerName="extract" Jul 23 22:00:20.368722 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:00:20.368443 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="2c72f0f0-99ff-4f6e-a4bc-8e630d97f177" containerName="extract" Jul 23 22:00:20.368722 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:00:20.368451 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="1c0bc115-5e71-4a61-b2a9-8cee460568ce" containerName="extract" Jul 23 22:00:20.371378 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:00:20.371363 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jnrd4" Jul 23 22:00:20.377047 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:00:20.377026 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-gmpsq\"" Jul 23 22:00:20.386497 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:00:20.386472 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jnrd4"] Jul 23 22:00:20.480797 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:00:20.480763 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9872c26d-b195-44ba-ba89-05aae0c7ff98-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-jnrd4\" (UID: \"9872c26d-b195-44ba-ba89-05aae0c7ff98\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jnrd4" Jul 23 22:00:20.480980 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:00:20.480820 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9872c26d-b195-44ba-ba89-05aae0c7ff98-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-jnrd4\" (UID: \"9872c26d-b195-44ba-ba89-05aae0c7ff98\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jnrd4" Jul 23 22:00:20.480980 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:00:20.480873 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9872c26d-b195-44ba-ba89-05aae0c7ff98-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-jnrd4\" (UID: \"9872c26d-b195-44ba-ba89-05aae0c7ff98\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jnrd4" Jul 23 22:00:20.480980 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:00:20.480932 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z4sjz\" (UniqueName: \"kubernetes.io/projected/9872c26d-b195-44ba-ba89-05aae0c7ff98-kube-api-access-z4sjz\") pod \"maas-default-gateway-openshift-default-56654dcf7-jnrd4\" (UID: \"9872c26d-b195-44ba-ba89-05aae0c7ff98\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jnrd4" Jul 23 22:00:20.480980 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:00:20.480962 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9872c26d-b195-44ba-ba89-05aae0c7ff98-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-jnrd4\" (UID: \"9872c26d-b195-44ba-ba89-05aae0c7ff98\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jnrd4" Jul 23 22:00:20.480980 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:00:20.480977 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9872c26d-b195-44ba-ba89-05aae0c7ff98-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-jnrd4\" (UID: \"9872c26d-b195-44ba-ba89-05aae0c7ff98\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jnrd4" Jul 23 22:00:20.481161 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:00:20.481048 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9872c26d-b195-44ba-ba89-05aae0c7ff98-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-jnrd4\" (UID: \"9872c26d-b195-44ba-ba89-05aae0c7ff98\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jnrd4" Jul 23 22:00:20.481161 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:00:20.481088 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9872c26d-b195-44ba-ba89-05aae0c7ff98-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-jnrd4\" (UID: \"9872c26d-b195-44ba-ba89-05aae0c7ff98\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jnrd4" Jul 23 22:00:20.481161 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:00:20.481130 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9872c26d-b195-44ba-ba89-05aae0c7ff98-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-jnrd4\" (UID: \"9872c26d-b195-44ba-ba89-05aae0c7ff98\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jnrd4" Jul 23 22:00:20.581587 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:00:20.581531 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9872c26d-b195-44ba-ba89-05aae0c7ff98-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-jnrd4\" (UID: \"9872c26d-b195-44ba-ba89-05aae0c7ff98\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jnrd4" Jul 23 22:00:20.581587 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:00:20.581591 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9872c26d-b195-44ba-ba89-05aae0c7ff98-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-jnrd4\" (UID: \"9872c26d-b195-44ba-ba89-05aae0c7ff98\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jnrd4" Jul 23 22:00:20.581823 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:00:20.581703 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9872c26d-b195-44ba-ba89-05aae0c7ff98-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-jnrd4\" (UID: \"9872c26d-b195-44ba-ba89-05aae0c7ff98\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jnrd4" Jul 23 22:00:20.581823 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:00:20.581756 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9872c26d-b195-44ba-ba89-05aae0c7ff98-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-jnrd4\" (UID: \"9872c26d-b195-44ba-ba89-05aae0c7ff98\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jnrd4" Jul 23 22:00:20.581823 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:00:20.581796 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9872c26d-b195-44ba-ba89-05aae0c7ff98-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-jnrd4\" (UID: \"9872c26d-b195-44ba-ba89-05aae0c7ff98\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jnrd4" Jul 23 22:00:20.582028 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:00:20.582007 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9872c26d-b195-44ba-ba89-05aae0c7ff98-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-jnrd4\" (UID: \"9872c26d-b195-44ba-ba89-05aae0c7ff98\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jnrd4" Jul 23 22:00:20.582185 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:00:20.582149 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9872c26d-b195-44ba-ba89-05aae0c7ff98-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-jnrd4\" (UID: \"9872c26d-b195-44ba-ba89-05aae0c7ff98\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jnrd4" Jul 23 22:00:20.582312 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:00:20.582261 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9872c26d-b195-44ba-ba89-05aae0c7ff98-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-jnrd4\" (UID: \"9872c26d-b195-44ba-ba89-05aae0c7ff98\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jnrd4" Jul 23 22:00:20.582376 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:00:20.582345 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9872c26d-b195-44ba-ba89-05aae0c7ff98-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-jnrd4\" (UID: \"9872c26d-b195-44ba-ba89-05aae0c7ff98\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jnrd4" Jul 23 22:00:20.582436 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:00:20.582384 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9872c26d-b195-44ba-ba89-05aae0c7ff98-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-jnrd4\" (UID: \"9872c26d-b195-44ba-ba89-05aae0c7ff98\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jnrd4" Jul 23 22:00:20.582488 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:00:20.582450 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9872c26d-b195-44ba-ba89-05aae0c7ff98-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-jnrd4\" (UID: \"9872c26d-b195-44ba-ba89-05aae0c7ff98\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jnrd4" Jul 23 22:00:20.582488 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:00:20.582475 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9872c26d-b195-44ba-ba89-05aae0c7ff98-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-jnrd4\" (UID: \"9872c26d-b195-44ba-ba89-05aae0c7ff98\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jnrd4" Jul 23 22:00:20.582614 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:00:20.582563 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z4sjz\" (UniqueName: \"kubernetes.io/projected/9872c26d-b195-44ba-ba89-05aae0c7ff98-kube-api-access-z4sjz\") pod \"maas-default-gateway-openshift-default-56654dcf7-jnrd4\" (UID: \"9872c26d-b195-44ba-ba89-05aae0c7ff98\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jnrd4" Jul 23 22:00:20.582754 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:00:20.582731 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9872c26d-b195-44ba-ba89-05aae0c7ff98-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-jnrd4\" (UID: \"9872c26d-b195-44ba-ba89-05aae0c7ff98\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jnrd4" Jul 23 22:00:20.584779 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:00:20.584759 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9872c26d-b195-44ba-ba89-05aae0c7ff98-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-jnrd4\" (UID: \"9872c26d-b195-44ba-ba89-05aae0c7ff98\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jnrd4" Jul 23 22:00:20.585164 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:00:20.585140 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9872c26d-b195-44ba-ba89-05aae0c7ff98-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-jnrd4\" (UID: \"9872c26d-b195-44ba-ba89-05aae0c7ff98\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jnrd4" Jul 23 22:00:20.594957 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:00:20.594931 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z4sjz\" (UniqueName: \"kubernetes.io/projected/9872c26d-b195-44ba-ba89-05aae0c7ff98-kube-api-access-z4sjz\") pod \"maas-default-gateway-openshift-default-56654dcf7-jnrd4\" (UID: \"9872c26d-b195-44ba-ba89-05aae0c7ff98\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jnrd4" Jul 23 22:00:20.595903 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:00:20.595881 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9872c26d-b195-44ba-ba89-05aae0c7ff98-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-jnrd4\" (UID: \"9872c26d-b195-44ba-ba89-05aae0c7ff98\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jnrd4" Jul 23 22:00:20.680935 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:00:20.680838 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jnrd4" Jul 23 22:00:20.824922 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:00:20.824896 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jnrd4"] Jul 23 22:00:20.826577 ip-10-0-136-157 kubenswrapper[2578]: W0723 22:00:20.826525 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9872c26d_b195_44ba_ba89_05aae0c7ff98.slice/crio-8d1e2438d90b863678312d14af7452a31c4f0734178196496a4874bc5424bd4e WatchSource:0}: Error finding container 8d1e2438d90b863678312d14af7452a31c4f0734178196496a4874bc5424bd4e: Status 404 returned error can't find the container with id 8d1e2438d90b863678312d14af7452a31c4f0734178196496a4874bc5424bd4e Jul 23 22:00:20.828524 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:00:20.828497 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 23 22:00:20.828602 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:00:20.828587 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 23 22:00:20.828643 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:00:20.828625 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 23 22:00:21.526283 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:00:21.526250 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jnrd4" event={"ID":"9872c26d-b195-44ba-ba89-05aae0c7ff98","Type":"ContainerStarted","Data":"771ab5d9bf2d4a2db5ee69f5a89376f30742fc05805f90befeaadd0b51c44185"} Jul 23 22:00:21.526689 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:00:21.526298 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jnrd4" event={"ID":"9872c26d-b195-44ba-ba89-05aae0c7ff98","Type":"ContainerStarted","Data":"8d1e2438d90b863678312d14af7452a31c4f0734178196496a4874bc5424bd4e"} Jul 23 22:00:21.552190 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:00:21.552134 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jnrd4" podStartSLOduration=1.552118776 podStartE2EDuration="1.552118776s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 22:00:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 22:00:21.55005367 +0000 UTC m=+523.791961546" watchObservedRunningTime="2026-07-23 22:00:21.552118776 +0000 UTC m=+523.794026649" Jul 23 22:00:21.681362 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:00:21.681328 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jnrd4" Jul 23 22:00:21.685657 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:00:21.685631 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jnrd4" Jul 23 22:00:22.530613 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:00:22.530580 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jnrd4" Jul 23 22:00:22.531673 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:00:22.531654 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jnrd4" Jul 23 22:01:14.098826 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:01:14.098778 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6f9d846d69-nvw5h"] Jul 23 22:01:14.101880 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:01:14.101865 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6f9d846d69-nvw5h" Jul 23 22:01:14.104076 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:01:14.104045 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 23 22:01:14.104215 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:01:14.104153 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 23 22:01:14.104215 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:01:14.104153 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-rlp68\"" Jul 23 22:01:14.116809 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:01:14.116784 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6f9d846d69-nvw5h"] Jul 23 22:01:14.219586 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:01:14.219518 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b957e99c-4c6c-4c66-8111-e8420e04bf00-webhook-cert\") pod \"maas-controller-6f9d846d69-nvw5h\" (UID: \"b957e99c-4c6c-4c66-8111-e8420e04bf00\") " pod="opendatahub/maas-controller-6f9d846d69-nvw5h" Jul 23 22:01:14.219586 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:01:14.219588 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-947mw\" (UniqueName: \"kubernetes.io/projected/b957e99c-4c6c-4c66-8111-e8420e04bf00-kube-api-access-947mw\") pod \"maas-controller-6f9d846d69-nvw5h\" (UID: \"b957e99c-4c6c-4c66-8111-e8420e04bf00\") " pod="opendatahub/maas-controller-6f9d846d69-nvw5h" Jul 23 22:01:14.290007 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:01:14.289979 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7c697dc6f4-69b5g"] Jul 23 22:01:14.292999 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:01:14.292980 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7c697dc6f4-69b5g" Jul 23 22:01:14.306893 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:01:14.306871 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7c697dc6f4-69b5g"] Jul 23 22:01:14.320690 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:01:14.320655 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b957e99c-4c6c-4c66-8111-e8420e04bf00-webhook-cert\") pod \"maas-controller-6f9d846d69-nvw5h\" (UID: \"b957e99c-4c6c-4c66-8111-e8420e04bf00\") " pod="opendatahub/maas-controller-6f9d846d69-nvw5h" Jul 23 22:01:14.320829 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:01:14.320692 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-947mw\" (UniqueName: \"kubernetes.io/projected/b957e99c-4c6c-4c66-8111-e8420e04bf00-kube-api-access-947mw\") pod \"maas-controller-6f9d846d69-nvw5h\" (UID: \"b957e99c-4c6c-4c66-8111-e8420e04bf00\") " pod="opendatahub/maas-controller-6f9d846d69-nvw5h" Jul 23 22:01:14.320829 ip-10-0-136-157 kubenswrapper[2578]: E0723 22:01:14.320790 2578 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found Jul 23 22:01:14.320949 ip-10-0-136-157 kubenswrapper[2578]: E0723 22:01:14.320844 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b957e99c-4c6c-4c66-8111-e8420e04bf00-webhook-cert podName:b957e99c-4c6c-4c66-8111-e8420e04bf00 nodeName:}" failed. No retries permitted until 2026-07-23 22:01:14.820826779 +0000 UTC m=+577.062734631 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/b957e99c-4c6c-4c66-8111-e8420e04bf00-webhook-cert") pod "maas-controller-6f9d846d69-nvw5h" (UID: "b957e99c-4c6c-4c66-8111-e8420e04bf00") : secret "maas-controller-webhook-cert" not found Jul 23 22:01:14.330861 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:01:14.330825 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-947mw\" (UniqueName: \"kubernetes.io/projected/b957e99c-4c6c-4c66-8111-e8420e04bf00-kube-api-access-947mw\") pod \"maas-controller-6f9d846d69-nvw5h\" (UID: \"b957e99c-4c6c-4c66-8111-e8420e04bf00\") " pod="opendatahub/maas-controller-6f9d846d69-nvw5h" Jul 23 22:01:14.421754 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:01:14.421667 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gb2wz\" (UniqueName: \"kubernetes.io/projected/56c86278-94b0-48eb-85c6-4a99632012ca-kube-api-access-gb2wz\") pod \"maas-controller-7c697dc6f4-69b5g\" (UID: \"56c86278-94b0-48eb-85c6-4a99632012ca\") " pod="opendatahub/maas-controller-7c697dc6f4-69b5g" Jul 23 22:01:14.421754 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:01:14.421738 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/56c86278-94b0-48eb-85c6-4a99632012ca-webhook-cert\") pod \"maas-controller-7c697dc6f4-69b5g\" (UID: \"56c86278-94b0-48eb-85c6-4a99632012ca\") " pod="opendatahub/maas-controller-7c697dc6f4-69b5g" Jul 23 22:01:14.523053 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:01:14.523024 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/56c86278-94b0-48eb-85c6-4a99632012ca-webhook-cert\") pod \"maas-controller-7c697dc6f4-69b5g\" (UID: \"56c86278-94b0-48eb-85c6-4a99632012ca\") " pod="opendatahub/maas-controller-7c697dc6f4-69b5g" Jul 23 22:01:14.523209 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:01:14.523102 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gb2wz\" (UniqueName: \"kubernetes.io/projected/56c86278-94b0-48eb-85c6-4a99632012ca-kube-api-access-gb2wz\") pod \"maas-controller-7c697dc6f4-69b5g\" (UID: \"56c86278-94b0-48eb-85c6-4a99632012ca\") " pod="opendatahub/maas-controller-7c697dc6f4-69b5g" Jul 23 22:01:14.525574 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:01:14.525522 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/56c86278-94b0-48eb-85c6-4a99632012ca-webhook-cert\") pod \"maas-controller-7c697dc6f4-69b5g\" (UID: \"56c86278-94b0-48eb-85c6-4a99632012ca\") " pod="opendatahub/maas-controller-7c697dc6f4-69b5g" Jul 23 22:01:14.532273 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:01:14.532252 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gb2wz\" (UniqueName: \"kubernetes.io/projected/56c86278-94b0-48eb-85c6-4a99632012ca-kube-api-access-gb2wz\") pod \"maas-controller-7c697dc6f4-69b5g\" (UID: \"56c86278-94b0-48eb-85c6-4a99632012ca\") " pod="opendatahub/maas-controller-7c697dc6f4-69b5g" Jul 23 22:01:14.605210 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:01:14.605183 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7c697dc6f4-69b5g" Jul 23 22:01:14.733682 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:01:14.733658 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7c697dc6f4-69b5g"] Jul 23 22:01:14.735690 ip-10-0-136-157 kubenswrapper[2578]: W0723 22:01:14.735660 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56c86278_94b0_48eb_85c6_4a99632012ca.slice/crio-cf8637aaec642b77753c10a99d14fb10d0cac17c76617fb30bd26126f25597de WatchSource:0}: Error finding container cf8637aaec642b77753c10a99d14fb10d0cac17c76617fb30bd26126f25597de: Status 404 returned error can't find the container with id cf8637aaec642b77753c10a99d14fb10d0cac17c76617fb30bd26126f25597de Jul 23 22:01:14.783098 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:01:14.783069 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7c697dc6f4-69b5g" event={"ID":"56c86278-94b0-48eb-85c6-4a99632012ca","Type":"ContainerStarted","Data":"cf8637aaec642b77753c10a99d14fb10d0cac17c76617fb30bd26126f25597de"} Jul 23 22:01:14.826381 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:01:14.826355 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b957e99c-4c6c-4c66-8111-e8420e04bf00-webhook-cert\") pod \"maas-controller-6f9d846d69-nvw5h\" (UID: \"b957e99c-4c6c-4c66-8111-e8420e04bf00\") " pod="opendatahub/maas-controller-6f9d846d69-nvw5h" Jul 23 22:01:14.828799 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:01:14.828773 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b957e99c-4c6c-4c66-8111-e8420e04bf00-webhook-cert\") pod \"maas-controller-6f9d846d69-nvw5h\" (UID: \"b957e99c-4c6c-4c66-8111-e8420e04bf00\") " pod="opendatahub/maas-controller-6f9d846d69-nvw5h" Jul 23 22:01:15.012296 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:01:15.012218 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6f9d846d69-nvw5h" Jul 23 22:01:15.140035 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:01:15.140005 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6f9d846d69-nvw5h"] Jul 23 22:01:15.141140 ip-10-0-136-157 kubenswrapper[2578]: W0723 22:01:15.141114 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb957e99c_4c6c_4c66_8111_e8420e04bf00.slice/crio-a28b5ae9261fd95fc5f9cc3a29921c7968ea03a2fcb60e7d0f7d3aa99cebaf2d WatchSource:0}: Error finding container a28b5ae9261fd95fc5f9cc3a29921c7968ea03a2fcb60e7d0f7d3aa99cebaf2d: Status 404 returned error can't find the container with id a28b5ae9261fd95fc5f9cc3a29921c7968ea03a2fcb60e7d0f7d3aa99cebaf2d Jul 23 22:01:15.795512 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:01:15.795461 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6f9d846d69-nvw5h" event={"ID":"b957e99c-4c6c-4c66-8111-e8420e04bf00","Type":"ContainerStarted","Data":"a28b5ae9261fd95fc5f9cc3a29921c7968ea03a2fcb60e7d0f7d3aa99cebaf2d"} Jul 23 22:01:17.808583 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:01:17.808520 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7c697dc6f4-69b5g" event={"ID":"56c86278-94b0-48eb-85c6-4a99632012ca","Type":"ContainerStarted","Data":"e474c4612c9d88daa860259a0fa79bef3def633b49d90d0d11930dda7140d21f"} Jul 23 22:01:17.809034 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:01:17.808618 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7c697dc6f4-69b5g" Jul 23 22:01:17.810110 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:01:17.810089 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6f9d846d69-nvw5h" event={"ID":"b957e99c-4c6c-4c66-8111-e8420e04bf00","Type":"ContainerStarted","Data":"3b1329a205e6bd800f65e2438b7f3c64093152552456e31566075f6b7439d2ee"} Jul 23 22:01:17.810207 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:01:17.810129 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6f9d846d69-nvw5h" Jul 23 22:01:17.831487 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:01:17.831435 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7c697dc6f4-69b5g" podStartSLOduration=1.431588864 podStartE2EDuration="3.831420392s" totalImagesPullingTime="2.399831528s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 22:01:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 22:01:17.829083815 +0000 UTC m=+580.070991690" watchObservedRunningTime="2026-07-23 22:01:17.831420392 +0000 UTC m=+580.073328263" Jul 23 22:01:17.850948 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:01:17.850881 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6f9d846d69-nvw5h" podStartSLOduration=1.849610078 podStartE2EDuration="3.850858797s" totalImagesPullingTime="2.001248719s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 22:01:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 22:01:17.848532247 +0000 UTC m=+580.090440122" watchObservedRunningTime="2026-07-23 22:01:17.850858797 +0000 UTC m=+580.092766672" Jul 23 22:01:28.816841 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:01:28.816808 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7c697dc6f4-69b5g" Jul 23 22:01:28.817226 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:01:28.816888 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6f9d846d69-nvw5h" Jul 23 22:01:28.881388 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:01:28.881351 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6f9d846d69-nvw5h"] Jul 23 22:01:28.881748 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:01:28.881714 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-6f9d846d69-nvw5h" podUID="b957e99c-4c6c-4c66-8111-e8420e04bf00" containerName="manager" containerID="cri-o://3b1329a205e6bd800f65e2438b7f3c64093152552456e31566075f6b7439d2ee" gracePeriod=30 Jul 23 22:01:29.123886 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:01:29.123859 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6f9d846d69-nvw5h" Jul 23 22:01:29.260388 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:01:29.260357 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-947mw\" (UniqueName: \"kubernetes.io/projected/b957e99c-4c6c-4c66-8111-e8420e04bf00-kube-api-access-947mw\") pod \"b957e99c-4c6c-4c66-8111-e8420e04bf00\" (UID: \"b957e99c-4c6c-4c66-8111-e8420e04bf00\") " Jul 23 22:01:29.260585 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:01:29.260440 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b957e99c-4c6c-4c66-8111-e8420e04bf00-webhook-cert\") pod \"b957e99c-4c6c-4c66-8111-e8420e04bf00\" (UID: \"b957e99c-4c6c-4c66-8111-e8420e04bf00\") " Jul 23 22:01:29.262732 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:01:29.262701 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b957e99c-4c6c-4c66-8111-e8420e04bf00-kube-api-access-947mw" (OuterVolumeSpecName: "kube-api-access-947mw") pod "b957e99c-4c6c-4c66-8111-e8420e04bf00" (UID: "b957e99c-4c6c-4c66-8111-e8420e04bf00"). InnerVolumeSpecName "kube-api-access-947mw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 22:01:29.262821 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:01:29.262751 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b957e99c-4c6c-4c66-8111-e8420e04bf00-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "b957e99c-4c6c-4c66-8111-e8420e04bf00" (UID: "b957e99c-4c6c-4c66-8111-e8420e04bf00"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 22:01:29.361604 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:01:29.361494 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-947mw\" (UniqueName: \"kubernetes.io/projected/b957e99c-4c6c-4c66-8111-e8420e04bf00-kube-api-access-947mw\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\"" Jul 23 22:01:29.361604 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:01:29.361523 2578 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b957e99c-4c6c-4c66-8111-e8420e04bf00-webhook-cert\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\"" Jul 23 22:01:29.867273 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:01:29.867234 2578 generic.go:356] "Generic (PLEG): container finished" podID="b957e99c-4c6c-4c66-8111-e8420e04bf00" containerID="3b1329a205e6bd800f65e2438b7f3c64093152552456e31566075f6b7439d2ee" exitCode=0 Jul 23 22:01:29.867714 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:01:29.867281 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6f9d846d69-nvw5h" Jul 23 22:01:29.867714 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:01:29.867277 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6f9d846d69-nvw5h" event={"ID":"b957e99c-4c6c-4c66-8111-e8420e04bf00","Type":"ContainerDied","Data":"3b1329a205e6bd800f65e2438b7f3c64093152552456e31566075f6b7439d2ee"} Jul 23 22:01:29.867714 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:01:29.867331 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6f9d846d69-nvw5h" event={"ID":"b957e99c-4c6c-4c66-8111-e8420e04bf00","Type":"ContainerDied","Data":"a28b5ae9261fd95fc5f9cc3a29921c7968ea03a2fcb60e7d0f7d3aa99cebaf2d"} Jul 23 22:01:29.867714 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:01:29.867353 2578 scope.go:117] "RemoveContainer" containerID="3b1329a205e6bd800f65e2438b7f3c64093152552456e31566075f6b7439d2ee" Jul 23 22:01:29.876798 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:01:29.876782 2578 scope.go:117] "RemoveContainer" containerID="3b1329a205e6bd800f65e2438b7f3c64093152552456e31566075f6b7439d2ee" Jul 23 22:01:29.877065 ip-10-0-136-157 kubenswrapper[2578]: E0723 22:01:29.877043 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3b1329a205e6bd800f65e2438b7f3c64093152552456e31566075f6b7439d2ee\": container with ID starting with 3b1329a205e6bd800f65e2438b7f3c64093152552456e31566075f6b7439d2ee not found: ID does not exist" containerID="3b1329a205e6bd800f65e2438b7f3c64093152552456e31566075f6b7439d2ee" Jul 23 22:01:29.877109 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:01:29.877073 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3b1329a205e6bd800f65e2438b7f3c64093152552456e31566075f6b7439d2ee"} err="failed to get container status \"3b1329a205e6bd800f65e2438b7f3c64093152552456e31566075f6b7439d2ee\": rpc error: code = NotFound desc = could not find container \"3b1329a205e6bd800f65e2438b7f3c64093152552456e31566075f6b7439d2ee\": container with ID starting with 3b1329a205e6bd800f65e2438b7f3c64093152552456e31566075f6b7439d2ee not found: ID does not exist" Jul 23 22:01:29.888256 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:01:29.888233 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6f9d846d69-nvw5h"] Jul 23 22:01:29.891668 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:01:29.891646 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6f9d846d69-nvw5h"] Jul 23 22:01:30.250895 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:01:30.250820 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b957e99c-4c6c-4c66-8111-e8420e04bf00" path="/var/lib/kubelet/pods/b957e99c-4c6c-4c66-8111-e8420e04bf00/volumes" Jul 23 22:01:38.528709 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:01:38.528532 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7jkjm_7342a9f6-72b1-4e3c-8d50-0b0326454a86/ovn-acl-logging/0.log" Jul 23 22:01:38.552450 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:01:38.529219 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7jkjm_7342a9f6-72b1-4e3c-8d50-0b0326454a86/ovn-acl-logging/0.log" Jul 23 22:01:50.875390 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:01:50.875354 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-6mbp8"] Jul 23 22:01:50.875889 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:01:50.875704 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b957e99c-4c6c-4c66-8111-e8420e04bf00" containerName="manager" Jul 23 22:01:50.875889 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:01:50.875715 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="b957e99c-4c6c-4c66-8111-e8420e04bf00" containerName="manager" Jul 23 22:01:50.875889 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:01:50.875769 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="b957e99c-4c6c-4c66-8111-e8420e04bf00" containerName="manager" Jul 23 22:01:50.887389 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:01:50.887360 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7677b774bb-6mbp8" Jul 23 22:01:50.893405 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:01:50.890949 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 23 22:01:50.893405 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:01:50.891440 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-gm464\"" Jul 23 22:01:50.897136 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:01:50.897097 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-6mbp8"] Jul 23 22:01:50.928973 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:01:50.928936 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ec20a4df-f4f0-4817-b844-7f2134c331dc-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-6mbp8\" (UID: \"ec20a4df-f4f0-4817-b844-7f2134c331dc\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-6mbp8" Jul 23 22:01:50.929139 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:01:50.928986 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q77wh\" (UniqueName: \"kubernetes.io/projected/ec20a4df-f4f0-4817-b844-7f2134c331dc-kube-api-access-q77wh\") pod \"payload-pre-processing-7677b774bb-6mbp8\" (UID: \"ec20a4df-f4f0-4817-b844-7f2134c331dc\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-6mbp8" Jul 23 22:01:51.029755 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:01:51.029717 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ec20a4df-f4f0-4817-b844-7f2134c331dc-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-6mbp8\" (UID: \"ec20a4df-f4f0-4817-b844-7f2134c331dc\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-6mbp8" Jul 23 22:01:51.029755 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:01:51.029764 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q77wh\" (UniqueName: \"kubernetes.io/projected/ec20a4df-f4f0-4817-b844-7f2134c331dc-kube-api-access-q77wh\") pod \"payload-pre-processing-7677b774bb-6mbp8\" (UID: \"ec20a4df-f4f0-4817-b844-7f2134c331dc\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-6mbp8" Jul 23 22:01:51.030417 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:01:51.030393 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ec20a4df-f4f0-4817-b844-7f2134c331dc-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-6mbp8\" (UID: \"ec20a4df-f4f0-4817-b844-7f2134c331dc\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-6mbp8" Jul 23 22:01:51.038382 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:01:51.038348 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q77wh\" (UniqueName: \"kubernetes.io/projected/ec20a4df-f4f0-4817-b844-7f2134c331dc-kube-api-access-q77wh\") pod \"payload-pre-processing-7677b774bb-6mbp8\" (UID: \"ec20a4df-f4f0-4817-b844-7f2134c331dc\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-6mbp8" Jul 23 22:01:51.205903 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:01:51.205811 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7677b774bb-6mbp8" Jul 23 22:01:51.345044 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:01:51.345015 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-6mbp8"] Jul 23 22:01:51.346867 ip-10-0-136-157 kubenswrapper[2578]: W0723 22:01:51.346827 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec20a4df_f4f0_4817_b844_7f2134c331dc.slice/crio-cb969dd7751fcfe623c7c2aa7b3e33519eeacf301978a385ffe48e367f713c1d WatchSource:0}: Error finding container cb969dd7751fcfe623c7c2aa7b3e33519eeacf301978a385ffe48e367f713c1d: Status 404 returned error can't find the container with id cb969dd7751fcfe623c7c2aa7b3e33519eeacf301978a385ffe48e367f713c1d Jul 23 22:01:51.348960 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:01:51.348944 2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 23 22:01:51.977584 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:01:51.977474 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7677b774bb-6mbp8" event={"ID":"ec20a4df-f4f0-4817-b844-7f2134c331dc","Type":"ContainerStarted","Data":"cb969dd7751fcfe623c7c2aa7b3e33519eeacf301978a385ffe48e367f713c1d"} Jul 23 22:01:53.989125 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:01:53.989091 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7677b774bb-6mbp8" event={"ID":"ec20a4df-f4f0-4817-b844-7f2134c331dc","Type":"ContainerStarted","Data":"4d2d8e57216a140ca1c77cdd813425b08ea68fb2b02e42cd5ef3c870ade2ba50"} Jul 23 22:01:53.989561 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:01:53.989149 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7677b774bb-6mbp8" Jul 23 22:01:54.010531 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:01:54.010484 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7677b774bb-6mbp8" podStartSLOduration=2.085271967 podStartE2EDuration="4.01046894s" totalImagesPullingTime="1.925196973s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 22:01:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 22:01:54.008896447 +0000 UTC m=+616.250804321" watchObservedRunningTime="2026-07-23 22:01:54.01046894 +0000 UTC m=+616.252376815" Jul 23 22:01:59.019589 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:01:59.019521 2578 generic.go:356] "Generic (PLEG): container finished" podID="9872c26d-b195-44ba-ba89-05aae0c7ff98" containerID="771ab5d9bf2d4a2db5ee69f5a89376f30742fc05805f90befeaadd0b51c44185" exitCode=0 Jul 23 22:01:59.020031 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:01:59.019601 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jnrd4" event={"ID":"9872c26d-b195-44ba-ba89-05aae0c7ff98","Type":"ContainerDied","Data":"771ab5d9bf2d4a2db5ee69f5a89376f30742fc05805f90befeaadd0b51c44185"} Jul 23 22:01:59.020031 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:01:59.020017 2578 scope.go:117] "RemoveContainer" containerID="771ab5d9bf2d4a2db5ee69f5a89376f30742fc05805f90befeaadd0b51c44185" Jul 23 22:01:59.021039 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:01:59.021011 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 23 22:01:59.021143 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:01:59.021094 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 23 22:01:59.021199 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:01:59.021146 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 23 22:02:00.026366 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:02:00.026336 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jnrd4" event={"ID":"9872c26d-b195-44ba-ba89-05aae0c7ff98","Type":"ContainerStarted","Data":"c4a02e10f4dd324b1b127b2e2a4c78ca5d071f93ab896fb7c170aba70efd67da"} Jul 23 22:02:00.681690 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:02:00.681653 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jnrd4" Jul 23 22:02:00.681923 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:02:00.681902 2578 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-jnrd4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" start-of-body= Jul 23 22:02:00.682021 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:02:00.681958 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jnrd4" podUID="9872c26d-b195-44ba-ba89-05aae0c7ff98" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" Jul 23 22:02:01.681760 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:02:01.681719 2578 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-jnrd4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" start-of-body= Jul 23 22:02:01.682166 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:02:01.681810 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jnrd4" podUID="9872c26d-b195-44ba-ba89-05aae0c7ff98" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" Jul 23 22:02:02.681832 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:02:02.681796 2578 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-jnrd4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" start-of-body= Jul 23 22:02:02.682225 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:02:02.681862 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jnrd4" podUID="9872c26d-b195-44ba-ba89-05aae0c7ff98" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" Jul 23 22:02:03.685445 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:02:03.685417 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jnrd4" Jul 23 22:02:03.685822 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:02:03.685803 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jnrd4" Jul 23 22:02:03.686518 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:02:03.686498 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-jnrd4" Jul 23 22:02:04.996134 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:02:04.996106 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7677b774bb-6mbp8" Jul 23 22:02:36.102711 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:02:36.102680 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-2sjvq"] Jul 23 22:02:36.107025 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:02:36.107003 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2sjvq" Jul 23 22:02:36.110898 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:02:36.110874 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 23 22:02:36.111150 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:02:36.111133 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 23 22:02:36.111272 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:02:36.111157 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-skzdc\"" Jul 23 22:02:36.111337 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:02:36.111313 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 23 22:02:36.118079 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:02:36.118057 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-2sjvq"] Jul 23 22:02:36.135426 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:02:36.135403 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2173d13e-134c-41da-b2cd-1fea1e82e754-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2sjvq\" (UID: \"2173d13e-134c-41da-b2cd-1fea1e82e754\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2sjvq" Jul 23 22:02:36.135532 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:02:36.135446 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2173d13e-134c-41da-b2cd-1fea1e82e754-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2sjvq\" (UID: \"2173d13e-134c-41da-b2cd-1fea1e82e754\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2sjvq" Jul 23 22:02:36.135532 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:02:36.135464 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2173d13e-134c-41da-b2cd-1fea1e82e754-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2sjvq\" (UID: \"2173d13e-134c-41da-b2cd-1fea1e82e754\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2sjvq" Jul 23 22:02:36.135634 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:02:36.135596 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-brj59\" (UniqueName: \"kubernetes.io/projected/2173d13e-134c-41da-b2cd-1fea1e82e754-kube-api-access-brj59\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2sjvq\" (UID: \"2173d13e-134c-41da-b2cd-1fea1e82e754\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2sjvq" Jul 23 22:02:36.135667 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:02:36.135631 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2173d13e-134c-41da-b2cd-1fea1e82e754-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2sjvq\" (UID: \"2173d13e-134c-41da-b2cd-1fea1e82e754\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2sjvq" Jul 23 22:02:36.135667 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:02:36.135655 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2173d13e-134c-41da-b2cd-1fea1e82e754-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2sjvq\" (UID: \"2173d13e-134c-41da-b2cd-1fea1e82e754\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2sjvq" Jul 23 22:02:36.236752 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:02:36.236722 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2173d13e-134c-41da-b2cd-1fea1e82e754-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2sjvq\" (UID: \"2173d13e-134c-41da-b2cd-1fea1e82e754\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2sjvq" Jul 23 22:02:36.236924 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:02:36.236759 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2173d13e-134c-41da-b2cd-1fea1e82e754-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2sjvq\" (UID: \"2173d13e-134c-41da-b2cd-1fea1e82e754\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2sjvq" Jul 23 22:02:36.236924 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:02:36.236796 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-brj59\" (UniqueName: \"kubernetes.io/projected/2173d13e-134c-41da-b2cd-1fea1e82e754-kube-api-access-brj59\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2sjvq\" (UID: \"2173d13e-134c-41da-b2cd-1fea1e82e754\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2sjvq" Jul 23 22:02:36.236924 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:02:36.236816 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2173d13e-134c-41da-b2cd-1fea1e82e754-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2sjvq\" (UID: \"2173d13e-134c-41da-b2cd-1fea1e82e754\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2sjvq" Jul 23 22:02:36.236924 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:02:36.236835 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2173d13e-134c-41da-b2cd-1fea1e82e754-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2sjvq\" (UID: \"2173d13e-134c-41da-b2cd-1fea1e82e754\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2sjvq" Jul 23 22:02:36.236924 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:02:36.236866 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2173d13e-134c-41da-b2cd-1fea1e82e754-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2sjvq\" (UID: \"2173d13e-134c-41da-b2cd-1fea1e82e754\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2sjvq" Jul 23 22:02:36.237278 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:02:36.237250 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2173d13e-134c-41da-b2cd-1fea1e82e754-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2sjvq\" (UID: \"2173d13e-134c-41da-b2cd-1fea1e82e754\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2sjvq" Jul 23 22:02:36.237391 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:02:36.237280 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2173d13e-134c-41da-b2cd-1fea1e82e754-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2sjvq\" (UID: \"2173d13e-134c-41da-b2cd-1fea1e82e754\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2sjvq" Jul 23 22:02:36.237391 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:02:36.237290 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2173d13e-134c-41da-b2cd-1fea1e82e754-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2sjvq\" (UID: \"2173d13e-134c-41da-b2cd-1fea1e82e754\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2sjvq" Jul 23 22:02:36.239124 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:02:36.239102 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2173d13e-134c-41da-b2cd-1fea1e82e754-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2sjvq\" (UID: \"2173d13e-134c-41da-b2cd-1fea1e82e754\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2sjvq" Jul 23 22:02:36.239282 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:02:36.239265 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2173d13e-134c-41da-b2cd-1fea1e82e754-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2sjvq\" (UID: \"2173d13e-134c-41da-b2cd-1fea1e82e754\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2sjvq" Jul 23 22:02:36.246584 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:02:36.246566 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-brj59\" (UniqueName: \"kubernetes.io/projected/2173d13e-134c-41da-b2cd-1fea1e82e754-kube-api-access-brj59\") pod \"e2e-trlp-test-simulated-kserve-598466548d-2sjvq\" (UID: \"2173d13e-134c-41da-b2cd-1fea1e82e754\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2sjvq" Jul 23 22:02:36.420072 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:02:36.419991 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2sjvq" Jul 23 22:02:36.552073 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:02:36.552042 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-2sjvq"] Jul 23 22:02:36.553765 ip-10-0-136-157 kubenswrapper[2578]: W0723 22:02:36.553737 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2173d13e_134c_41da_b2cd_1fea1e82e754.slice/crio-1b5dbda09a69b017d2b4f3d158f4aa1e9b83786bedee27ed2eff1aff05668fd9 WatchSource:0}: Error finding container 1b5dbda09a69b017d2b4f3d158f4aa1e9b83786bedee27ed2eff1aff05668fd9: Status 404 returned error can't find the container with id 1b5dbda09a69b017d2b4f3d158f4aa1e9b83786bedee27ed2eff1aff05668fd9 Jul 23 22:02:37.218431 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:02:37.218389 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2sjvq" event={"ID":"2173d13e-134c-41da-b2cd-1fea1e82e754","Type":"ContainerStarted","Data":"1b5dbda09a69b017d2b4f3d158f4aa1e9b83786bedee27ed2eff1aff05668fd9"} Jul 23 22:02:38.224009 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:02:38.223979 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2sjvq" event={"ID":"2173d13e-134c-41da-b2cd-1fea1e82e754","Type":"ContainerStarted","Data":"cfae668b893863c87b0591e7eb710c9f2a8993f668d652f2f2d505c595f32367"} Jul 23 22:02:38.244683 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:02:38.244628 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2sjvq" podStartSLOduration=1.222959994 podStartE2EDuration="2.244614599s" totalImagesPullingTime="1.021654605s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 22:02:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 22:02:38.24248397 +0000 UTC m=+660.484391843" watchObservedRunningTime="2026-07-23 22:02:38.244614599 +0000 UTC m=+660.486522472" Jul 23 22:02:46.420635 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:02:46.420606 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2sjvq" Jul 23 22:02:46.421066 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:02:46.420663 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2sjvq" Jul 23 22:02:46.432891 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:02:46.432870 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2sjvq" Jul 23 22:02:47.275698 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:02:47.275667 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-2sjvq" Jul 23 22:02:54.614178 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:02:54.614144 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-7q2mv"] Jul 23 22:02:54.616699 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:02:54.616678 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7q2mv" Jul 23 22:02:54.619362 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:02:54.619344 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 23 22:02:54.627128 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:02:54.627105 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-7q2mv"] Jul 23 22:02:54.689751 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:02:54.689725 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0ca6b8e9-c6c5-4d38-a8e8-98a73b38c977-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-7q2mv\" (UID: \"0ca6b8e9-c6c5-4d38-a8e8-98a73b38c977\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7q2mv" Jul 23 22:02:54.689751 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:02:54.689751 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wsrcc\" (UniqueName: \"kubernetes.io/projected/0ca6b8e9-c6c5-4d38-a8e8-98a73b38c977-kube-api-access-wsrcc\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-7q2mv\" (UID: \"0ca6b8e9-c6c5-4d38-a8e8-98a73b38c977\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7q2mv" Jul 23 22:02:54.689908 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:02:54.689793 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0ca6b8e9-c6c5-4d38-a8e8-98a73b38c977-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-7q2mv\" (UID: \"0ca6b8e9-c6c5-4d38-a8e8-98a73b38c977\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7q2mv" Jul 23 22:02:54.689953 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:02:54.689927 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0ca6b8e9-c6c5-4d38-a8e8-98a73b38c977-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-7q2mv\" (UID: \"0ca6b8e9-c6c5-4d38-a8e8-98a73b38c977\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7q2mv" Jul 23 22:02:54.689996 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:02:54.689966 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0ca6b8e9-c6c5-4d38-a8e8-98a73b38c977-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-7q2mv\" (UID: \"0ca6b8e9-c6c5-4d38-a8e8-98a73b38c977\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7q2mv" Jul 23 22:02:54.689996 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:02:54.689987 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0ca6b8e9-c6c5-4d38-a8e8-98a73b38c977-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-7q2mv\" (UID: \"0ca6b8e9-c6c5-4d38-a8e8-98a73b38c977\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7q2mv" Jul 23 22:02:54.790529 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:02:54.790502 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0ca6b8e9-c6c5-4d38-a8e8-98a73b38c977-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-7q2mv\" (UID: \"0ca6b8e9-c6c5-4d38-a8e8-98a73b38c977\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7q2mv" Jul 23 22:02:54.790687 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:02:54.790567 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0ca6b8e9-c6c5-4d38-a8e8-98a73b38c977-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-7q2mv\" (UID: \"0ca6b8e9-c6c5-4d38-a8e8-98a73b38c977\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7q2mv" Jul 23 22:02:54.790687 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:02:54.790595 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0ca6b8e9-c6c5-4d38-a8e8-98a73b38c977-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-7q2mv\" (UID: \"0ca6b8e9-c6c5-4d38-a8e8-98a73b38c977\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7q2mv" Jul 23 22:02:54.790687 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:02:54.790633 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0ca6b8e9-c6c5-4d38-a8e8-98a73b38c977-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-7q2mv\" (UID: \"0ca6b8e9-c6c5-4d38-a8e8-98a73b38c977\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7q2mv" Jul 23 22:02:54.790687 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:02:54.790679 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wsrcc\" (UniqueName: \"kubernetes.io/projected/0ca6b8e9-c6c5-4d38-a8e8-98a73b38c977-kube-api-access-wsrcc\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-7q2mv\" (UID: \"0ca6b8e9-c6c5-4d38-a8e8-98a73b38c977\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7q2mv" Jul 23 22:02:54.790924 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:02:54.790799 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0ca6b8e9-c6c5-4d38-a8e8-98a73b38c977-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-7q2mv\" (UID: \"0ca6b8e9-c6c5-4d38-a8e8-98a73b38c977\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7q2mv" Jul 23 22:02:54.791012 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:02:54.790988 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0ca6b8e9-c6c5-4d38-a8e8-98a73b38c977-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-7q2mv\" (UID: \"0ca6b8e9-c6c5-4d38-a8e8-98a73b38c977\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7q2mv" Jul 23 22:02:54.791178 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:02:54.791138 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0ca6b8e9-c6c5-4d38-a8e8-98a73b38c977-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-7q2mv\" (UID: \"0ca6b8e9-c6c5-4d38-a8e8-98a73b38c977\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7q2mv" Jul 23 22:02:54.791396 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:02:54.791378 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0ca6b8e9-c6c5-4d38-a8e8-98a73b38c977-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-7q2mv\" (UID: \"0ca6b8e9-c6c5-4d38-a8e8-98a73b38c977\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7q2mv" Jul 23 22:02:54.793108 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:02:54.793083 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0ca6b8e9-c6c5-4d38-a8e8-98a73b38c977-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-7q2mv\" (UID: \"0ca6b8e9-c6c5-4d38-a8e8-98a73b38c977\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7q2mv" Jul 23 22:02:54.793860 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:02:54.793834 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0ca6b8e9-c6c5-4d38-a8e8-98a73b38c977-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-7q2mv\" (UID: \"0ca6b8e9-c6c5-4d38-a8e8-98a73b38c977\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7q2mv" Jul 23 22:02:54.799798 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:02:54.799772 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wsrcc\" (UniqueName: \"kubernetes.io/projected/0ca6b8e9-c6c5-4d38-a8e8-98a73b38c977-kube-api-access-wsrcc\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-7q2mv\" (UID: \"0ca6b8e9-c6c5-4d38-a8e8-98a73b38c977\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7q2mv" Jul 23 22:02:54.927368 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:02:54.927302 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7q2mv" Jul 23 22:02:55.056719 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:02:55.056694 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-7q2mv"] Jul 23 22:02:55.057844 ip-10-0-136-157 kubenswrapper[2578]: W0723 22:02:55.057820 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ca6b8e9_c6c5_4d38_a8e8_98a73b38c977.slice/crio-bb5ce1ea4848ce0de3b58eed43aa67801fb019b25bc32326a7c24cd78614377d WatchSource:0}: Error finding container bb5ce1ea4848ce0de3b58eed43aa67801fb019b25bc32326a7c24cd78614377d: Status 404 returned error can't find the container with id bb5ce1ea4848ce0de3b58eed43aa67801fb019b25bc32326a7c24cd78614377d Jul 23 22:02:55.303647 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:02:55.303619 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7q2mv" event={"ID":"0ca6b8e9-c6c5-4d38-a8e8-98a73b38c977","Type":"ContainerStarted","Data":"bb5ce1ea4848ce0de3b58eed43aa67801fb019b25bc32326a7c24cd78614377d"} Jul 23 22:02:56.309064 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:02:56.309030 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7q2mv" event={"ID":"0ca6b8e9-c6c5-4d38-a8e8-98a73b38c977","Type":"ContainerStarted","Data":"aa585bd3ce9e5b41d78815c1ea6a4a93155738a656d19e052f7194b78e432859"} Jul 23 22:02:56.331930 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:02:56.331879 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7q2mv" podStartSLOduration=2.108189873 podStartE2EDuration="2.331866092s" totalImagesPullingTime="223.676219ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 22:02:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 22:02:56.329323683 +0000 UTC m=+678.571231558" watchObservedRunningTime="2026-07-23 22:02:56.331866092 +0000 UTC m=+678.573773965" Jul 23 22:03:04.928114 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:03:04.928080 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7q2mv" Jul 23 22:03:04.928607 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:03:04.928154 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7q2mv" Jul 23 22:03:04.940164 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:03:04.940143 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7q2mv" Jul 23 22:03:05.361667 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:03:05.361641 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7q2mv" Jul 23 22:04:54.082657 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:04:54.082616 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7c697dc6f4-69b5g"] Jul 23 22:04:54.083239 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:04:54.082884 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-7c697dc6f4-69b5g" podUID="56c86278-94b0-48eb-85c6-4a99632012ca" containerName="manager" containerID="cri-o://e474c4612c9d88daa860259a0fa79bef3def633b49d90d0d11930dda7140d21f" gracePeriod=30 Jul 23 22:04:54.320050 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:04:54.320026 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7c697dc6f4-69b5g" Jul 23 22:04:54.489706 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:04:54.489642 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gb2wz\" (UniqueName: \"kubernetes.io/projected/56c86278-94b0-48eb-85c6-4a99632012ca-kube-api-access-gb2wz\") pod \"56c86278-94b0-48eb-85c6-4a99632012ca\" (UID: \"56c86278-94b0-48eb-85c6-4a99632012ca\") " Jul 23 22:04:54.489706 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:04:54.489686 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/56c86278-94b0-48eb-85c6-4a99632012ca-webhook-cert\") pod \"56c86278-94b0-48eb-85c6-4a99632012ca\" (UID: \"56c86278-94b0-48eb-85c6-4a99632012ca\") " Jul 23 22:04:54.491814 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:04:54.491784 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/56c86278-94b0-48eb-85c6-4a99632012ca-kube-api-access-gb2wz" (OuterVolumeSpecName: "kube-api-access-gb2wz") pod "56c86278-94b0-48eb-85c6-4a99632012ca" (UID: "56c86278-94b0-48eb-85c6-4a99632012ca"). InnerVolumeSpecName "kube-api-access-gb2wz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 22:04:54.491916 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:04:54.491821 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/56c86278-94b0-48eb-85c6-4a99632012ca-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "56c86278-94b0-48eb-85c6-4a99632012ca" (UID: "56c86278-94b0-48eb-85c6-4a99632012ca"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 22:04:54.591106 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:04:54.591074 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gb2wz\" (UniqueName: \"kubernetes.io/projected/56c86278-94b0-48eb-85c6-4a99632012ca-kube-api-access-gb2wz\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\"" Jul 23 22:04:54.591106 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:04:54.591103 2578 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/56c86278-94b0-48eb-85c6-4a99632012ca-webhook-cert\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\"" Jul 23 22:04:54.858032 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:04:54.858001 2578 generic.go:356] "Generic (PLEG): container finished" podID="56c86278-94b0-48eb-85c6-4a99632012ca" containerID="e474c4612c9d88daa860259a0fa79bef3def633b49d90d0d11930dda7140d21f" exitCode=0 Jul 23 22:04:54.858123 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:04:54.858048 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7c697dc6f4-69b5g" Jul 23 22:04:54.858123 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:04:54.858088 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7c697dc6f4-69b5g" event={"ID":"56c86278-94b0-48eb-85c6-4a99632012ca","Type":"ContainerDied","Data":"e474c4612c9d88daa860259a0fa79bef3def633b49d90d0d11930dda7140d21f"} Jul 23 22:04:54.858197 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:04:54.858139 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7c697dc6f4-69b5g" event={"ID":"56c86278-94b0-48eb-85c6-4a99632012ca","Type":"ContainerDied","Data":"cf8637aaec642b77753c10a99d14fb10d0cac17c76617fb30bd26126f25597de"} Jul 23 22:04:54.858197 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:04:54.858162 2578 scope.go:117] "RemoveContainer" containerID="e474c4612c9d88daa860259a0fa79bef3def633b49d90d0d11930dda7140d21f" Jul 23 22:04:54.866810 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:04:54.866793 2578 scope.go:117] "RemoveContainer" containerID="e474c4612c9d88daa860259a0fa79bef3def633b49d90d0d11930dda7140d21f" Jul 23 22:04:54.867054 ip-10-0-136-157 kubenswrapper[2578]: E0723 22:04:54.867037 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e474c4612c9d88daa860259a0fa79bef3def633b49d90d0d11930dda7140d21f\": container with ID starting with e474c4612c9d88daa860259a0fa79bef3def633b49d90d0d11930dda7140d21f not found: ID does not exist" containerID="e474c4612c9d88daa860259a0fa79bef3def633b49d90d0d11930dda7140d21f" Jul 23 22:04:54.867111 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:04:54.867062 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e474c4612c9d88daa860259a0fa79bef3def633b49d90d0d11930dda7140d21f"} err="failed to get container status \"e474c4612c9d88daa860259a0fa79bef3def633b49d90d0d11930dda7140d21f\": rpc error: code = NotFound desc = could not find container \"e474c4612c9d88daa860259a0fa79bef3def633b49d90d0d11930dda7140d21f\": container with ID starting with e474c4612c9d88daa860259a0fa79bef3def633b49d90d0d11930dda7140d21f not found: ID does not exist" Jul 23 22:04:54.879099 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:04:54.879078 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7c697dc6f4-69b5g"] Jul 23 22:04:54.882686 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:04:54.882658 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7c697dc6f4-69b5g"] Jul 23 22:04:56.250761 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:04:56.250729 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="56c86278-94b0-48eb-85c6-4a99632012ca" path="/var/lib/kubelet/pods/56c86278-94b0-48eb-85c6-4a99632012ca/volumes" Jul 23 22:05:55.753004 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:05:55.752971 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7c697dc6f4-sm2xc"] Jul 23 22:05:55.753450 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:05:55.753300 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="56c86278-94b0-48eb-85c6-4a99632012ca" containerName="manager" Jul 23 22:05:55.753450 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:05:55.753310 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="56c86278-94b0-48eb-85c6-4a99632012ca" containerName="manager" Jul 23 22:05:55.753450 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:05:55.753369 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="56c86278-94b0-48eb-85c6-4a99632012ca" containerName="manager" Jul 23 22:05:55.757552 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:05:55.757525 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7c697dc6f4-sm2xc" Jul 23 22:05:55.760861 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:05:55.760834 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 23 22:05:55.760861 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:05:55.760854 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 23 22:05:55.761037 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:05:55.760874 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-rlp68\"" Jul 23 22:05:55.766282 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:05:55.766262 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7c697dc6f4-sm2xc"] Jul 23 22:05:55.858676 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:05:55.858651 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f3cf2417-a6d7-41ca-928c-4874ea154f3a-webhook-cert\") pod \"maas-controller-7c697dc6f4-sm2xc\" (UID: \"f3cf2417-a6d7-41ca-928c-4874ea154f3a\") " pod="opendatahub/maas-controller-7c697dc6f4-sm2xc" Jul 23 22:05:55.858772 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:05:55.858718 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9dj6x\" (UniqueName: \"kubernetes.io/projected/f3cf2417-a6d7-41ca-928c-4874ea154f3a-kube-api-access-9dj6x\") pod \"maas-controller-7c697dc6f4-sm2xc\" (UID: \"f3cf2417-a6d7-41ca-928c-4874ea154f3a\") " pod="opendatahub/maas-controller-7c697dc6f4-sm2xc" Jul 23 22:05:55.959687 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:05:55.959662 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9dj6x\" (UniqueName: \"kubernetes.io/projected/f3cf2417-a6d7-41ca-928c-4874ea154f3a-kube-api-access-9dj6x\") pod \"maas-controller-7c697dc6f4-sm2xc\" (UID: \"f3cf2417-a6d7-41ca-928c-4874ea154f3a\") " pod="opendatahub/maas-controller-7c697dc6f4-sm2xc" Jul 23 22:05:55.959785 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:05:55.959729 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f3cf2417-a6d7-41ca-928c-4874ea154f3a-webhook-cert\") pod \"maas-controller-7c697dc6f4-sm2xc\" (UID: \"f3cf2417-a6d7-41ca-928c-4874ea154f3a\") " pod="opendatahub/maas-controller-7c697dc6f4-sm2xc" Jul 23 22:05:55.962031 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:05:55.962014 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f3cf2417-a6d7-41ca-928c-4874ea154f3a-webhook-cert\") pod \"maas-controller-7c697dc6f4-sm2xc\" (UID: \"f3cf2417-a6d7-41ca-928c-4874ea154f3a\") " pod="opendatahub/maas-controller-7c697dc6f4-sm2xc" Jul 23 22:05:55.967496 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:05:55.967476 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9dj6x\" (UniqueName: \"kubernetes.io/projected/f3cf2417-a6d7-41ca-928c-4874ea154f3a-kube-api-access-9dj6x\") pod \"maas-controller-7c697dc6f4-sm2xc\" (UID: \"f3cf2417-a6d7-41ca-928c-4874ea154f3a\") " pod="opendatahub/maas-controller-7c697dc6f4-sm2xc" Jul 23 22:05:56.067913 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:05:56.067894 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7c697dc6f4-sm2xc" Jul 23 22:05:56.196500 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:05:56.196477 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7c697dc6f4-sm2xc"] Jul 23 22:05:56.197888 ip-10-0-136-157 kubenswrapper[2578]: W0723 22:05:56.197857 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3cf2417_a6d7_41ca_928c_4874ea154f3a.slice/crio-5f2e4563949bbbf0f21c4631e2d012346ee882fbc2643f05a01f3aef7e17e6ad WatchSource:0}: Error finding container 5f2e4563949bbbf0f21c4631e2d012346ee882fbc2643f05a01f3aef7e17e6ad: Status 404 returned error can't find the container with id 5f2e4563949bbbf0f21c4631e2d012346ee882fbc2643f05a01f3aef7e17e6ad Jul 23 22:05:57.154006 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:05:57.153975 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7c697dc6f4-sm2xc" event={"ID":"f3cf2417-a6d7-41ca-928c-4874ea154f3a","Type":"ContainerStarted","Data":"61181ea74bc9620b190f8bea690a6f6ff07a1ee99d16cd30632b2a9a59e28a20"} Jul 23 22:05:57.154391 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:05:57.154022 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7c697dc6f4-sm2xc" event={"ID":"f3cf2417-a6d7-41ca-928c-4874ea154f3a","Type":"ContainerStarted","Data":"5f2e4563949bbbf0f21c4631e2d012346ee882fbc2643f05a01f3aef7e17e6ad"} Jul 23 22:05:57.154391 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:05:57.154059 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7c697dc6f4-sm2xc" Jul 23 22:05:57.172905 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:05:57.172857 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7c697dc6f4-sm2xc" podStartSLOduration=1.676839743 podStartE2EDuration="2.172844942s" totalImagesPullingTime="496.005199ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 22:05:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 22:05:57.171297685 +0000 UTC m=+859.413205558" watchObservedRunningTime="2026-07-23 22:05:57.172844942 +0000 UTC m=+859.414752816" Jul 23 22:06:08.162290 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:06:08.162259 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7c697dc6f4-sm2xc" Jul 23 22:06:12.958638 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:06:12.958602 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7c697dc6f4-sm2xc"] Jul 23 22:06:12.959100 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:06:12.958802 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-7c697dc6f4-sm2xc" podUID="f3cf2417-a6d7-41ca-928c-4874ea154f3a" containerName="manager" containerID="cri-o://61181ea74bc9620b190f8bea690a6f6ff07a1ee99d16cd30632b2a9a59e28a20" gracePeriod=30 Jul 23 22:06:13.206791 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:06:13.206768 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7c697dc6f4-sm2xc" Jul 23 22:06:13.239477 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:06:13.239407 2578 generic.go:356] "Generic (PLEG): container finished" podID="f3cf2417-a6d7-41ca-928c-4874ea154f3a" containerID="61181ea74bc9620b190f8bea690a6f6ff07a1ee99d16cd30632b2a9a59e28a20" exitCode=0 Jul 23 22:06:13.239477 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:06:13.239451 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7c697dc6f4-sm2xc" event={"ID":"f3cf2417-a6d7-41ca-928c-4874ea154f3a","Type":"ContainerDied","Data":"61181ea74bc9620b190f8bea690a6f6ff07a1ee99d16cd30632b2a9a59e28a20"} Jul 23 22:06:13.239477 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:06:13.239473 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7c697dc6f4-sm2xc" Jul 23 22:06:13.239755 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:06:13.239577 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7c697dc6f4-sm2xc" event={"ID":"f3cf2417-a6d7-41ca-928c-4874ea154f3a","Type":"ContainerDied","Data":"5f2e4563949bbbf0f21c4631e2d012346ee882fbc2643f05a01f3aef7e17e6ad"} Jul 23 22:06:13.239755 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:06:13.239610 2578 scope.go:117] "RemoveContainer" containerID="61181ea74bc9620b190f8bea690a6f6ff07a1ee99d16cd30632b2a9a59e28a20" Jul 23 22:06:13.249772 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:06:13.249755 2578 scope.go:117] "RemoveContainer" containerID="61181ea74bc9620b190f8bea690a6f6ff07a1ee99d16cd30632b2a9a59e28a20" Jul 23 22:06:13.250047 ip-10-0-136-157 kubenswrapper[2578]: E0723 22:06:13.250023 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"61181ea74bc9620b190f8bea690a6f6ff07a1ee99d16cd30632b2a9a59e28a20\": container with ID starting with 61181ea74bc9620b190f8bea690a6f6ff07a1ee99d16cd30632b2a9a59e28a20 not found: ID does not exist" containerID="61181ea74bc9620b190f8bea690a6f6ff07a1ee99d16cd30632b2a9a59e28a20" Jul 23 22:06:13.250135 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:06:13.250055 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"61181ea74bc9620b190f8bea690a6f6ff07a1ee99d16cd30632b2a9a59e28a20"} err="failed to get container status \"61181ea74bc9620b190f8bea690a6f6ff07a1ee99d16cd30632b2a9a59e28a20\": rpc error: code = NotFound desc = could not find container \"61181ea74bc9620b190f8bea690a6f6ff07a1ee99d16cd30632b2a9a59e28a20\": container with ID starting with 61181ea74bc9620b190f8bea690a6f6ff07a1ee99d16cd30632b2a9a59e28a20 not found: ID does not exist" Jul 23 22:06:13.292407 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:06:13.292384 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f3cf2417-a6d7-41ca-928c-4874ea154f3a-webhook-cert\") pod \"f3cf2417-a6d7-41ca-928c-4874ea154f3a\" (UID: \"f3cf2417-a6d7-41ca-928c-4874ea154f3a\") " Jul 23 22:06:13.292510 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:06:13.292426 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9dj6x\" (UniqueName: \"kubernetes.io/projected/f3cf2417-a6d7-41ca-928c-4874ea154f3a-kube-api-access-9dj6x\") pod \"f3cf2417-a6d7-41ca-928c-4874ea154f3a\" (UID: \"f3cf2417-a6d7-41ca-928c-4874ea154f3a\") " Jul 23 22:06:13.294341 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:06:13.294320 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f3cf2417-a6d7-41ca-928c-4874ea154f3a-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "f3cf2417-a6d7-41ca-928c-4874ea154f3a" (UID: "f3cf2417-a6d7-41ca-928c-4874ea154f3a"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 22:06:13.294425 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:06:13.294381 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f3cf2417-a6d7-41ca-928c-4874ea154f3a-kube-api-access-9dj6x" (OuterVolumeSpecName: "kube-api-access-9dj6x") pod "f3cf2417-a6d7-41ca-928c-4874ea154f3a" (UID: "f3cf2417-a6d7-41ca-928c-4874ea154f3a"). InnerVolumeSpecName "kube-api-access-9dj6x". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 22:06:13.393703 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:06:13.393672 2578 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f3cf2417-a6d7-41ca-928c-4874ea154f3a-webhook-cert\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\"" Jul 23 22:06:13.393703 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:06:13.393698 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9dj6x\" (UniqueName: \"kubernetes.io/projected/f3cf2417-a6d7-41ca-928c-4874ea154f3a-kube-api-access-9dj6x\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\"" Jul 23 22:06:13.562123 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:06:13.562099 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7c697dc6f4-sm2xc"] Jul 23 22:06:13.566025 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:06:13.565999 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7c697dc6f4-sm2xc"] Jul 23 22:06:14.250142 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:06:14.250119 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f3cf2417-a6d7-41ca-928c-4874ea154f3a" path="/var/lib/kubelet/pods/f3cf2417-a6d7-41ca-928c-4874ea154f3a/volumes" Jul 23 22:06:14.465415 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:06:14.465390 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7c697dc6f4-vsxn9"] Jul 23 22:06:14.465728 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:06:14.465714 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f3cf2417-a6d7-41ca-928c-4874ea154f3a" containerName="manager" Jul 23 22:06:14.465776 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:06:14.465730 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="f3cf2417-a6d7-41ca-928c-4874ea154f3a" containerName="manager" Jul 23 22:06:14.465808 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:06:14.465780 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="f3cf2417-a6d7-41ca-928c-4874ea154f3a" containerName="manager" Jul 23 22:06:14.469950 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:06:14.469935 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7c697dc6f4-vsxn9" Jul 23 22:06:14.473152 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:06:14.473132 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-rlp68\"" Jul 23 22:06:14.473152 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:06:14.473141 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 23 22:06:14.473297 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:06:14.473158 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 23 22:06:14.479213 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:06:14.479182 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7c697dc6f4-vsxn9"] Jul 23 22:06:14.501652 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:06:14.501589 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/89f2a4a2-1302-470c-90ab-c09dbcd525a4-webhook-cert\") pod \"maas-controller-7c697dc6f4-vsxn9\" (UID: \"89f2a4a2-1302-470c-90ab-c09dbcd525a4\") " pod="opendatahub/maas-controller-7c697dc6f4-vsxn9" Jul 23 22:06:14.501652 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:06:14.501625 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-24pfr\" (UniqueName: \"kubernetes.io/projected/89f2a4a2-1302-470c-90ab-c09dbcd525a4-kube-api-access-24pfr\") pod \"maas-controller-7c697dc6f4-vsxn9\" (UID: \"89f2a4a2-1302-470c-90ab-c09dbcd525a4\") " pod="opendatahub/maas-controller-7c697dc6f4-vsxn9" Jul 23 22:06:14.601990 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:06:14.601961 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/89f2a4a2-1302-470c-90ab-c09dbcd525a4-webhook-cert\") pod \"maas-controller-7c697dc6f4-vsxn9\" (UID: \"89f2a4a2-1302-470c-90ab-c09dbcd525a4\") " pod="opendatahub/maas-controller-7c697dc6f4-vsxn9" Jul 23 22:06:14.601990 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:06:14.601994 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-24pfr\" (UniqueName: \"kubernetes.io/projected/89f2a4a2-1302-470c-90ab-c09dbcd525a4-kube-api-access-24pfr\") pod \"maas-controller-7c697dc6f4-vsxn9\" (UID: \"89f2a4a2-1302-470c-90ab-c09dbcd525a4\") " pod="opendatahub/maas-controller-7c697dc6f4-vsxn9" Jul 23 22:06:14.604433 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:06:14.604411 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/89f2a4a2-1302-470c-90ab-c09dbcd525a4-webhook-cert\") pod \"maas-controller-7c697dc6f4-vsxn9\" (UID: \"89f2a4a2-1302-470c-90ab-c09dbcd525a4\") " pod="opendatahub/maas-controller-7c697dc6f4-vsxn9" Jul 23 22:06:14.611442 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:06:14.611422 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-24pfr\" (UniqueName: \"kubernetes.io/projected/89f2a4a2-1302-470c-90ab-c09dbcd525a4-kube-api-access-24pfr\") pod \"maas-controller-7c697dc6f4-vsxn9\" (UID: \"89f2a4a2-1302-470c-90ab-c09dbcd525a4\") " pod="opendatahub/maas-controller-7c697dc6f4-vsxn9" Jul 23 22:06:14.780897 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:06:14.780845 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7c697dc6f4-vsxn9" Jul 23 22:06:15.115131 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:06:15.115105 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7c697dc6f4-vsxn9"] Jul 23 22:06:15.116811 ip-10-0-136-157 kubenswrapper[2578]: W0723 22:06:15.116782 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89f2a4a2_1302_470c_90ab_c09dbcd525a4.slice/crio-dfb91315286163ad9fdea940644ad1e6c06c91edeb1943170c3789d217718e33 WatchSource:0}: Error finding container dfb91315286163ad9fdea940644ad1e6c06c91edeb1943170c3789d217718e33: Status 404 returned error can't find the container with id dfb91315286163ad9fdea940644ad1e6c06c91edeb1943170c3789d217718e33 Jul 23 22:06:15.251354 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:06:15.251330 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7c697dc6f4-vsxn9" event={"ID":"89f2a4a2-1302-470c-90ab-c09dbcd525a4","Type":"ContainerStarted","Data":"dfb91315286163ad9fdea940644ad1e6c06c91edeb1943170c3789d217718e33"} Jul 23 22:06:16.257300 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:06:16.257273 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7c697dc6f4-vsxn9" event={"ID":"89f2a4a2-1302-470c-90ab-c09dbcd525a4","Type":"ContainerStarted","Data":"1c4f2e08eca4191deb10e47f60111eff691902b7c052050acf9b6501a837c1b8"} Jul 23 22:06:16.257653 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:06:16.257415 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7c697dc6f4-vsxn9" Jul 23 22:06:16.275810 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:06:16.275764 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7c697dc6f4-vsxn9" podStartSLOduration=1.892092215 podStartE2EDuration="2.275751924s" totalImagesPullingTime="383.659709ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 22:06:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 22:06:16.273255524 +0000 UTC m=+878.515163397" watchObservedRunningTime="2026-07-23 22:06:16.275751924 +0000 UTC m=+878.517659798" Jul 23 22:06:27.265451 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:06:27.265417 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7c697dc6f4-vsxn9" Jul 23 22:06:38.560170 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:06:38.560135 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7jkjm_7342a9f6-72b1-4e3c-8d50-0b0326454a86/ovn-acl-logging/0.log" Jul 23 22:06:38.561795 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:06:38.561776 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7jkjm_7342a9f6-72b1-4e3c-8d50-0b0326454a86/ovn-acl-logging/0.log" Jul 23 22:11:38.591314 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:11:38.591279 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7jkjm_7342a9f6-72b1-4e3c-8d50-0b0326454a86/ovn-acl-logging/0.log" Jul 23 22:11:38.594101 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:11:38.594081 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7jkjm_7342a9f6-72b1-4e3c-8d50-0b0326454a86/ovn-acl-logging/0.log" Jul 23 22:16:38.630093 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:16:38.630062 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7jkjm_7342a9f6-72b1-4e3c-8d50-0b0326454a86/ovn-acl-logging/0.log" Jul 23 22:16:38.633464 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:16:38.633443 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7jkjm_7342a9f6-72b1-4e3c-8d50-0b0326454a86/ovn-acl-logging/0.log" Jul 23 22:19:48.388580 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:19:48.388530 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-487a121b-openshift-default-5cdb89895f-25jg2"] Jul 23 22:19:48.391951 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:19:48.391931 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-487a121b-openshift-default-5cdb89895f-25jg2" Jul 23 22:19:48.394292 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:19:48.394271 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-487a121b-openshift-default-dockercfg-wxn9t\"" Jul 23 22:19:48.420310 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:19:48.420282 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-487a121b-openshift-default-5cdb89895f-25jg2"] Jul 23 22:19:48.494982 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:19:48.494952 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4050b672-38cb-4428-a0b0-4c643ba4887f-credential-socket\") pod \"e2e-ait-487a121b-openshift-default-5cdb89895f-25jg2\" (UID: \"4050b672-38cb-4428-a0b0-4c643ba4887f\") " pod="openshift-ingress/e2e-ait-487a121b-openshift-default-5cdb89895f-25jg2" Jul 23 22:19:48.495155 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:19:48.494996 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4050b672-38cb-4428-a0b0-4c643ba4887f-istiod-ca-cert\") pod \"e2e-ait-487a121b-openshift-default-5cdb89895f-25jg2\" (UID: \"4050b672-38cb-4428-a0b0-4c643ba4887f\") " pod="openshift-ingress/e2e-ait-487a121b-openshift-default-5cdb89895f-25jg2" Jul 23 22:19:48.495155 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:19:48.495028 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4050b672-38cb-4428-a0b0-4c643ba4887f-istio-token\") pod \"e2e-ait-487a121b-openshift-default-5cdb89895f-25jg2\" (UID: \"4050b672-38cb-4428-a0b0-4c643ba4887f\") " pod="openshift-ingress/e2e-ait-487a121b-openshift-default-5cdb89895f-25jg2" Jul 23 22:19:48.495155 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:19:48.495055 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4050b672-38cb-4428-a0b0-4c643ba4887f-istio-envoy\") pod \"e2e-ait-487a121b-openshift-default-5cdb89895f-25jg2\" (UID: \"4050b672-38cb-4428-a0b0-4c643ba4887f\") " pod="openshift-ingress/e2e-ait-487a121b-openshift-default-5cdb89895f-25jg2" Jul 23 22:19:48.495155 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:19:48.495079 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4050b672-38cb-4428-a0b0-4c643ba4887f-workload-socket\") pod \"e2e-ait-487a121b-openshift-default-5cdb89895f-25jg2\" (UID: \"4050b672-38cb-4428-a0b0-4c643ba4887f\") " pod="openshift-ingress/e2e-ait-487a121b-openshift-default-5cdb89895f-25jg2" Jul 23 22:19:48.495155 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:19:48.495101 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4050b672-38cb-4428-a0b0-4c643ba4887f-istio-data\") pod \"e2e-ait-487a121b-openshift-default-5cdb89895f-25jg2\" (UID: \"4050b672-38cb-4428-a0b0-4c643ba4887f\") " pod="openshift-ingress/e2e-ait-487a121b-openshift-default-5cdb89895f-25jg2" Jul 23 22:19:48.495155 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:19:48.495115 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4050b672-38cb-4428-a0b0-4c643ba4887f-istio-podinfo\") pod \"e2e-ait-487a121b-openshift-default-5cdb89895f-25jg2\" (UID: \"4050b672-38cb-4428-a0b0-4c643ba4887f\") " pod="openshift-ingress/e2e-ait-487a121b-openshift-default-5cdb89895f-25jg2" Jul 23 22:19:48.495363 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:19:48.495194 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4050b672-38cb-4428-a0b0-4c643ba4887f-workload-certs\") pod \"e2e-ait-487a121b-openshift-default-5cdb89895f-25jg2\" (UID: \"4050b672-38cb-4428-a0b0-4c643ba4887f\") " pod="openshift-ingress/e2e-ait-487a121b-openshift-default-5cdb89895f-25jg2" Jul 23 22:19:48.495363 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:19:48.495222 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5lmtm\" (UniqueName: \"kubernetes.io/projected/4050b672-38cb-4428-a0b0-4c643ba4887f-kube-api-access-5lmtm\") pod \"e2e-ait-487a121b-openshift-default-5cdb89895f-25jg2\" (UID: \"4050b672-38cb-4428-a0b0-4c643ba4887f\") " pod="openshift-ingress/e2e-ait-487a121b-openshift-default-5cdb89895f-25jg2" Jul 23 22:19:48.595725 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:19:48.595692 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4050b672-38cb-4428-a0b0-4c643ba4887f-istiod-ca-cert\") pod \"e2e-ait-487a121b-openshift-default-5cdb89895f-25jg2\" (UID: \"4050b672-38cb-4428-a0b0-4c643ba4887f\") " pod="openshift-ingress/e2e-ait-487a121b-openshift-default-5cdb89895f-25jg2" Jul 23 22:19:48.595948 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:19:48.595752 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4050b672-38cb-4428-a0b0-4c643ba4887f-istio-token\") pod \"e2e-ait-487a121b-openshift-default-5cdb89895f-25jg2\" (UID: \"4050b672-38cb-4428-a0b0-4c643ba4887f\") " pod="openshift-ingress/e2e-ait-487a121b-openshift-default-5cdb89895f-25jg2" Jul 23 22:19:48.595948 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:19:48.595773 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4050b672-38cb-4428-a0b0-4c643ba4887f-istio-envoy\") pod \"e2e-ait-487a121b-openshift-default-5cdb89895f-25jg2\" (UID: \"4050b672-38cb-4428-a0b0-4c643ba4887f\") " pod="openshift-ingress/e2e-ait-487a121b-openshift-default-5cdb89895f-25jg2" Jul 23 22:19:48.595948 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:19:48.595788 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4050b672-38cb-4428-a0b0-4c643ba4887f-workload-socket\") pod \"e2e-ait-487a121b-openshift-default-5cdb89895f-25jg2\" (UID: \"4050b672-38cb-4428-a0b0-4c643ba4887f\") " pod="openshift-ingress/e2e-ait-487a121b-openshift-default-5cdb89895f-25jg2" Jul 23 22:19:48.595948 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:19:48.595813 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4050b672-38cb-4428-a0b0-4c643ba4887f-istio-data\") pod \"e2e-ait-487a121b-openshift-default-5cdb89895f-25jg2\" (UID: \"4050b672-38cb-4428-a0b0-4c643ba4887f\") " pod="openshift-ingress/e2e-ait-487a121b-openshift-default-5cdb89895f-25jg2" Jul 23 22:19:48.595948 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:19:48.595934 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4050b672-38cb-4428-a0b0-4c643ba4887f-istio-podinfo\") pod \"e2e-ait-487a121b-openshift-default-5cdb89895f-25jg2\" (UID: \"4050b672-38cb-4428-a0b0-4c643ba4887f\") " pod="openshift-ingress/e2e-ait-487a121b-openshift-default-5cdb89895f-25jg2" Jul 23 22:19:48.596196 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:19:48.596008 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4050b672-38cb-4428-a0b0-4c643ba4887f-workload-certs\") pod \"e2e-ait-487a121b-openshift-default-5cdb89895f-25jg2\" (UID: \"4050b672-38cb-4428-a0b0-4c643ba4887f\") " pod="openshift-ingress/e2e-ait-487a121b-openshift-default-5cdb89895f-25jg2" Jul 23 22:19:48.596196 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:19:48.596033 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5lmtm\" (UniqueName: \"kubernetes.io/projected/4050b672-38cb-4428-a0b0-4c643ba4887f-kube-api-access-5lmtm\") pod \"e2e-ait-487a121b-openshift-default-5cdb89895f-25jg2\" (UID: \"4050b672-38cb-4428-a0b0-4c643ba4887f\") " pod="openshift-ingress/e2e-ait-487a121b-openshift-default-5cdb89895f-25jg2" Jul 23 22:19:48.596196 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:19:48.596085 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4050b672-38cb-4428-a0b0-4c643ba4887f-credential-socket\") pod \"e2e-ait-487a121b-openshift-default-5cdb89895f-25jg2\" (UID: \"4050b672-38cb-4428-a0b0-4c643ba4887f\") " pod="openshift-ingress/e2e-ait-487a121b-openshift-default-5cdb89895f-25jg2" Jul 23 22:19:48.596345 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:19:48.596280 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4050b672-38cb-4428-a0b0-4c643ba4887f-istio-data\") pod \"e2e-ait-487a121b-openshift-default-5cdb89895f-25jg2\" (UID: \"4050b672-38cb-4428-a0b0-4c643ba4887f\") " pod="openshift-ingress/e2e-ait-487a121b-openshift-default-5cdb89895f-25jg2" Jul 23 22:19:48.596521 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:19:48.596497 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4050b672-38cb-4428-a0b0-4c643ba4887f-istiod-ca-cert\") pod \"e2e-ait-487a121b-openshift-default-5cdb89895f-25jg2\" (UID: \"4050b672-38cb-4428-a0b0-4c643ba4887f\") " pod="openshift-ingress/e2e-ait-487a121b-openshift-default-5cdb89895f-25jg2" Jul 23 22:19:48.596677 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:19:48.596655 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4050b672-38cb-4428-a0b0-4c643ba4887f-workload-certs\") pod \"e2e-ait-487a121b-openshift-default-5cdb89895f-25jg2\" (UID: \"4050b672-38cb-4428-a0b0-4c643ba4887f\") " pod="openshift-ingress/e2e-ait-487a121b-openshift-default-5cdb89895f-25jg2" Jul 23 22:19:48.596740 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:19:48.596711 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4050b672-38cb-4428-a0b0-4c643ba4887f-credential-socket\") pod \"e2e-ait-487a121b-openshift-default-5cdb89895f-25jg2\" (UID: \"4050b672-38cb-4428-a0b0-4c643ba4887f\") " pod="openshift-ingress/e2e-ait-487a121b-openshift-default-5cdb89895f-25jg2" Jul 23 22:19:48.597119 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:19:48.597094 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4050b672-38cb-4428-a0b0-4c643ba4887f-workload-socket\") pod \"e2e-ait-487a121b-openshift-default-5cdb89895f-25jg2\" (UID: \"4050b672-38cb-4428-a0b0-4c643ba4887f\") " pod="openshift-ingress/e2e-ait-487a121b-openshift-default-5cdb89895f-25jg2" Jul 23 22:19:48.598754 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:19:48.598732 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4050b672-38cb-4428-a0b0-4c643ba4887f-istio-envoy\") pod \"e2e-ait-487a121b-openshift-default-5cdb89895f-25jg2\" (UID: \"4050b672-38cb-4428-a0b0-4c643ba4887f\") " pod="openshift-ingress/e2e-ait-487a121b-openshift-default-5cdb89895f-25jg2" Jul 23 22:19:48.599099 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:19:48.599080 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4050b672-38cb-4428-a0b0-4c643ba4887f-istio-podinfo\") pod \"e2e-ait-487a121b-openshift-default-5cdb89895f-25jg2\" (UID: \"4050b672-38cb-4428-a0b0-4c643ba4887f\") " pod="openshift-ingress/e2e-ait-487a121b-openshift-default-5cdb89895f-25jg2" Jul 23 22:19:48.611364 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:19:48.611338 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4050b672-38cb-4428-a0b0-4c643ba4887f-istio-token\") pod \"e2e-ait-487a121b-openshift-default-5cdb89895f-25jg2\" (UID: \"4050b672-38cb-4428-a0b0-4c643ba4887f\") " pod="openshift-ingress/e2e-ait-487a121b-openshift-default-5cdb89895f-25jg2" Jul 23 22:19:48.612797 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:19:48.612777 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5lmtm\" (UniqueName: \"kubernetes.io/projected/4050b672-38cb-4428-a0b0-4c643ba4887f-kube-api-access-5lmtm\") pod \"e2e-ait-487a121b-openshift-default-5cdb89895f-25jg2\" (UID: \"4050b672-38cb-4428-a0b0-4c643ba4887f\") " pod="openshift-ingress/e2e-ait-487a121b-openshift-default-5cdb89895f-25jg2" Jul 23 22:19:48.702270 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:19:48.702169 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-487a121b-openshift-default-5cdb89895f-25jg2" Jul 23 22:19:48.847520 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:19:48.847493 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-487a121b-openshift-default-5cdb89895f-25jg2"] Jul 23 22:19:48.849759 ip-10-0-136-157 kubenswrapper[2578]: W0723 22:19:48.849734 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4050b672_38cb_4428_a0b0_4c643ba4887f.slice/crio-094aead6bf038da3348e5973e8286ca2da1cbe4591aa9b18f58669044bf6409e WatchSource:0}: Error finding container 094aead6bf038da3348e5973e8286ca2da1cbe4591aa9b18f58669044bf6409e: Status 404 returned error can't find the container with id 094aead6bf038da3348e5973e8286ca2da1cbe4591aa9b18f58669044bf6409e Jul 23 22:19:48.851777 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:19:48.851749 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 23 22:19:48.851846 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:19:48.851808 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 23 22:19:48.851846 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:19:48.851835 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 23 22:19:49.170213 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:19:49.170172 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-487a121b-openshift-default-5cdb89895f-25jg2" event={"ID":"4050b672-38cb-4428-a0b0-4c643ba4887f","Type":"ContainerStarted","Data":"e6065307f30d17d82ef3da4d36e6bf3b3f18bc7775b4b957b9e22b19fe24fe12"} Jul 23 22:19:49.170383 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:19:49.170255 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-487a121b-openshift-default-5cdb89895f-25jg2" event={"ID":"4050b672-38cb-4428-a0b0-4c643ba4887f","Type":"ContainerStarted","Data":"094aead6bf038da3348e5973e8286ca2da1cbe4591aa9b18f58669044bf6409e"} Jul 23 22:19:49.191588 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:19:49.191502 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-487a121b-openshift-default-5cdb89895f-25jg2" podStartSLOduration=1.19148647 podStartE2EDuration="1.19148647s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 22:19:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 22:19:49.189750315 +0000 UTC m=+1691.431658187" watchObservedRunningTime="2026-07-23 22:19:49.19148647 +0000 UTC m=+1691.433394343" Jul 23 22:19:49.702621 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:19:49.702588 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-487a121b-openshift-default-5cdb89895f-25jg2" Jul 23 22:19:49.707499 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:19:49.707477 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-487a121b-openshift-default-5cdb89895f-25jg2" Jul 23 22:19:50.176129 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:19:50.176093 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-487a121b-openshift-default-5cdb89895f-25jg2" Jul 23 22:19:50.177164 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:19:50.177143 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-487a121b-openshift-default-5cdb89895f-25jg2" Jul 23 22:20:02.237368 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:20:02.237332 2578 generic.go:356] "Generic (PLEG): container finished" podID="4050b672-38cb-4428-a0b0-4c643ba4887f" containerID="e6065307f30d17d82ef3da4d36e6bf3b3f18bc7775b4b957b9e22b19fe24fe12" exitCode=0 Jul 23 22:20:02.237368 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:20:02.237371 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-487a121b-openshift-default-5cdb89895f-25jg2" event={"ID":"4050b672-38cb-4428-a0b0-4c643ba4887f","Type":"ContainerDied","Data":"e6065307f30d17d82ef3da4d36e6bf3b3f18bc7775b4b957b9e22b19fe24fe12"} Jul 23 22:20:02.237814 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:20:02.237741 2578 scope.go:117] "RemoveContainer" containerID="e6065307f30d17d82ef3da4d36e6bf3b3f18bc7775b4b957b9e22b19fe24fe12" Jul 23 22:20:02.238778 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:20:02.238752 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 23 22:20:02.238839 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:20:02.238819 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 23 22:20:02.238873 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:20:02.238850 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 23 22:20:03.252304 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:20:03.252257 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-487a121b-openshift-default-5cdb89895f-25jg2" event={"ID":"4050b672-38cb-4428-a0b0-4c643ba4887f","Type":"ContainerStarted","Data":"6e6ec79c604603d64fcf2cf770846b33303c852f2fc12a5bb59c9aeff13b2fea"} Jul 23 22:20:03.702321 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:20:03.702291 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-487a121b-openshift-default-5cdb89895f-25jg2" Jul 23 22:20:03.703058 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:20:03.703029 2578 patch_prober.go:28] interesting pod/e2e-ait-487a121b-openshift-default-5cdb89895f-25jg2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body= Jul 23 22:20:03.703185 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:20:03.703086 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-487a121b-openshift-default-5cdb89895f-25jg2" podUID="4050b672-38cb-4428-a0b0-4c643ba4887f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" Jul 23 22:20:04.703511 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:20:04.703475 2578 patch_prober.go:28] interesting pod/e2e-ait-487a121b-openshift-default-5cdb89895f-25jg2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body= Jul 23 22:20:04.703952 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:20:04.703529 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-487a121b-openshift-default-5cdb89895f-25jg2" podUID="4050b672-38cb-4428-a0b0-4c643ba4887f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" Jul 23 22:20:05.176949 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:20:05.176900 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-487a121b-openshift-default-5cdb89895f-25jg2" Jul 23 22:20:05.702978 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:20:05.702945 2578 patch_prober.go:28] interesting pod/e2e-ait-487a121b-openshift-default-5cdb89895f-25jg2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body= Jul 23 22:20:05.703151 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:20:05.702999 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-487a121b-openshift-default-5cdb89895f-25jg2" podUID="4050b672-38cb-4428-a0b0-4c643ba4887f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" Jul 23 22:20:06.702634 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:20:06.702600 2578 patch_prober.go:28] interesting pod/e2e-ait-487a121b-openshift-default-5cdb89895f-25jg2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body= Jul 23 22:20:06.703025 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:20:06.702654 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-487a121b-openshift-default-5cdb89895f-25jg2" podUID="4050b672-38cb-4428-a0b0-4c643ba4887f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" Jul 23 22:20:07.706190 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:20:07.706158 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-487a121b-openshift-default-5cdb89895f-25jg2" Jul 23 22:20:07.707160 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:20:07.707143 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-487a121b-openshift-default-5cdb89895f-25jg2" Jul 23 22:20:31.415387 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:20:31.415354 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-487a121b-openshift-default-5cdb89895f-25jg2"] Jul 23 22:20:31.415908 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:20:31.415627 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-487a121b-openshift-default-5cdb89895f-25jg2" podUID="4050b672-38cb-4428-a0b0-4c643ba4887f" containerName="istio-proxy" containerID="cri-o://6e6ec79c604603d64fcf2cf770846b33303c852f2fc12a5bb59c9aeff13b2fea" gracePeriod=30 Jul 23 22:20:36.664645 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:20:36.664620 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-487a121b-openshift-default-5cdb89895f-25jg2" Jul 23 22:20:36.723010 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:20:36.722933 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4050b672-38cb-4428-a0b0-4c643ba4887f-istiod-ca-cert\") pod \"4050b672-38cb-4428-a0b0-4c643ba4887f\" (UID: \"4050b672-38cb-4428-a0b0-4c643ba4887f\") " Jul 23 22:20:36.723010 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:20:36.722964 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4050b672-38cb-4428-a0b0-4c643ba4887f-istio-token\") pod \"4050b672-38cb-4428-a0b0-4c643ba4887f\" (UID: \"4050b672-38cb-4428-a0b0-4c643ba4887f\") " Jul 23 22:20:36.723010 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:20:36.723000 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4050b672-38cb-4428-a0b0-4c643ba4887f-workload-socket\") pod \"4050b672-38cb-4428-a0b0-4c643ba4887f\" (UID: \"4050b672-38cb-4428-a0b0-4c643ba4887f\") " Jul 23 22:20:36.723269 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:20:36.723020 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4050b672-38cb-4428-a0b0-4c643ba4887f-istio-data\") pod \"4050b672-38cb-4428-a0b0-4c643ba4887f\" (UID: \"4050b672-38cb-4428-a0b0-4c643ba4887f\") " Jul 23 22:20:36.723269 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:20:36.723100 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4050b672-38cb-4428-a0b0-4c643ba4887f-workload-certs\") pod \"4050b672-38cb-4428-a0b0-4c643ba4887f\" (UID: \"4050b672-38cb-4428-a0b0-4c643ba4887f\") " Jul 23 22:20:36.723269 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:20:36.723146 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4050b672-38cb-4428-a0b0-4c643ba4887f-istio-podinfo\") pod \"4050b672-38cb-4428-a0b0-4c643ba4887f\" (UID: \"4050b672-38cb-4428-a0b0-4c643ba4887f\") " Jul 23 22:20:36.723269 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:20:36.723174 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4050b672-38cb-4428-a0b0-4c643ba4887f-istio-envoy\") pod \"4050b672-38cb-4428-a0b0-4c643ba4887f\" (UID: \"4050b672-38cb-4428-a0b0-4c643ba4887f\") " Jul 23 22:20:36.723269 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:20:36.723242 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5lmtm\" (UniqueName: \"kubernetes.io/projected/4050b672-38cb-4428-a0b0-4c643ba4887f-kube-api-access-5lmtm\") pod \"4050b672-38cb-4428-a0b0-4c643ba4887f\" (UID: \"4050b672-38cb-4428-a0b0-4c643ba4887f\") " Jul 23 22:20:36.723499 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:20:36.723284 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4050b672-38cb-4428-a0b0-4c643ba4887f-credential-socket\") pod \"4050b672-38cb-4428-a0b0-4c643ba4887f\" (UID: \"4050b672-38cb-4428-a0b0-4c643ba4887f\") " Jul 23 22:20:36.723499 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:20:36.723345 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4050b672-38cb-4428-a0b0-4c643ba4887f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "4050b672-38cb-4428-a0b0-4c643ba4887f" (UID: "4050b672-38cb-4428-a0b0-4c643ba4887f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 22:20:36.723499 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:20:36.723437 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4050b672-38cb-4428-a0b0-4c643ba4887f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "4050b672-38cb-4428-a0b0-4c643ba4887f" (UID: "4050b672-38cb-4428-a0b0-4c643ba4887f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 22:20:36.723682 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:20:36.723604 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4050b672-38cb-4428-a0b0-4c643ba4887f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "4050b672-38cb-4428-a0b0-4c643ba4887f" (UID: "4050b672-38cb-4428-a0b0-4c643ba4887f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 22:20:36.723682 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:20:36.723675 2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4050b672-38cb-4428-a0b0-4c643ba4887f-credential-socket\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\"" Jul 23 22:20:36.723795 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:20:36.723694 2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4050b672-38cb-4428-a0b0-4c643ba4887f-istiod-ca-cert\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\"" Jul 23 22:20:36.723795 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:20:36.723694 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4050b672-38cb-4428-a0b0-4c643ba4887f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "4050b672-38cb-4428-a0b0-4c643ba4887f" (UID: "4050b672-38cb-4428-a0b0-4c643ba4887f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 22:20:36.723795 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:20:36.723708 2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4050b672-38cb-4428-a0b0-4c643ba4887f-workload-socket\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\"" Jul 23 22:20:36.723916 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:20:36.723798 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4050b672-38cb-4428-a0b0-4c643ba4887f-istio-data" (OuterVolumeSpecName: "istio-data") pod "4050b672-38cb-4428-a0b0-4c643ba4887f" (UID: "4050b672-38cb-4428-a0b0-4c643ba4887f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 22:20:36.725509 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:20:36.725480 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4050b672-38cb-4428-a0b0-4c643ba4887f-istio-token" (OuterVolumeSpecName: "istio-token") pod "4050b672-38cb-4428-a0b0-4c643ba4887f" (UID: "4050b672-38cb-4428-a0b0-4c643ba4887f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 22:20:36.725716 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:20:36.725510 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4050b672-38cb-4428-a0b0-4c643ba4887f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "4050b672-38cb-4428-a0b0-4c643ba4887f" (UID: "4050b672-38cb-4428-a0b0-4c643ba4887f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 23 22:20:36.725716 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:20:36.725525 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4050b672-38cb-4428-a0b0-4c643ba4887f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "4050b672-38cb-4428-a0b0-4c643ba4887f" (UID: "4050b672-38cb-4428-a0b0-4c643ba4887f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 22:20:36.725808 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:20:36.725707 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4050b672-38cb-4428-a0b0-4c643ba4887f-kube-api-access-5lmtm" (OuterVolumeSpecName: "kube-api-access-5lmtm") pod "4050b672-38cb-4428-a0b0-4c643ba4887f" (UID: "4050b672-38cb-4428-a0b0-4c643ba4887f"). InnerVolumeSpecName "kube-api-access-5lmtm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 22:20:36.824357 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:20:36.824327 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5lmtm\" (UniqueName: \"kubernetes.io/projected/4050b672-38cb-4428-a0b0-4c643ba4887f-kube-api-access-5lmtm\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\"" Jul 23 22:20:36.824357 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:20:36.824354 2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4050b672-38cb-4428-a0b0-4c643ba4887f-istio-token\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\"" Jul 23 22:20:36.824357 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:20:36.824364 2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4050b672-38cb-4428-a0b0-4c643ba4887f-istio-data\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\"" Jul 23 22:20:36.824587 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:20:36.824373 2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4050b672-38cb-4428-a0b0-4c643ba4887f-workload-certs\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\"" Jul 23 22:20:36.824587 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:20:36.824380 2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4050b672-38cb-4428-a0b0-4c643ba4887f-istio-podinfo\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\"" Jul 23 22:20:36.824587 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:20:36.824388 2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4050b672-38cb-4428-a0b0-4c643ba4887f-istio-envoy\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\"" Jul 23 22:20:37.432437 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:20:37.432398 2578 generic.go:356] "Generic (PLEG): container finished" podID="4050b672-38cb-4428-a0b0-4c643ba4887f" containerID="6e6ec79c604603d64fcf2cf770846b33303c852f2fc12a5bb59c9aeff13b2fea" exitCode=0 Jul 23 22:20:37.432621 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:20:37.432464 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-487a121b-openshift-default-5cdb89895f-25jg2" Jul 23 22:20:37.432621 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:20:37.432489 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-487a121b-openshift-default-5cdb89895f-25jg2" event={"ID":"4050b672-38cb-4428-a0b0-4c643ba4887f","Type":"ContainerDied","Data":"6e6ec79c604603d64fcf2cf770846b33303c852f2fc12a5bb59c9aeff13b2fea"} Jul 23 22:20:37.432621 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:20:37.432588 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-487a121b-openshift-default-5cdb89895f-25jg2" event={"ID":"4050b672-38cb-4428-a0b0-4c643ba4887f","Type":"ContainerDied","Data":"094aead6bf038da3348e5973e8286ca2da1cbe4591aa9b18f58669044bf6409e"} Jul 23 22:20:37.432621 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:20:37.432611 2578 scope.go:117] "RemoveContainer" containerID="6e6ec79c604603d64fcf2cf770846b33303c852f2fc12a5bb59c9aeff13b2fea" Jul 23 22:20:37.442270 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:20:37.442253 2578 scope.go:117] "RemoveContainer" containerID="e6065307f30d17d82ef3da4d36e6bf3b3f18bc7775b4b957b9e22b19fe24fe12" Jul 23 22:20:37.450901 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:20:37.450884 2578 scope.go:117] "RemoveContainer" containerID="6e6ec79c604603d64fcf2cf770846b33303c852f2fc12a5bb59c9aeff13b2fea" Jul 23 22:20:37.451157 ip-10-0-136-157 kubenswrapper[2578]: E0723 22:20:37.451134 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6e6ec79c604603d64fcf2cf770846b33303c852f2fc12a5bb59c9aeff13b2fea\": container with ID starting with 6e6ec79c604603d64fcf2cf770846b33303c852f2fc12a5bb59c9aeff13b2fea not found: ID does not exist" containerID="6e6ec79c604603d64fcf2cf770846b33303c852f2fc12a5bb59c9aeff13b2fea" Jul 23 22:20:37.451201 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:20:37.451168 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e6ec79c604603d64fcf2cf770846b33303c852f2fc12a5bb59c9aeff13b2fea"} err="failed to get container status \"6e6ec79c604603d64fcf2cf770846b33303c852f2fc12a5bb59c9aeff13b2fea\": rpc error: code = NotFound desc = could not find container \"6e6ec79c604603d64fcf2cf770846b33303c852f2fc12a5bb59c9aeff13b2fea\": container with ID starting with 6e6ec79c604603d64fcf2cf770846b33303c852f2fc12a5bb59c9aeff13b2fea not found: ID does not exist" Jul 23 22:20:37.451201 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:20:37.451186 2578 scope.go:117] "RemoveContainer" containerID="e6065307f30d17d82ef3da4d36e6bf3b3f18bc7775b4b957b9e22b19fe24fe12" Jul 23 22:20:37.451447 ip-10-0-136-157 kubenswrapper[2578]: E0723 22:20:37.451429 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e6065307f30d17d82ef3da4d36e6bf3b3f18bc7775b4b957b9e22b19fe24fe12\": container with ID starting with e6065307f30d17d82ef3da4d36e6bf3b3f18bc7775b4b957b9e22b19fe24fe12 not found: ID does not exist" containerID="e6065307f30d17d82ef3da4d36e6bf3b3f18bc7775b4b957b9e22b19fe24fe12" Jul 23 22:20:37.451491 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:20:37.451454 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e6065307f30d17d82ef3da4d36e6bf3b3f18bc7775b4b957b9e22b19fe24fe12"} err="failed to get container status \"e6065307f30d17d82ef3da4d36e6bf3b3f18bc7775b4b957b9e22b19fe24fe12\": rpc error: code = NotFound desc = could not find container \"e6065307f30d17d82ef3da4d36e6bf3b3f18bc7775b4b957b9e22b19fe24fe12\": container with ID starting with e6065307f30d17d82ef3da4d36e6bf3b3f18bc7775b4b957b9e22b19fe24fe12 not found: ID does not exist" Jul 23 22:20:37.454591 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:20:37.454566 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-487a121b-openshift-default-5cdb89895f-25jg2"] Jul 23 22:20:37.457071 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:20:37.457051 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-487a121b-openshift-default-5cdb89895f-25jg2"] Jul 23 22:20:38.251330 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:20:38.251298 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4050b672-38cb-4428-a0b0-4c643ba4887f" path="/var/lib/kubelet/pods/4050b672-38cb-4428-a0b0-4c643ba4887f/volumes" Jul 23 22:21:18.665499 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:21:18.665466 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-0cf92335-openshift-default-547796bdc7-54md6"] Jul 23 22:21:18.666101 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:21:18.665815 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4050b672-38cb-4428-a0b0-4c643ba4887f" containerName="istio-proxy" Jul 23 22:21:18.666101 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:21:18.665827 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="4050b672-38cb-4428-a0b0-4c643ba4887f" containerName="istio-proxy" Jul 23 22:21:18.666101 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:21:18.665835 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4050b672-38cb-4428-a0b0-4c643ba4887f" containerName="istio-proxy" Jul 23 22:21:18.666101 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:21:18.665840 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="4050b672-38cb-4428-a0b0-4c643ba4887f" containerName="istio-proxy" Jul 23 22:21:18.666101 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:21:18.665903 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="4050b672-38cb-4428-a0b0-4c643ba4887f" containerName="istio-proxy" Jul 23 22:21:18.666101 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:21:18.665911 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="4050b672-38cb-4428-a0b0-4c643ba4887f" containerName="istio-proxy" Jul 23 22:21:18.668723 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:21:18.668701 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-0cf92335-openshift-default-547796bdc7-54md6" Jul 23 22:21:18.671341 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:21:18.671322 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-0cf92335-openshift-default-dockercfg-d4zj8\"" Jul 23 22:21:18.684569 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:21:18.684519 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-0cf92335-openshift-default-547796bdc7-54md6"] Jul 23 22:21:18.766364 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:21:18.766330 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4217af42-8645-4d40-a131-7a94a132db8b-istio-podinfo\") pod \"e2e-ait-0cf92335-openshift-default-547796bdc7-54md6\" (UID: \"4217af42-8645-4d40-a131-7a94a132db8b\") " pod="openshift-ingress/e2e-ait-0cf92335-openshift-default-547796bdc7-54md6" Jul 23 22:21:18.766825 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:21:18.766782 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4217af42-8645-4d40-a131-7a94a132db8b-credential-socket\") pod \"e2e-ait-0cf92335-openshift-default-547796bdc7-54md6\" (UID: \"4217af42-8645-4d40-a131-7a94a132db8b\") " pod="openshift-ingress/e2e-ait-0cf92335-openshift-default-547796bdc7-54md6" Jul 23 22:21:18.766952 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:21:18.766857 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4217af42-8645-4d40-a131-7a94a132db8b-workload-socket\") pod \"e2e-ait-0cf92335-openshift-default-547796bdc7-54md6\" (UID: \"4217af42-8645-4d40-a131-7a94a132db8b\") " pod="openshift-ingress/e2e-ait-0cf92335-openshift-default-547796bdc7-54md6" Jul 23 22:21:18.766952 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:21:18.766880 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4217af42-8645-4d40-a131-7a94a132db8b-istio-token\") pod \"e2e-ait-0cf92335-openshift-default-547796bdc7-54md6\" (UID: \"4217af42-8645-4d40-a131-7a94a132db8b\") " pod="openshift-ingress/e2e-ait-0cf92335-openshift-default-547796bdc7-54md6" Jul 23 22:21:18.766952 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:21:18.766929 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4217af42-8645-4d40-a131-7a94a132db8b-istio-envoy\") pod \"e2e-ait-0cf92335-openshift-default-547796bdc7-54md6\" (UID: \"4217af42-8645-4d40-a131-7a94a132db8b\") " pod="openshift-ingress/e2e-ait-0cf92335-openshift-default-547796bdc7-54md6" Jul 23 22:21:18.767108 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:21:18.766951 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-727jq\" (UniqueName: \"kubernetes.io/projected/4217af42-8645-4d40-a131-7a94a132db8b-kube-api-access-727jq\") pod \"e2e-ait-0cf92335-openshift-default-547796bdc7-54md6\" (UID: \"4217af42-8645-4d40-a131-7a94a132db8b\") " pod="openshift-ingress/e2e-ait-0cf92335-openshift-default-547796bdc7-54md6" Jul 23 22:21:18.767108 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:21:18.766989 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4217af42-8645-4d40-a131-7a94a132db8b-workload-certs\") pod \"e2e-ait-0cf92335-openshift-default-547796bdc7-54md6\" (UID: \"4217af42-8645-4d40-a131-7a94a132db8b\") " pod="openshift-ingress/e2e-ait-0cf92335-openshift-default-547796bdc7-54md6" Jul 23 22:21:18.767108 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:21:18.767036 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4217af42-8645-4d40-a131-7a94a132db8b-istio-data\") pod \"e2e-ait-0cf92335-openshift-default-547796bdc7-54md6\" (UID: \"4217af42-8645-4d40-a131-7a94a132db8b\") " pod="openshift-ingress/e2e-ait-0cf92335-openshift-default-547796bdc7-54md6" Jul 23 22:21:18.767108 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:21:18.767058 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4217af42-8645-4d40-a131-7a94a132db8b-istiod-ca-cert\") pod \"e2e-ait-0cf92335-openshift-default-547796bdc7-54md6\" (UID: \"4217af42-8645-4d40-a131-7a94a132db8b\") " pod="openshift-ingress/e2e-ait-0cf92335-openshift-default-547796bdc7-54md6" Jul 23 22:21:18.868665 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:21:18.868638 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4217af42-8645-4d40-a131-7a94a132db8b-workload-socket\") pod \"e2e-ait-0cf92335-openshift-default-547796bdc7-54md6\" (UID: \"4217af42-8645-4d40-a131-7a94a132db8b\") " pod="openshift-ingress/e2e-ait-0cf92335-openshift-default-547796bdc7-54md6" Jul 23 22:21:18.868837 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:21:18.868670 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4217af42-8645-4d40-a131-7a94a132db8b-istio-token\") pod \"e2e-ait-0cf92335-openshift-default-547796bdc7-54md6\" (UID: \"4217af42-8645-4d40-a131-7a94a132db8b\") " pod="openshift-ingress/e2e-ait-0cf92335-openshift-default-547796bdc7-54md6" Jul 23 22:21:18.868837 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:21:18.868710 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4217af42-8645-4d40-a131-7a94a132db8b-istio-envoy\") pod \"e2e-ait-0cf92335-openshift-default-547796bdc7-54md6\" (UID: \"4217af42-8645-4d40-a131-7a94a132db8b\") " pod="openshift-ingress/e2e-ait-0cf92335-openshift-default-547796bdc7-54md6" Jul 23 22:21:18.868837 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:21:18.868734 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-727jq\" (UniqueName: \"kubernetes.io/projected/4217af42-8645-4d40-a131-7a94a132db8b-kube-api-access-727jq\") pod \"e2e-ait-0cf92335-openshift-default-547796bdc7-54md6\" (UID: \"4217af42-8645-4d40-a131-7a94a132db8b\") " pod="openshift-ingress/e2e-ait-0cf92335-openshift-default-547796bdc7-54md6" Jul 23 22:21:18.868837 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:21:18.868770 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4217af42-8645-4d40-a131-7a94a132db8b-workload-certs\") pod \"e2e-ait-0cf92335-openshift-default-547796bdc7-54md6\" (UID: \"4217af42-8645-4d40-a131-7a94a132db8b\") " pod="openshift-ingress/e2e-ait-0cf92335-openshift-default-547796bdc7-54md6" Jul 23 22:21:18.868837 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:21:18.868814 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4217af42-8645-4d40-a131-7a94a132db8b-istio-data\") pod \"e2e-ait-0cf92335-openshift-default-547796bdc7-54md6\" (UID: \"4217af42-8645-4d40-a131-7a94a132db8b\") " pod="openshift-ingress/e2e-ait-0cf92335-openshift-default-547796bdc7-54md6" Jul 23 22:21:18.869131 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:21:18.868839 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4217af42-8645-4d40-a131-7a94a132db8b-istiod-ca-cert\") pod \"e2e-ait-0cf92335-openshift-default-547796bdc7-54md6\" (UID: \"4217af42-8645-4d40-a131-7a94a132db8b\") " pod="openshift-ingress/e2e-ait-0cf92335-openshift-default-547796bdc7-54md6" Jul 23 22:21:18.869131 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:21:18.868878 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4217af42-8645-4d40-a131-7a94a132db8b-istio-podinfo\") pod \"e2e-ait-0cf92335-openshift-default-547796bdc7-54md6\" (UID: \"4217af42-8645-4d40-a131-7a94a132db8b\") " pod="openshift-ingress/e2e-ait-0cf92335-openshift-default-547796bdc7-54md6" Jul 23 22:21:18.869131 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:21:18.868915 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4217af42-8645-4d40-a131-7a94a132db8b-credential-socket\") pod \"e2e-ait-0cf92335-openshift-default-547796bdc7-54md6\" (UID: \"4217af42-8645-4d40-a131-7a94a132db8b\") " pod="openshift-ingress/e2e-ait-0cf92335-openshift-default-547796bdc7-54md6" Jul 23 22:21:18.869131 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:21:18.869107 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4217af42-8645-4d40-a131-7a94a132db8b-workload-socket\") pod \"e2e-ait-0cf92335-openshift-default-547796bdc7-54md6\" (UID: \"4217af42-8645-4d40-a131-7a94a132db8b\") " pod="openshift-ingress/e2e-ait-0cf92335-openshift-default-547796bdc7-54md6" Jul 23 22:21:18.869328 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:21:18.869234 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4217af42-8645-4d40-a131-7a94a132db8b-workload-certs\") pod \"e2e-ait-0cf92335-openshift-default-547796bdc7-54md6\" (UID: \"4217af42-8645-4d40-a131-7a94a132db8b\") " pod="openshift-ingress/e2e-ait-0cf92335-openshift-default-547796bdc7-54md6" Jul 23 22:21:18.869328 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:21:18.869266 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4217af42-8645-4d40-a131-7a94a132db8b-istio-data\") pod \"e2e-ait-0cf92335-openshift-default-547796bdc7-54md6\" (UID: \"4217af42-8645-4d40-a131-7a94a132db8b\") " pod="openshift-ingress/e2e-ait-0cf92335-openshift-default-547796bdc7-54md6" Jul 23 22:21:18.869328 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:21:18.869284 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4217af42-8645-4d40-a131-7a94a132db8b-credential-socket\") pod \"e2e-ait-0cf92335-openshift-default-547796bdc7-54md6\" (UID: \"4217af42-8645-4d40-a131-7a94a132db8b\") " pod="openshift-ingress/e2e-ait-0cf92335-openshift-default-547796bdc7-54md6" Jul 23 22:21:18.869602 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:21:18.869580 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4217af42-8645-4d40-a131-7a94a132db8b-istiod-ca-cert\") pod \"e2e-ait-0cf92335-openshift-default-547796bdc7-54md6\" (UID: \"4217af42-8645-4d40-a131-7a94a132db8b\") " pod="openshift-ingress/e2e-ait-0cf92335-openshift-default-547796bdc7-54md6" Jul 23 22:21:18.871143 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:21:18.871121 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4217af42-8645-4d40-a131-7a94a132db8b-istio-envoy\") pod \"e2e-ait-0cf92335-openshift-default-547796bdc7-54md6\" (UID: \"4217af42-8645-4d40-a131-7a94a132db8b\") " pod="openshift-ingress/e2e-ait-0cf92335-openshift-default-547796bdc7-54md6" Jul 23 22:21:18.871356 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:21:18.871339 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4217af42-8645-4d40-a131-7a94a132db8b-istio-podinfo\") pod \"e2e-ait-0cf92335-openshift-default-547796bdc7-54md6\" (UID: \"4217af42-8645-4d40-a131-7a94a132db8b\") " pod="openshift-ingress/e2e-ait-0cf92335-openshift-default-547796bdc7-54md6" Jul 23 22:21:18.877389 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:21:18.877365 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4217af42-8645-4d40-a131-7a94a132db8b-istio-token\") pod \"e2e-ait-0cf92335-openshift-default-547796bdc7-54md6\" (UID: \"4217af42-8645-4d40-a131-7a94a132db8b\") " pod="openshift-ingress/e2e-ait-0cf92335-openshift-default-547796bdc7-54md6" Jul 23 22:21:18.878136 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:21:18.878119 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-727jq\" (UniqueName: \"kubernetes.io/projected/4217af42-8645-4d40-a131-7a94a132db8b-kube-api-access-727jq\") pod \"e2e-ait-0cf92335-openshift-default-547796bdc7-54md6\" (UID: \"4217af42-8645-4d40-a131-7a94a132db8b\") " pod="openshift-ingress/e2e-ait-0cf92335-openshift-default-547796bdc7-54md6" Jul 23 22:21:18.978899 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:21:18.978835 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-0cf92335-openshift-default-547796bdc7-54md6" Jul 23 22:21:19.112785 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:21:19.112758 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-0cf92335-openshift-default-547796bdc7-54md6"] Jul 23 22:21:19.114249 ip-10-0-136-157 kubenswrapper[2578]: W0723 22:21:19.114212 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4217af42_8645_4d40_a131_7a94a132db8b.slice/crio-bb3d082decabda18abf70f0007a3f41cffa5e0e6e298184cb45202ae036d1bdc WatchSource:0}: Error finding container bb3d082decabda18abf70f0007a3f41cffa5e0e6e298184cb45202ae036d1bdc: Status 404 returned error can't find the container with id bb3d082decabda18abf70f0007a3f41cffa5e0e6e298184cb45202ae036d1bdc Jul 23 22:21:19.116292 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:21:19.116263 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 23 22:21:19.116371 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:21:19.116327 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 23 22:21:19.116371 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:21:19.116354 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 23 22:21:19.639860 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:21:19.639828 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0cf92335-openshift-default-547796bdc7-54md6" event={"ID":"4217af42-8645-4d40-a131-7a94a132db8b","Type":"ContainerStarted","Data":"23da22923ff991029c6796ce9784e6f2735fbc6cd33b2dbe8c71c1d3e454860e"} Jul 23 22:21:19.640020 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:21:19.639878 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0cf92335-openshift-default-547796bdc7-54md6" event={"ID":"4217af42-8645-4d40-a131-7a94a132db8b","Type":"ContainerStarted","Data":"bb3d082decabda18abf70f0007a3f41cffa5e0e6e298184cb45202ae036d1bdc"} Jul 23 22:21:19.669328 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:21:19.669280 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-0cf92335-openshift-default-547796bdc7-54md6" podStartSLOduration=1.669250757 podStartE2EDuration="1.669250757s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 22:21:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 22:21:19.66607162 +0000 UTC m=+1781.907979499" watchObservedRunningTime="2026-07-23 22:21:19.669250757 +0000 UTC m=+1781.911158631" Jul 23 22:21:19.979218 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:21:19.979122 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-0cf92335-openshift-default-547796bdc7-54md6" Jul 23 22:21:20.985447 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:21:20.985417 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-0cf92335-openshift-default-547796bdc7-54md6" Jul 23 22:21:21.650231 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:21:21.650204 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-0cf92335-openshift-default-547796bdc7-54md6" Jul 23 22:21:21.651136 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:21:21.651115 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-0cf92335-openshift-default-547796bdc7-54md6" Jul 23 22:21:36.652354 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:21:36.652315 2578 patch_prober.go:28] interesting pod/e2e-ait-0cf92335-openshift-default-547796bdc7-54md6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body= Jul 23 22:21:36.652861 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:21:36.652389 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-0cf92335-openshift-default-547796bdc7-54md6" podUID="4217af42-8645-4d40-a131-7a94a132db8b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" Jul 23 22:21:36.724318 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:21:36.724287 2578 generic.go:356] "Generic (PLEG): container finished" podID="4217af42-8645-4d40-a131-7a94a132db8b" containerID="23da22923ff991029c6796ce9784e6f2735fbc6cd33b2dbe8c71c1d3e454860e" exitCode=0 Jul 23 22:21:36.724470 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:21:36.724360 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0cf92335-openshift-default-547796bdc7-54md6" event={"ID":"4217af42-8645-4d40-a131-7a94a132db8b","Type":"ContainerDied","Data":"23da22923ff991029c6796ce9784e6f2735fbc6cd33b2dbe8c71c1d3e454860e"} Jul 23 22:21:36.724774 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:21:36.724760 2578 scope.go:117] "RemoveContainer" containerID="23da22923ff991029c6796ce9784e6f2735fbc6cd33b2dbe8c71c1d3e454860e" Jul 23 22:21:36.725738 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:21:36.725711 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 23 22:21:36.725848 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:21:36.725795 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 23 22:21:36.725848 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:21:36.725840 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 23 22:21:37.739573 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:21:37.739284 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0cf92335-openshift-default-547796bdc7-54md6" event={"ID":"4217af42-8645-4d40-a131-7a94a132db8b","Type":"ContainerStarted","Data":"4eda4c5a7d56056e571d2eb53fed9043a21e6152f416bf89dbfdc9274c25d0c7"} Jul 23 22:21:37.979306 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:21:37.979268 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-0cf92335-openshift-default-547796bdc7-54md6" Jul 23 22:21:37.979575 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:21:37.979515 2578 patch_prober.go:28] interesting pod/e2e-ait-0cf92335-openshift-default-547796bdc7-54md6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body= Jul 23 22:21:37.979681 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:21:37.979608 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-0cf92335-openshift-default-547796bdc7-54md6" podUID="4217af42-8645-4d40-a131-7a94a132db8b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" Jul 23 22:21:38.679097 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:21:38.679069 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7jkjm_7342a9f6-72b1-4e3c-8d50-0b0326454a86/ovn-acl-logging/0.log" Jul 23 22:21:38.681885 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:21:38.681860 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7jkjm_7342a9f6-72b1-4e3c-8d50-0b0326454a86/ovn-acl-logging/0.log" Jul 23 22:21:38.979940 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:21:38.979845 2578 patch_prober.go:28] interesting pod/e2e-ait-0cf92335-openshift-default-547796bdc7-54md6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body= Jul 23 22:21:38.979940 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:21:38.979919 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-0cf92335-openshift-default-547796bdc7-54md6" podUID="4217af42-8645-4d40-a131-7a94a132db8b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" Jul 23 22:21:39.979924 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:21:39.979886 2578 patch_prober.go:28] interesting pod/e2e-ait-0cf92335-openshift-default-547796bdc7-54md6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body= Jul 23 22:21:39.980092 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:21:39.979947 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-0cf92335-openshift-default-547796bdc7-54md6" podUID="4217af42-8645-4d40-a131-7a94a132db8b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" Jul 23 22:21:40.979369 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:21:40.979335 2578 patch_prober.go:28] interesting pod/e2e-ait-0cf92335-openshift-default-547796bdc7-54md6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" start-of-body= Jul 23 22:21:40.979576 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:21:40.979392 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-0cf92335-openshift-default-547796bdc7-54md6" podUID="4217af42-8645-4d40-a131-7a94a132db8b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.41:15021/healthz/ready\": dial tcp 10.133.0.41:15021: connect: connection refused" Jul 23 22:21:41.983066 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:21:41.983039 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-0cf92335-openshift-default-547796bdc7-54md6" Jul 23 22:21:41.983438 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:21:41.983280 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-0cf92335-openshift-default-547796bdc7-54md6" Jul 23 22:21:41.983968 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:21:41.983953 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-0cf92335-openshift-default-547796bdc7-54md6" Jul 23 22:22:03.282515 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:22:03.282484 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-0cf92335-openshift-default-547796bdc7-54md6"] Jul 23 22:22:03.283034 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:22:03.282740 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-0cf92335-openshift-default-547796bdc7-54md6" podUID="4217af42-8645-4d40-a131-7a94a132db8b" containerName="istio-proxy" containerID="cri-o://4eda4c5a7d56056e571d2eb53fed9043a21e6152f416bf89dbfdc9274c25d0c7" gracePeriod=30 Jul 23 22:22:08.537998 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:22:08.537973 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-0cf92335-openshift-default-547796bdc7-54md6" Jul 23 22:22:08.714319 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:22:08.714293 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4217af42-8645-4d40-a131-7a94a132db8b-workload-socket\") pod \"4217af42-8645-4d40-a131-7a94a132db8b\" (UID: \"4217af42-8645-4d40-a131-7a94a132db8b\") " Jul 23 22:22:08.714481 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:22:08.714336 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4217af42-8645-4d40-a131-7a94a132db8b-istio-data\") pod \"4217af42-8645-4d40-a131-7a94a132db8b\" (UID: \"4217af42-8645-4d40-a131-7a94a132db8b\") " Jul 23 22:22:08.714481 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:22:08.714377 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-727jq\" (UniqueName: \"kubernetes.io/projected/4217af42-8645-4d40-a131-7a94a132db8b-kube-api-access-727jq\") pod \"4217af42-8645-4d40-a131-7a94a132db8b\" (UID: \"4217af42-8645-4d40-a131-7a94a132db8b\") " Jul 23 22:22:08.714481 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:22:08.714401 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4217af42-8645-4d40-a131-7a94a132db8b-istio-token\") pod \"4217af42-8645-4d40-a131-7a94a132db8b\" (UID: \"4217af42-8645-4d40-a131-7a94a132db8b\") " Jul 23 22:22:08.714481 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:22:08.714436 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4217af42-8645-4d40-a131-7a94a132db8b-istio-podinfo\") pod \"4217af42-8645-4d40-a131-7a94a132db8b\" (UID: \"4217af42-8645-4d40-a131-7a94a132db8b\") " Jul 23 22:22:08.714720 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:22:08.714485 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4217af42-8645-4d40-a131-7a94a132db8b-istiod-ca-cert\") pod \"4217af42-8645-4d40-a131-7a94a132db8b\" (UID: \"4217af42-8645-4d40-a131-7a94a132db8b\") " Jul 23 22:22:08.714720 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:22:08.714509 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4217af42-8645-4d40-a131-7a94a132db8b-credential-socket\") pod \"4217af42-8645-4d40-a131-7a94a132db8b\" (UID: \"4217af42-8645-4d40-a131-7a94a132db8b\") " Jul 23 22:22:08.714720 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:22:08.714553 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4217af42-8645-4d40-a131-7a94a132db8b-workload-certs\") pod \"4217af42-8645-4d40-a131-7a94a132db8b\" (UID: \"4217af42-8645-4d40-a131-7a94a132db8b\") " Jul 23 22:22:08.714720 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:22:08.714597 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4217af42-8645-4d40-a131-7a94a132db8b-istio-envoy\") pod \"4217af42-8645-4d40-a131-7a94a132db8b\" (UID: \"4217af42-8645-4d40-a131-7a94a132db8b\") " Jul 23 22:22:08.714907 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:22:08.714842 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4217af42-8645-4d40-a131-7a94a132db8b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "4217af42-8645-4d40-a131-7a94a132db8b" (UID: "4217af42-8645-4d40-a131-7a94a132db8b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 22:22:08.714907 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:22:08.714853 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4217af42-8645-4d40-a131-7a94a132db8b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "4217af42-8645-4d40-a131-7a94a132db8b" (UID: "4217af42-8645-4d40-a131-7a94a132db8b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 22:22:08.714979 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:22:08.714893 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4217af42-8645-4d40-a131-7a94a132db8b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "4217af42-8645-4d40-a131-7a94a132db8b" (UID: "4217af42-8645-4d40-a131-7a94a132db8b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 22:22:08.715315 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:22:08.715118 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4217af42-8645-4d40-a131-7a94a132db8b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "4217af42-8645-4d40-a131-7a94a132db8b" (UID: "4217af42-8645-4d40-a131-7a94a132db8b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 22:22:08.716008 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:22:08.715981 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4217af42-8645-4d40-a131-7a94a132db8b-istio-data" (OuterVolumeSpecName: "istio-data") pod "4217af42-8645-4d40-a131-7a94a132db8b" (UID: "4217af42-8645-4d40-a131-7a94a132db8b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 22:22:08.716743 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:22:08.716719 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4217af42-8645-4d40-a131-7a94a132db8b-istio-token" (OuterVolumeSpecName: "istio-token") pod "4217af42-8645-4d40-a131-7a94a132db8b" (UID: "4217af42-8645-4d40-a131-7a94a132db8b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 22:22:08.716950 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:22:08.716917 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4217af42-8645-4d40-a131-7a94a132db8b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "4217af42-8645-4d40-a131-7a94a132db8b" (UID: "4217af42-8645-4d40-a131-7a94a132db8b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 22:22:08.716950 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:22:08.716933 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4217af42-8645-4d40-a131-7a94a132db8b-kube-api-access-727jq" (OuterVolumeSpecName: "kube-api-access-727jq") pod "4217af42-8645-4d40-a131-7a94a132db8b" (UID: "4217af42-8645-4d40-a131-7a94a132db8b"). InnerVolumeSpecName "kube-api-access-727jq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 22:22:08.716950 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:22:08.716937 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4217af42-8645-4d40-a131-7a94a132db8b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "4217af42-8645-4d40-a131-7a94a132db8b" (UID: "4217af42-8645-4d40-a131-7a94a132db8b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 23 22:22:08.815577 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:22:08.815553 2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4217af42-8645-4d40-a131-7a94a132db8b-istio-token\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\"" Jul 23 22:22:08.815577 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:22:08.815574 2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4217af42-8645-4d40-a131-7a94a132db8b-istio-podinfo\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\"" Jul 23 22:22:08.815692 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:22:08.815583 2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4217af42-8645-4d40-a131-7a94a132db8b-istiod-ca-cert\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\"" Jul 23 22:22:08.815692 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:22:08.815590 2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4217af42-8645-4d40-a131-7a94a132db8b-credential-socket\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\"" Jul 23 22:22:08.815692 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:22:08.815599 2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4217af42-8645-4d40-a131-7a94a132db8b-workload-certs\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\"" Jul 23 22:22:08.815692 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:22:08.815607 2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4217af42-8645-4d40-a131-7a94a132db8b-istio-envoy\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\"" Jul 23 22:22:08.815692 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:22:08.815614 2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4217af42-8645-4d40-a131-7a94a132db8b-workload-socket\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\"" Jul 23 22:22:08.815692 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:22:08.815622 2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4217af42-8645-4d40-a131-7a94a132db8b-istio-data\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\"" Jul 23 22:22:08.815692 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:22:08.815630 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-727jq\" (UniqueName: \"kubernetes.io/projected/4217af42-8645-4d40-a131-7a94a132db8b-kube-api-access-727jq\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\"" Jul 23 22:22:08.916702 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:22:08.916672 2578 generic.go:356] "Generic (PLEG): container finished" podID="4217af42-8645-4d40-a131-7a94a132db8b" containerID="4eda4c5a7d56056e571d2eb53fed9043a21e6152f416bf89dbfdc9274c25d0c7" exitCode=0 Jul 23 22:22:08.916802 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:22:08.916717 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-0cf92335-openshift-default-547796bdc7-54md6" Jul 23 22:22:08.916802 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:22:08.916757 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0cf92335-openshift-default-547796bdc7-54md6" event={"ID":"4217af42-8645-4d40-a131-7a94a132db8b","Type":"ContainerDied","Data":"4eda4c5a7d56056e571d2eb53fed9043a21e6152f416bf89dbfdc9274c25d0c7"} Jul 23 22:22:08.916877 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:22:08.916816 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0cf92335-openshift-default-547796bdc7-54md6" event={"ID":"4217af42-8645-4d40-a131-7a94a132db8b","Type":"ContainerDied","Data":"bb3d082decabda18abf70f0007a3f41cffa5e0e6e298184cb45202ae036d1bdc"} Jul 23 22:22:08.916877 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:22:08.916840 2578 scope.go:117] "RemoveContainer" containerID="4eda4c5a7d56056e571d2eb53fed9043a21e6152f416bf89dbfdc9274c25d0c7" Jul 23 22:22:08.925882 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:22:08.925867 2578 scope.go:117] "RemoveContainer" containerID="23da22923ff991029c6796ce9784e6f2735fbc6cd33b2dbe8c71c1d3e454860e" Jul 23 22:22:08.933905 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:22:08.933888 2578 scope.go:117] "RemoveContainer" containerID="4eda4c5a7d56056e571d2eb53fed9043a21e6152f416bf89dbfdc9274c25d0c7" Jul 23 22:22:08.934167 ip-10-0-136-157 kubenswrapper[2578]: E0723 22:22:08.934145 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4eda4c5a7d56056e571d2eb53fed9043a21e6152f416bf89dbfdc9274c25d0c7\": container with ID starting with 4eda4c5a7d56056e571d2eb53fed9043a21e6152f416bf89dbfdc9274c25d0c7 not found: ID does not exist" containerID="4eda4c5a7d56056e571d2eb53fed9043a21e6152f416bf89dbfdc9274c25d0c7" Jul 23 22:22:08.934224 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:22:08.934180 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4eda4c5a7d56056e571d2eb53fed9043a21e6152f416bf89dbfdc9274c25d0c7"} err="failed to get container status \"4eda4c5a7d56056e571d2eb53fed9043a21e6152f416bf89dbfdc9274c25d0c7\": rpc error: code = NotFound desc = could not find container \"4eda4c5a7d56056e571d2eb53fed9043a21e6152f416bf89dbfdc9274c25d0c7\": container with ID starting with 4eda4c5a7d56056e571d2eb53fed9043a21e6152f416bf89dbfdc9274c25d0c7 not found: ID does not exist" Jul 23 22:22:08.934224 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:22:08.934203 2578 scope.go:117] "RemoveContainer" containerID="23da22923ff991029c6796ce9784e6f2735fbc6cd33b2dbe8c71c1d3e454860e" Jul 23 22:22:08.934437 ip-10-0-136-157 kubenswrapper[2578]: E0723 22:22:08.934418 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"23da22923ff991029c6796ce9784e6f2735fbc6cd33b2dbe8c71c1d3e454860e\": container with ID starting with 23da22923ff991029c6796ce9784e6f2735fbc6cd33b2dbe8c71c1d3e454860e not found: ID does not exist" containerID="23da22923ff991029c6796ce9784e6f2735fbc6cd33b2dbe8c71c1d3e454860e" Jul 23 22:22:08.934480 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:22:08.934444 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"23da22923ff991029c6796ce9784e6f2735fbc6cd33b2dbe8c71c1d3e454860e"} err="failed to get container status \"23da22923ff991029c6796ce9784e6f2735fbc6cd33b2dbe8c71c1d3e454860e\": rpc error: code = NotFound desc = could not find container \"23da22923ff991029c6796ce9784e6f2735fbc6cd33b2dbe8c71c1d3e454860e\": container with ID starting with 23da22923ff991029c6796ce9784e6f2735fbc6cd33b2dbe8c71c1d3e454860e not found: ID does not exist" Jul 23 22:22:08.944478 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:22:08.944455 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-0cf92335-openshift-default-547796bdc7-54md6"] Jul 23 22:22:08.948167 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:22:08.948144 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-0cf92335-openshift-default-547796bdc7-54md6"] Jul 23 22:22:10.250588 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:22:10.250554 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4217af42-8645-4d40-a131-7a94a132db8b" path="/var/lib/kubelet/pods/4217af42-8645-4d40-a131-7a94a132db8b/volumes" Jul 23 22:26:38.719756 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:26:38.719718 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7jkjm_7342a9f6-72b1-4e3c-8d50-0b0326454a86/ovn-acl-logging/0.log" Jul 23 22:26:38.723651 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:26:38.723628 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7jkjm_7342a9f6-72b1-4e3c-8d50-0b0326454a86/ovn-acl-logging/0.log" Jul 23 22:28:36.525560 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:28:36.525512 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-8a3e27-openshift-default-76fcf6495f-4l6r5"] Jul 23 22:28:36.526164 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:28:36.526071 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4217af42-8645-4d40-a131-7a94a132db8b" containerName="istio-proxy" Jul 23 22:28:36.526164 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:28:36.526090 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="4217af42-8645-4d40-a131-7a94a132db8b" containerName="istio-proxy" Jul 23 22:28:36.526164 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:28:36.526113 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4217af42-8645-4d40-a131-7a94a132db8b" containerName="istio-proxy" Jul 23 22:28:36.526164 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:28:36.526120 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="4217af42-8645-4d40-a131-7a94a132db8b" containerName="istio-proxy" Jul 23 22:28:36.526418 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:28:36.526214 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="4217af42-8645-4d40-a131-7a94a132db8b" containerName="istio-proxy" Jul 23 22:28:36.526418 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:28:36.526231 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="4217af42-8645-4d40-a131-7a94a132db8b" containerName="istio-proxy" Jul 23 22:28:36.529347 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:28:36.529326 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-8a3e27-openshift-default-76fcf6495f-4l6r5" Jul 23 22:28:36.531869 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:28:36.531848 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-8a3e27-openshift-default-dockercfg-v4zrr\"" Jul 23 22:28:36.542361 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:28:36.542336 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-8a3e27-openshift-default-76fcf6495f-4l6r5"] Jul 23 22:28:36.616030 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:28:36.616000 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/be46532d-4c4e-4092-b94b-31d8f3f84ce3-istio-token\") pod \"e2e-inf-b-8a3e27-openshift-default-76fcf6495f-4l6r5\" (UID: \"be46532d-4c4e-4092-b94b-31d8f3f84ce3\") " pod="openshift-ingress/e2e-inf-b-8a3e27-openshift-default-76fcf6495f-4l6r5" Jul 23 22:28:36.616213 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:28:36.616059 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/be46532d-4c4e-4092-b94b-31d8f3f84ce3-workload-certs\") pod \"e2e-inf-b-8a3e27-openshift-default-76fcf6495f-4l6r5\" (UID: \"be46532d-4c4e-4092-b94b-31d8f3f84ce3\") " pod="openshift-ingress/e2e-inf-b-8a3e27-openshift-default-76fcf6495f-4l6r5" Jul 23 22:28:36.616213 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:28:36.616151 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/be46532d-4c4e-4092-b94b-31d8f3f84ce3-istio-data\") pod \"e2e-inf-b-8a3e27-openshift-default-76fcf6495f-4l6r5\" (UID: \"be46532d-4c4e-4092-b94b-31d8f3f84ce3\") " pod="openshift-ingress/e2e-inf-b-8a3e27-openshift-default-76fcf6495f-4l6r5" Jul 23 22:28:36.616347 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:28:36.616234 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/be46532d-4c4e-4092-b94b-31d8f3f84ce3-istio-envoy\") pod \"e2e-inf-b-8a3e27-openshift-default-76fcf6495f-4l6r5\" (UID: \"be46532d-4c4e-4092-b94b-31d8f3f84ce3\") " pod="openshift-ingress/e2e-inf-b-8a3e27-openshift-default-76fcf6495f-4l6r5" Jul 23 22:28:36.616347 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:28:36.616288 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/be46532d-4c4e-4092-b94b-31d8f3f84ce3-credential-socket\") pod \"e2e-inf-b-8a3e27-openshift-default-76fcf6495f-4l6r5\" (UID: \"be46532d-4c4e-4092-b94b-31d8f3f84ce3\") " pod="openshift-ingress/e2e-inf-b-8a3e27-openshift-default-76fcf6495f-4l6r5" Jul 23 22:28:36.616347 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:28:36.616318 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/be46532d-4c4e-4092-b94b-31d8f3f84ce3-istiod-ca-cert\") pod \"e2e-inf-b-8a3e27-openshift-default-76fcf6495f-4l6r5\" (UID: \"be46532d-4c4e-4092-b94b-31d8f3f84ce3\") " pod="openshift-ingress/e2e-inf-b-8a3e27-openshift-default-76fcf6495f-4l6r5" Jul 23 22:28:36.616473 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:28:36.616383 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/be46532d-4c4e-4092-b94b-31d8f3f84ce3-istio-podinfo\") pod \"e2e-inf-b-8a3e27-openshift-default-76fcf6495f-4l6r5\" (UID: \"be46532d-4c4e-4092-b94b-31d8f3f84ce3\") " pod="openshift-ingress/e2e-inf-b-8a3e27-openshift-default-76fcf6495f-4l6r5" Jul 23 22:28:36.616473 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:28:36.616438 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fwfh8\" (UniqueName: \"kubernetes.io/projected/be46532d-4c4e-4092-b94b-31d8f3f84ce3-kube-api-access-fwfh8\") pod \"e2e-inf-b-8a3e27-openshift-default-76fcf6495f-4l6r5\" (UID: \"be46532d-4c4e-4092-b94b-31d8f3f84ce3\") " pod="openshift-ingress/e2e-inf-b-8a3e27-openshift-default-76fcf6495f-4l6r5" Jul 23 22:28:36.616633 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:28:36.616470 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/be46532d-4c4e-4092-b94b-31d8f3f84ce3-workload-socket\") pod \"e2e-inf-b-8a3e27-openshift-default-76fcf6495f-4l6r5\" (UID: \"be46532d-4c4e-4092-b94b-31d8f3f84ce3\") " pod="openshift-ingress/e2e-inf-b-8a3e27-openshift-default-76fcf6495f-4l6r5" Jul 23 22:28:36.717770 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:28:36.717738 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/be46532d-4c4e-4092-b94b-31d8f3f84ce3-istio-token\") pod \"e2e-inf-b-8a3e27-openshift-default-76fcf6495f-4l6r5\" (UID: \"be46532d-4c4e-4092-b94b-31d8f3f84ce3\") " pod="openshift-ingress/e2e-inf-b-8a3e27-openshift-default-76fcf6495f-4l6r5" Jul 23 22:28:36.717951 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:28:36.717794 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/be46532d-4c4e-4092-b94b-31d8f3f84ce3-workload-certs\") pod \"e2e-inf-b-8a3e27-openshift-default-76fcf6495f-4l6r5\" (UID: \"be46532d-4c4e-4092-b94b-31d8f3f84ce3\") " pod="openshift-ingress/e2e-inf-b-8a3e27-openshift-default-76fcf6495f-4l6r5" Jul 23 22:28:36.717951 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:28:36.717818 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/be46532d-4c4e-4092-b94b-31d8f3f84ce3-istio-data\") pod \"e2e-inf-b-8a3e27-openshift-default-76fcf6495f-4l6r5\" (UID: \"be46532d-4c4e-4092-b94b-31d8f3f84ce3\") " pod="openshift-ingress/e2e-inf-b-8a3e27-openshift-default-76fcf6495f-4l6r5" Jul 23 22:28:36.717951 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:28:36.717861 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/be46532d-4c4e-4092-b94b-31d8f3f84ce3-istio-envoy\") pod \"e2e-inf-b-8a3e27-openshift-default-76fcf6495f-4l6r5\" (UID: \"be46532d-4c4e-4092-b94b-31d8f3f84ce3\") " pod="openshift-ingress/e2e-inf-b-8a3e27-openshift-default-76fcf6495f-4l6r5" Jul 23 22:28:36.717951 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:28:36.717895 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/be46532d-4c4e-4092-b94b-31d8f3f84ce3-credential-socket\") pod \"e2e-inf-b-8a3e27-openshift-default-76fcf6495f-4l6r5\" (UID: \"be46532d-4c4e-4092-b94b-31d8f3f84ce3\") " pod="openshift-ingress/e2e-inf-b-8a3e27-openshift-default-76fcf6495f-4l6r5" Jul 23 22:28:36.717951 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:28:36.717919 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/be46532d-4c4e-4092-b94b-31d8f3f84ce3-istiod-ca-cert\") pod \"e2e-inf-b-8a3e27-openshift-default-76fcf6495f-4l6r5\" (UID: \"be46532d-4c4e-4092-b94b-31d8f3f84ce3\") " pod="openshift-ingress/e2e-inf-b-8a3e27-openshift-default-76fcf6495f-4l6r5" Jul 23 22:28:36.718177 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:28:36.717955 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/be46532d-4c4e-4092-b94b-31d8f3f84ce3-istio-podinfo\") pod \"e2e-inf-b-8a3e27-openshift-default-76fcf6495f-4l6r5\" (UID: \"be46532d-4c4e-4092-b94b-31d8f3f84ce3\") " pod="openshift-ingress/e2e-inf-b-8a3e27-openshift-default-76fcf6495f-4l6r5" Jul 23 22:28:36.718177 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:28:36.717979 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fwfh8\" (UniqueName: \"kubernetes.io/projected/be46532d-4c4e-4092-b94b-31d8f3f84ce3-kube-api-access-fwfh8\") pod \"e2e-inf-b-8a3e27-openshift-default-76fcf6495f-4l6r5\" (UID: \"be46532d-4c4e-4092-b94b-31d8f3f84ce3\") " pod="openshift-ingress/e2e-inf-b-8a3e27-openshift-default-76fcf6495f-4l6r5" Jul 23 22:28:36.718177 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:28:36.718014 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/be46532d-4c4e-4092-b94b-31d8f3f84ce3-workload-socket\") pod \"e2e-inf-b-8a3e27-openshift-default-76fcf6495f-4l6r5\" (UID: \"be46532d-4c4e-4092-b94b-31d8f3f84ce3\") " pod="openshift-ingress/e2e-inf-b-8a3e27-openshift-default-76fcf6495f-4l6r5" Jul 23 22:28:36.718279 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:28:36.718227 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/be46532d-4c4e-4092-b94b-31d8f3f84ce3-workload-certs\") pod \"e2e-inf-b-8a3e27-openshift-default-76fcf6495f-4l6r5\" (UID: \"be46532d-4c4e-4092-b94b-31d8f3f84ce3\") " pod="openshift-ingress/e2e-inf-b-8a3e27-openshift-default-76fcf6495f-4l6r5" Jul 23 22:28:36.718317 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:28:36.718279 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/be46532d-4c4e-4092-b94b-31d8f3f84ce3-istio-data\") pod \"e2e-inf-b-8a3e27-openshift-default-76fcf6495f-4l6r5\" (UID: \"be46532d-4c4e-4092-b94b-31d8f3f84ce3\") " pod="openshift-ingress/e2e-inf-b-8a3e27-openshift-default-76fcf6495f-4l6r5" Jul 23 22:28:36.718386 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:28:36.718364 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/be46532d-4c4e-4092-b94b-31d8f3f84ce3-credential-socket\") pod \"e2e-inf-b-8a3e27-openshift-default-76fcf6495f-4l6r5\" (UID: \"be46532d-4c4e-4092-b94b-31d8f3f84ce3\") " pod="openshift-ingress/e2e-inf-b-8a3e27-openshift-default-76fcf6495f-4l6r5" Jul 23 22:28:36.718602 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:28:36.718580 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/be46532d-4c4e-4092-b94b-31d8f3f84ce3-istiod-ca-cert\") pod \"e2e-inf-b-8a3e27-openshift-default-76fcf6495f-4l6r5\" (UID: \"be46532d-4c4e-4092-b94b-31d8f3f84ce3\") " pod="openshift-ingress/e2e-inf-b-8a3e27-openshift-default-76fcf6495f-4l6r5" Jul 23 22:28:36.718602 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:28:36.718596 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/be46532d-4c4e-4092-b94b-31d8f3f84ce3-workload-socket\") pod \"e2e-inf-b-8a3e27-openshift-default-76fcf6495f-4l6r5\" (UID: \"be46532d-4c4e-4092-b94b-31d8f3f84ce3\") " pod="openshift-ingress/e2e-inf-b-8a3e27-openshift-default-76fcf6495f-4l6r5" Jul 23 22:28:36.720392 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:28:36.720373 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/be46532d-4c4e-4092-b94b-31d8f3f84ce3-istio-envoy\") pod \"e2e-inf-b-8a3e27-openshift-default-76fcf6495f-4l6r5\" (UID: \"be46532d-4c4e-4092-b94b-31d8f3f84ce3\") " pod="openshift-ingress/e2e-inf-b-8a3e27-openshift-default-76fcf6495f-4l6r5" Jul 23 22:28:36.720555 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:28:36.720524 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/be46532d-4c4e-4092-b94b-31d8f3f84ce3-istio-podinfo\") pod \"e2e-inf-b-8a3e27-openshift-default-76fcf6495f-4l6r5\" (UID: \"be46532d-4c4e-4092-b94b-31d8f3f84ce3\") " pod="openshift-ingress/e2e-inf-b-8a3e27-openshift-default-76fcf6495f-4l6r5" Jul 23 22:28:36.725523 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:28:36.725499 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/be46532d-4c4e-4092-b94b-31d8f3f84ce3-istio-token\") pod \"e2e-inf-b-8a3e27-openshift-default-76fcf6495f-4l6r5\" (UID: \"be46532d-4c4e-4092-b94b-31d8f3f84ce3\") " pod="openshift-ingress/e2e-inf-b-8a3e27-openshift-default-76fcf6495f-4l6r5" Jul 23 22:28:36.725617 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:28:36.725559 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fwfh8\" (UniqueName: \"kubernetes.io/projected/be46532d-4c4e-4092-b94b-31d8f3f84ce3-kube-api-access-fwfh8\") pod \"e2e-inf-b-8a3e27-openshift-default-76fcf6495f-4l6r5\" (UID: \"be46532d-4c4e-4092-b94b-31d8f3f84ce3\") " pod="openshift-ingress/e2e-inf-b-8a3e27-openshift-default-76fcf6495f-4l6r5" Jul 23 22:28:36.839520 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:28:36.839482 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-8a3e27-openshift-default-76fcf6495f-4l6r5" Jul 23 22:28:36.975918 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:28:36.975894 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-8a3e27-openshift-default-76fcf6495f-4l6r5"] Jul 23 22:28:36.977499 ip-10-0-136-157 kubenswrapper[2578]: W0723 22:28:36.977473 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe46532d_4c4e_4092_b94b_31d8f3f84ce3.slice/crio-e62df3f1b136487467816a9cb440ca3552f75624d3a0c9c33c5cb84d52ea021a WatchSource:0}: Error finding container e62df3f1b136487467816a9cb440ca3552f75624d3a0c9c33c5cb84d52ea021a: Status 404 returned error can't find the container with id e62df3f1b136487467816a9cb440ca3552f75624d3a0c9c33c5cb84d52ea021a Jul 23 22:28:36.979584 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:28:36.979526 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 23 22:28:36.979688 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:28:36.979623 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 23 22:28:36.979688 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:28:36.979675 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 23 22:28:37.800611 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:28:37.800579 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-8a3e27-openshift-default-76fcf6495f-4l6r5" event={"ID":"be46532d-4c4e-4092-b94b-31d8f3f84ce3","Type":"ContainerStarted","Data":"56f15ffcf894cce5c4fb75e32bfe4e64efd9da31ce0cb8a5585fc01d02d3b31f"} Jul 23 22:28:37.800986 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:28:37.800629 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-8a3e27-openshift-default-76fcf6495f-4l6r5" event={"ID":"be46532d-4c4e-4092-b94b-31d8f3f84ce3","Type":"ContainerStarted","Data":"e62df3f1b136487467816a9cb440ca3552f75624d3a0c9c33c5cb84d52ea021a"} Jul 23 22:28:37.823614 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:28:37.823563 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-8a3e27-openshift-default-76fcf6495f-4l6r5" podStartSLOduration=1.823533074 podStartE2EDuration="1.823533074s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 22:28:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 22:28:37.822493315 +0000 UTC m=+2220.064401189" watchObservedRunningTime="2026-07-23 22:28:37.823533074 +0000 UTC m=+2220.065440947" Jul 23 22:28:37.839886 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:28:37.839862 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-8a3e27-openshift-default-76fcf6495f-4l6r5" Jul 23 22:28:38.845043 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:28:38.845018 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-8a3e27-openshift-default-76fcf6495f-4l6r5" Jul 23 22:28:39.810157 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:28:39.810127 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-8a3e27-openshift-default-76fcf6495f-4l6r5" Jul 23 22:28:39.811054 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:28:39.811036 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-8a3e27-openshift-default-76fcf6495f-4l6r5" Jul 23 22:28:50.874893 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:28:50.874859 2578 generic.go:356] "Generic (PLEG): container finished" podID="be46532d-4c4e-4092-b94b-31d8f3f84ce3" containerID="56f15ffcf894cce5c4fb75e32bfe4e64efd9da31ce0cb8a5585fc01d02d3b31f" exitCode=0 Jul 23 22:28:50.875325 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:28:50.874902 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-8a3e27-openshift-default-76fcf6495f-4l6r5" event={"ID":"be46532d-4c4e-4092-b94b-31d8f3f84ce3","Type":"ContainerDied","Data":"56f15ffcf894cce5c4fb75e32bfe4e64efd9da31ce0cb8a5585fc01d02d3b31f"} Jul 23 22:28:50.875325 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:28:50.875291 2578 scope.go:117] "RemoveContainer" containerID="56f15ffcf894cce5c4fb75e32bfe4e64efd9da31ce0cb8a5585fc01d02d3b31f" Jul 23 22:28:50.876367 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:28:50.876315 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 23 22:28:50.876662 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:28:50.876385 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 23 22:28:50.876662 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:28:50.876432 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 23 22:28:51.882713 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:28:51.882677 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-8a3e27-openshift-default-76fcf6495f-4l6r5" event={"ID":"be46532d-4c4e-4092-b94b-31d8f3f84ce3","Type":"ContainerStarted","Data":"be68e22e048dc15b117c3d752ba98493d6bd62e27c3014d246d0c0aa875da71a"} Jul 23 22:28:52.840795 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:28:52.840755 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-8a3e27-openshift-default-76fcf6495f-4l6r5" Jul 23 22:28:52.841915 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:28:52.841876 2578 patch_prober.go:28] interesting pod/e2e-inf-b-8a3e27-openshift-default-76fcf6495f-4l6r5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body= Jul 23 22:28:52.842054 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:28:52.841952 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-8a3e27-openshift-default-76fcf6495f-4l6r5" podUID="be46532d-4c4e-4092-b94b-31d8f3f84ce3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" Jul 23 22:28:53.840591 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:28:53.840526 2578 patch_prober.go:28] interesting pod/e2e-inf-b-8a3e27-openshift-default-76fcf6495f-4l6r5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body= Jul 23 22:28:53.841053 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:28:53.840620 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-8a3e27-openshift-default-76fcf6495f-4l6r5" podUID="be46532d-4c4e-4092-b94b-31d8f3f84ce3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" Jul 23 22:28:54.810290 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:28:54.810254 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-8a3e27-openshift-default-76fcf6495f-4l6r5" Jul 23 22:28:54.840779 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:28:54.840740 2578 patch_prober.go:28] interesting pod/e2e-inf-b-8a3e27-openshift-default-76fcf6495f-4l6r5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body= Jul 23 22:28:54.841246 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:28:54.840814 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-8a3e27-openshift-default-76fcf6495f-4l6r5" podUID="be46532d-4c4e-4092-b94b-31d8f3f84ce3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" Jul 23 22:28:55.844105 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:28:55.844070 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-8a3e27-openshift-default-76fcf6495f-4l6r5" Jul 23 22:28:55.845147 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:28:55.845123 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-8a3e27-openshift-default-76fcf6495f-4l6r5" Jul 23 22:30:00.142888 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:30:00.142853 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-8a3e27-29747430-qx2qc"] Jul 23 22:30:00.146160 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:30:00.146145 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-8a3e27-29747430-qx2qc" Jul 23 22:30:00.148624 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:30:00.148602 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lq4mm\"" Jul 23 22:30:00.157226 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:30:00.157206 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-8a3e27-29747430-qx2qc" Jul 23 22:30:00.158088 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:30:00.158066 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-8a3e27-29747430-qx2qc"] Jul 23 22:30:00.301880 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:30:00.301853 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-8a3e27-29747430-qx2qc"] Jul 23 22:30:00.303324 ip-10-0-136-157 kubenswrapper[2578]: W0723 22:30:00.303303 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddfbe8520_635e_4f79_85df_79cbc3caee93.slice/crio-1075ecc7dec887acccd5d70b89725e102cb6c8c11a5884bc3d0d0d19ea2a1ff6 WatchSource:0}: Error finding container 1075ecc7dec887acccd5d70b89725e102cb6c8c11a5884bc3d0d0d19ea2a1ff6: Status 404 returned error can't find the container with id 1075ecc7dec887acccd5d70b89725e102cb6c8c11a5884bc3d0d0d19ea2a1ff6 Jul 23 22:30:00.305198 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:30:00.305180 2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 23 22:30:01.227213 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:30:01.227180 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-8a3e27-29747430-qx2qc" event={"ID":"dfbe8520-635e-4f79-85df-79cbc3caee93","Type":"ContainerStarted","Data":"1075ecc7dec887acccd5d70b89725e102cb6c8c11a5884bc3d0d0d19ea2a1ff6"} Jul 23 22:30:03.241221 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:30:03.241136 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-8a3e27-29747430-qx2qc" event={"ID":"dfbe8520-635e-4f79-85df-79cbc3caee93","Type":"ContainerStarted","Data":"d1daff7a09c9c3300e6cfd7572d0115022a592565facbe438b58737f6b177695"} Jul 23 22:30:03.257534 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:30:03.257480 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-8a3e27-29747430-qx2qc" podStartSLOduration=0.996614705 podStartE2EDuration="3.257465173s" totalImagesPullingTime="2.260850468s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 22:30:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 22:30:03.255245348 +0000 UTC m=+2305.497153223" watchObservedRunningTime="2026-07-23 22:30:03.257465173 +0000 UTC m=+2305.499373047" Jul 23 22:30:04.247636 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:30:04.247607 2578 generic.go:356] "Generic (PLEG): container finished" podID="dfbe8520-635e-4f79-85df-79cbc3caee93" containerID="d1daff7a09c9c3300e6cfd7572d0115022a592565facbe438b58737f6b177695" exitCode=0 Jul 23 22:30:04.251550 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:30:04.251506 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-8a3e27-29747430-qx2qc" event={"ID":"dfbe8520-635e-4f79-85df-79cbc3caee93","Type":"ContainerDied","Data":"d1daff7a09c9c3300e6cfd7572d0115022a592565facbe438b58737f6b177695"} Jul 23 22:30:05.383713 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:30:05.383688 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-8a3e27-29747430-qx2qc" Jul 23 22:30:06.263416 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:30:06.263385 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-8a3e27-29747430-qx2qc" event={"ID":"dfbe8520-635e-4f79-85df-79cbc3caee93","Type":"ContainerDied","Data":"1075ecc7dec887acccd5d70b89725e102cb6c8c11a5884bc3d0d0d19ea2a1ff6"} Jul 23 22:30:06.263416 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:30:06.263411 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-8a3e27-29747430-qx2qc" Jul 23 22:30:06.263611 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:30:06.263434 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1075ecc7dec887acccd5d70b89725e102cb6c8c11a5884bc3d0d0d19ea2a1ff6" Jul 23 22:30:33.196579 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:30:33.196474 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-8a3e27-29747430-qx2qc"] Jul 23 22:30:33.198571 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:30:33.198533 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-8a3e27-29747430-qx2qc"] Jul 23 22:30:34.250802 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:30:34.250767 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dfbe8520-635e-4f79-85df-79cbc3caee93" path="/var/lib/kubelet/pods/dfbe8520-635e-4f79-85df-79cbc3caee93/volumes" Jul 23 22:30:44.417663 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:30:44.417629 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-8a3e27-openshift-default-76fcf6495f-4l6r5"] Jul 23 22:30:44.418183 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:30:44.417889 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-8a3e27-openshift-default-76fcf6495f-4l6r5" podUID="be46532d-4c4e-4092-b94b-31d8f3f84ce3" containerName="istio-proxy" containerID="cri-o://be68e22e048dc15b117c3d752ba98493d6bd62e27c3014d246d0c0aa875da71a" gracePeriod=30 Jul 23 22:30:49.672235 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:30:49.672208 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-8a3e27-openshift-default-76fcf6495f-4l6r5" Jul 23 22:30:49.789946 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:30:49.789884 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/be46532d-4c4e-4092-b94b-31d8f3f84ce3-workload-socket\") pod \"be46532d-4c4e-4092-b94b-31d8f3f84ce3\" (UID: \"be46532d-4c4e-4092-b94b-31d8f3f84ce3\") " Jul 23 22:30:49.789946 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:30:49.789924 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/be46532d-4c4e-4092-b94b-31d8f3f84ce3-credential-socket\") pod \"be46532d-4c4e-4092-b94b-31d8f3f84ce3\" (UID: \"be46532d-4c4e-4092-b94b-31d8f3f84ce3\") " Jul 23 22:30:49.790126 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:30:49.789955 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/be46532d-4c4e-4092-b94b-31d8f3f84ce3-istio-podinfo\") pod \"be46532d-4c4e-4092-b94b-31d8f3f84ce3\" (UID: \"be46532d-4c4e-4092-b94b-31d8f3f84ce3\") " Jul 23 22:30:49.790126 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:30:49.789982 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/be46532d-4c4e-4092-b94b-31d8f3f84ce3-workload-certs\") pod \"be46532d-4c4e-4092-b94b-31d8f3f84ce3\" (UID: \"be46532d-4c4e-4092-b94b-31d8f3f84ce3\") " Jul 23 22:30:49.790126 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:30:49.790031 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/be46532d-4c4e-4092-b94b-31d8f3f84ce3-istio-data\") pod \"be46532d-4c4e-4092-b94b-31d8f3f84ce3\" (UID: \"be46532d-4c4e-4092-b94b-31d8f3f84ce3\") " Jul 23 22:30:49.790126 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:30:49.790066 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/be46532d-4c4e-4092-b94b-31d8f3f84ce3-istio-token\") pod \"be46532d-4c4e-4092-b94b-31d8f3f84ce3\" (UID: \"be46532d-4c4e-4092-b94b-31d8f3f84ce3\") " Jul 23 22:30:49.790126 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:30:49.790105 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/be46532d-4c4e-4092-b94b-31d8f3f84ce3-istiod-ca-cert\") pod \"be46532d-4c4e-4092-b94b-31d8f3f84ce3\" (UID: \"be46532d-4c4e-4092-b94b-31d8f3f84ce3\") " Jul 23 22:30:49.790373 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:30:49.790122 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/be46532d-4c4e-4092-b94b-31d8f3f84ce3-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "be46532d-4c4e-4092-b94b-31d8f3f84ce3" (UID: "be46532d-4c4e-4092-b94b-31d8f3f84ce3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 22:30:49.790373 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:30:49.790137 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/be46532d-4c4e-4092-b94b-31d8f3f84ce3-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "be46532d-4c4e-4092-b94b-31d8f3f84ce3" (UID: "be46532d-4c4e-4092-b94b-31d8f3f84ce3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 22:30:49.790373 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:30:49.790278 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/be46532d-4c4e-4092-b94b-31d8f3f84ce3-istio-envoy\") pod \"be46532d-4c4e-4092-b94b-31d8f3f84ce3\" (UID: \"be46532d-4c4e-4092-b94b-31d8f3f84ce3\") " Jul 23 22:30:49.790373 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:30:49.790282 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/be46532d-4c4e-4092-b94b-31d8f3f84ce3-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "be46532d-4c4e-4092-b94b-31d8f3f84ce3" (UID: "be46532d-4c4e-4092-b94b-31d8f3f84ce3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 22:30:49.790373 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:30:49.790315 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fwfh8\" (UniqueName: \"kubernetes.io/projected/be46532d-4c4e-4092-b94b-31d8f3f84ce3-kube-api-access-fwfh8\") pod \"be46532d-4c4e-4092-b94b-31d8f3f84ce3\" (UID: \"be46532d-4c4e-4092-b94b-31d8f3f84ce3\") " Jul 23 22:30:49.790657 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:30:49.790453 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/be46532d-4c4e-4092-b94b-31d8f3f84ce3-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "be46532d-4c4e-4092-b94b-31d8f3f84ce3" (UID: "be46532d-4c4e-4092-b94b-31d8f3f84ce3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 22:30:49.790922 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:30:49.790682 2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/be46532d-4c4e-4092-b94b-31d8f3f84ce3-istiod-ca-cert\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\"" Jul 23 22:30:49.790922 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:30:49.790701 2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/be46532d-4c4e-4092-b94b-31d8f3f84ce3-workload-socket\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\"" Jul 23 22:30:49.790922 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:30:49.790713 2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/be46532d-4c4e-4092-b94b-31d8f3f84ce3-credential-socket\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\"" Jul 23 22:30:49.790922 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:30:49.790724 2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/be46532d-4c4e-4092-b94b-31d8f3f84ce3-workload-certs\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\"" Jul 23 22:30:49.791130 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:30:49.791085 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/be46532d-4c4e-4092-b94b-31d8f3f84ce3-istio-data" (OuterVolumeSpecName: "istio-data") pod "be46532d-4c4e-4092-b94b-31d8f3f84ce3" (UID: "be46532d-4c4e-4092-b94b-31d8f3f84ce3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 22:30:49.792308 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:30:49.792287 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/be46532d-4c4e-4092-b94b-31d8f3f84ce3-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "be46532d-4c4e-4092-b94b-31d8f3f84ce3" (UID: "be46532d-4c4e-4092-b94b-31d8f3f84ce3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 23 22:30:49.792513 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:30:49.792494 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/be46532d-4c4e-4092-b94b-31d8f3f84ce3-kube-api-access-fwfh8" (OuterVolumeSpecName: "kube-api-access-fwfh8") pod "be46532d-4c4e-4092-b94b-31d8f3f84ce3" (UID: "be46532d-4c4e-4092-b94b-31d8f3f84ce3"). InnerVolumeSpecName "kube-api-access-fwfh8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 22:30:49.792821 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:30:49.792800 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/be46532d-4c4e-4092-b94b-31d8f3f84ce3-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "be46532d-4c4e-4092-b94b-31d8f3f84ce3" (UID: "be46532d-4c4e-4092-b94b-31d8f3f84ce3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 22:30:49.792879 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:30:49.792822 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/be46532d-4c4e-4092-b94b-31d8f3f84ce3-istio-token" (OuterVolumeSpecName: "istio-token") pod "be46532d-4c4e-4092-b94b-31d8f3f84ce3" (UID: "be46532d-4c4e-4092-b94b-31d8f3f84ce3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 22:30:49.891416 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:30:49.891392 2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/be46532d-4c4e-4092-b94b-31d8f3f84ce3-istio-envoy\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\"" Jul 23 22:30:49.891416 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:30:49.891413 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fwfh8\" (UniqueName: \"kubernetes.io/projected/be46532d-4c4e-4092-b94b-31d8f3f84ce3-kube-api-access-fwfh8\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\"" Jul 23 22:30:49.891527 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:30:49.891424 2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/be46532d-4c4e-4092-b94b-31d8f3f84ce3-istio-podinfo\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\"" Jul 23 22:30:49.891527 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:30:49.891433 2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/be46532d-4c4e-4092-b94b-31d8f3f84ce3-istio-data\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\"" Jul 23 22:30:49.891527 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:30:49.891441 2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/be46532d-4c4e-4092-b94b-31d8f3f84ce3-istio-token\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\"" Jul 23 22:30:50.485689 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:30:50.485655 2578 generic.go:356] "Generic (PLEG): container finished" podID="be46532d-4c4e-4092-b94b-31d8f3f84ce3" containerID="be68e22e048dc15b117c3d752ba98493d6bd62e27c3014d246d0c0aa875da71a" exitCode=0 Jul 23 22:30:50.485846 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:30:50.485707 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-8a3e27-openshift-default-76fcf6495f-4l6r5" Jul 23 22:30:50.485846 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:30:50.485717 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-8a3e27-openshift-default-76fcf6495f-4l6r5" event={"ID":"be46532d-4c4e-4092-b94b-31d8f3f84ce3","Type":"ContainerDied","Data":"be68e22e048dc15b117c3d752ba98493d6bd62e27c3014d246d0c0aa875da71a"} Jul 23 22:30:50.485846 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:30:50.485764 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-8a3e27-openshift-default-76fcf6495f-4l6r5" event={"ID":"be46532d-4c4e-4092-b94b-31d8f3f84ce3","Type":"ContainerDied","Data":"e62df3f1b136487467816a9cb440ca3552f75624d3a0c9c33c5cb84d52ea021a"} Jul 23 22:30:50.485846 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:30:50.485789 2578 scope.go:117] "RemoveContainer" containerID="be68e22e048dc15b117c3d752ba98493d6bd62e27c3014d246d0c0aa875da71a" Jul 23 22:30:50.494839 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:30:50.494822 2578 scope.go:117] "RemoveContainer" containerID="56f15ffcf894cce5c4fb75e32bfe4e64efd9da31ce0cb8a5585fc01d02d3b31f" Jul 23 22:30:50.502910 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:30:50.502895 2578 scope.go:117] "RemoveContainer" containerID="be68e22e048dc15b117c3d752ba98493d6bd62e27c3014d246d0c0aa875da71a" Jul 23 22:30:50.503188 ip-10-0-136-157 kubenswrapper[2578]: E0723 22:30:50.503169 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"be68e22e048dc15b117c3d752ba98493d6bd62e27c3014d246d0c0aa875da71a\": container with ID starting with be68e22e048dc15b117c3d752ba98493d6bd62e27c3014d246d0c0aa875da71a not found: ID does not exist" containerID="be68e22e048dc15b117c3d752ba98493d6bd62e27c3014d246d0c0aa875da71a" Jul 23 22:30:50.503233 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:30:50.503197 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"be68e22e048dc15b117c3d752ba98493d6bd62e27c3014d246d0c0aa875da71a"} err="failed to get container status \"be68e22e048dc15b117c3d752ba98493d6bd62e27c3014d246d0c0aa875da71a\": rpc error: code = NotFound desc = could not find container \"be68e22e048dc15b117c3d752ba98493d6bd62e27c3014d246d0c0aa875da71a\": container with ID starting with be68e22e048dc15b117c3d752ba98493d6bd62e27c3014d246d0c0aa875da71a not found: ID does not exist" Jul 23 22:30:50.503233 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:30:50.503212 2578 scope.go:117] "RemoveContainer" containerID="56f15ffcf894cce5c4fb75e32bfe4e64efd9da31ce0cb8a5585fc01d02d3b31f" Jul 23 22:30:50.503447 ip-10-0-136-157 kubenswrapper[2578]: E0723 22:30:50.503430 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"56f15ffcf894cce5c4fb75e32bfe4e64efd9da31ce0cb8a5585fc01d02d3b31f\": container with ID starting with 56f15ffcf894cce5c4fb75e32bfe4e64efd9da31ce0cb8a5585fc01d02d3b31f not found: ID does not exist" containerID="56f15ffcf894cce5c4fb75e32bfe4e64efd9da31ce0cb8a5585fc01d02d3b31f" Jul 23 22:30:50.503489 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:30:50.503454 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"56f15ffcf894cce5c4fb75e32bfe4e64efd9da31ce0cb8a5585fc01d02d3b31f"} err="failed to get container status \"56f15ffcf894cce5c4fb75e32bfe4e64efd9da31ce0cb8a5585fc01d02d3b31f\": rpc error: code = NotFound desc = could not find container \"56f15ffcf894cce5c4fb75e32bfe4e64efd9da31ce0cb8a5585fc01d02d3b31f\": container with ID starting with 56f15ffcf894cce5c4fb75e32bfe4e64efd9da31ce0cb8a5585fc01d02d3b31f not found: ID does not exist" Jul 23 22:30:50.504836 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:30:50.504819 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-8a3e27-openshift-default-76fcf6495f-4l6r5"] Jul 23 22:30:50.508187 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:30:50.508167 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-8a3e27-openshift-default-76fcf6495f-4l6r5"] Jul 23 22:30:52.251385 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:30:52.251352 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="be46532d-4c4e-4092-b94b-31d8f3f84ce3" path="/var/lib/kubelet/pods/be46532d-4c4e-4092-b94b-31d8f3f84ce3/volumes" Jul 23 22:30:53.315972 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:30:53.315943 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-fc9d5e-openshift-default-d7c7f87fc-qcvv5"] Jul 23 22:30:53.316312 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:30:53.316261 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dfbe8520-635e-4f79-85df-79cbc3caee93" containerName="cleanup" Jul 23 22:30:53.316312 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:30:53.316272 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="dfbe8520-635e-4f79-85df-79cbc3caee93" containerName="cleanup" Jul 23 22:30:53.316312 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:30:53.316284 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="be46532d-4c4e-4092-b94b-31d8f3f84ce3" containerName="istio-proxy" Jul 23 22:30:53.316312 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:30:53.316289 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="be46532d-4c4e-4092-b94b-31d8f3f84ce3" containerName="istio-proxy" Jul 23 22:30:53.316312 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:30:53.316304 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="be46532d-4c4e-4092-b94b-31d8f3f84ce3" containerName="istio-proxy" Jul 23 22:30:53.316312 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:30:53.316309 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="be46532d-4c4e-4092-b94b-31d8f3f84ce3" containerName="istio-proxy" Jul 23 22:30:53.316491 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:30:53.316364 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="be46532d-4c4e-4092-b94b-31d8f3f84ce3" containerName="istio-proxy" Jul 23 22:30:53.316491 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:30:53.316372 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="dfbe8520-635e-4f79-85df-79cbc3caee93" containerName="cleanup" Jul 23 22:30:53.316491 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:30:53.316381 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="be46532d-4c4e-4092-b94b-31d8f3f84ce3" containerName="istio-proxy" Jul 23 22:30:53.319867 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:30:53.319844 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-fc9d5e-openshift-default-d7c7f87fc-qcvv5" Jul 23 22:30:53.322288 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:30:53.322268 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-fc9d5e-openshift-default-dockercfg-fsrtd\"" Jul 23 22:30:53.335602 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:30:53.335580 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-fc9d5e-openshift-default-d7c7f87fc-qcvv5"] Jul 23 22:30:53.419382 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:30:53.419355 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0316ef32-8546-41c6-a32c-0613fff7599a-istiod-ca-cert\") pod \"e2e-api-b-fc9d5e-openshift-default-d7c7f87fc-qcvv5\" (UID: \"0316ef32-8546-41c6-a32c-0613fff7599a\") " pod="openshift-ingress/e2e-api-b-fc9d5e-openshift-default-d7c7f87fc-qcvv5" Jul 23 22:30:53.419486 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:30:53.419388 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0316ef32-8546-41c6-a32c-0613fff7599a-istio-envoy\") pod \"e2e-api-b-fc9d5e-openshift-default-d7c7f87fc-qcvv5\" (UID: \"0316ef32-8546-41c6-a32c-0613fff7599a\") " pod="openshift-ingress/e2e-api-b-fc9d5e-openshift-default-d7c7f87fc-qcvv5" Jul 23 22:30:53.419486 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:30:53.419408 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-279q9\" (UniqueName: \"kubernetes.io/projected/0316ef32-8546-41c6-a32c-0613fff7599a-kube-api-access-279q9\") pod \"e2e-api-b-fc9d5e-openshift-default-d7c7f87fc-qcvv5\" (UID: \"0316ef32-8546-41c6-a32c-0613fff7599a\") " pod="openshift-ingress/e2e-api-b-fc9d5e-openshift-default-d7c7f87fc-qcvv5" Jul 23 22:30:53.419486 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:30:53.419436 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0316ef32-8546-41c6-a32c-0613fff7599a-istio-token\") pod \"e2e-api-b-fc9d5e-openshift-default-d7c7f87fc-qcvv5\" (UID: \"0316ef32-8546-41c6-a32c-0613fff7599a\") " pod="openshift-ingress/e2e-api-b-fc9d5e-openshift-default-d7c7f87fc-qcvv5" Jul 23 22:30:53.419486 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:30:53.419480 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0316ef32-8546-41c6-a32c-0613fff7599a-workload-socket\") pod \"e2e-api-b-fc9d5e-openshift-default-d7c7f87fc-qcvv5\" (UID: \"0316ef32-8546-41c6-a32c-0613fff7599a\") " pod="openshift-ingress/e2e-api-b-fc9d5e-openshift-default-d7c7f87fc-qcvv5" Jul 23 22:30:53.419709 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:30:53.419504 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0316ef32-8546-41c6-a32c-0613fff7599a-credential-socket\") pod \"e2e-api-b-fc9d5e-openshift-default-d7c7f87fc-qcvv5\" (UID: \"0316ef32-8546-41c6-a32c-0613fff7599a\") " pod="openshift-ingress/e2e-api-b-fc9d5e-openshift-default-d7c7f87fc-qcvv5" Jul 23 22:30:53.419709 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:30:53.419587 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0316ef32-8546-41c6-a32c-0613fff7599a-istio-data\") pod \"e2e-api-b-fc9d5e-openshift-default-d7c7f87fc-qcvv5\" (UID: \"0316ef32-8546-41c6-a32c-0613fff7599a\") " pod="openshift-ingress/e2e-api-b-fc9d5e-openshift-default-d7c7f87fc-qcvv5" Jul 23 22:30:53.419709 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:30:53.419624 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0316ef32-8546-41c6-a32c-0613fff7599a-workload-certs\") pod \"e2e-api-b-fc9d5e-openshift-default-d7c7f87fc-qcvv5\" (UID: \"0316ef32-8546-41c6-a32c-0613fff7599a\") " pod="openshift-ingress/e2e-api-b-fc9d5e-openshift-default-d7c7f87fc-qcvv5" Jul 23 22:30:53.419709 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:30:53.419659 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0316ef32-8546-41c6-a32c-0613fff7599a-istio-podinfo\") pod \"e2e-api-b-fc9d5e-openshift-default-d7c7f87fc-qcvv5\" (UID: \"0316ef32-8546-41c6-a32c-0613fff7599a\") " pod="openshift-ingress/e2e-api-b-fc9d5e-openshift-default-d7c7f87fc-qcvv5" Jul 23 22:30:53.520975 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:30:53.520945 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0316ef32-8546-41c6-a32c-0613fff7599a-istiod-ca-cert\") pod \"e2e-api-b-fc9d5e-openshift-default-d7c7f87fc-qcvv5\" (UID: \"0316ef32-8546-41c6-a32c-0613fff7599a\") " pod="openshift-ingress/e2e-api-b-fc9d5e-openshift-default-d7c7f87fc-qcvv5" Jul 23 22:30:53.521168 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:30:53.520989 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0316ef32-8546-41c6-a32c-0613fff7599a-istio-envoy\") pod \"e2e-api-b-fc9d5e-openshift-default-d7c7f87fc-qcvv5\" (UID: \"0316ef32-8546-41c6-a32c-0613fff7599a\") " pod="openshift-ingress/e2e-api-b-fc9d5e-openshift-default-d7c7f87fc-qcvv5" Jul 23 22:30:53.521168 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:30:53.521028 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-279q9\" (UniqueName: \"kubernetes.io/projected/0316ef32-8546-41c6-a32c-0613fff7599a-kube-api-access-279q9\") pod \"e2e-api-b-fc9d5e-openshift-default-d7c7f87fc-qcvv5\" (UID: \"0316ef32-8546-41c6-a32c-0613fff7599a\") " pod="openshift-ingress/e2e-api-b-fc9d5e-openshift-default-d7c7f87fc-qcvv5" Jul 23 22:30:53.521168 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:30:53.521067 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0316ef32-8546-41c6-a32c-0613fff7599a-istio-token\") pod \"e2e-api-b-fc9d5e-openshift-default-d7c7f87fc-qcvv5\" (UID: \"0316ef32-8546-41c6-a32c-0613fff7599a\") " pod="openshift-ingress/e2e-api-b-fc9d5e-openshift-default-d7c7f87fc-qcvv5" Jul 23 22:30:53.521345 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:30:53.521281 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0316ef32-8546-41c6-a32c-0613fff7599a-workload-socket\") pod \"e2e-api-b-fc9d5e-openshift-default-d7c7f87fc-qcvv5\" (UID: \"0316ef32-8546-41c6-a32c-0613fff7599a\") " pod="openshift-ingress/e2e-api-b-fc9d5e-openshift-default-d7c7f87fc-qcvv5" Jul 23 22:30:53.521345 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:30:53.521339 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0316ef32-8546-41c6-a32c-0613fff7599a-credential-socket\") pod \"e2e-api-b-fc9d5e-openshift-default-d7c7f87fc-qcvv5\" (UID: \"0316ef32-8546-41c6-a32c-0613fff7599a\") " pod="openshift-ingress/e2e-api-b-fc9d5e-openshift-default-d7c7f87fc-qcvv5" Jul 23 22:30:53.521439 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:30:53.521403 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0316ef32-8546-41c6-a32c-0613fff7599a-istio-data\") pod \"e2e-api-b-fc9d5e-openshift-default-d7c7f87fc-qcvv5\" (UID: \"0316ef32-8546-41c6-a32c-0613fff7599a\") " pod="openshift-ingress/e2e-api-b-fc9d5e-openshift-default-d7c7f87fc-qcvv5" Jul 23 22:30:53.521490 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:30:53.521456 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0316ef32-8546-41c6-a32c-0613fff7599a-workload-certs\") pod \"e2e-api-b-fc9d5e-openshift-default-d7c7f87fc-qcvv5\" (UID: \"0316ef32-8546-41c6-a32c-0613fff7599a\") " pod="openshift-ingress/e2e-api-b-fc9d5e-openshift-default-d7c7f87fc-qcvv5" Jul 23 22:30:53.521576 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:30:53.521530 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0316ef32-8546-41c6-a32c-0613fff7599a-istio-podinfo\") pod \"e2e-api-b-fc9d5e-openshift-default-d7c7f87fc-qcvv5\" (UID: \"0316ef32-8546-41c6-a32c-0613fff7599a\") " pod="openshift-ingress/e2e-api-b-fc9d5e-openshift-default-d7c7f87fc-qcvv5" Jul 23 22:30:53.521641 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:30:53.521603 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0316ef32-8546-41c6-a32c-0613fff7599a-credential-socket\") pod \"e2e-api-b-fc9d5e-openshift-default-d7c7f87fc-qcvv5\" (UID: \"0316ef32-8546-41c6-a32c-0613fff7599a\") " pod="openshift-ingress/e2e-api-b-fc9d5e-openshift-default-d7c7f87fc-qcvv5" Jul 23 22:30:53.521699 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:30:53.521664 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0316ef32-8546-41c6-a32c-0613fff7599a-workload-socket\") pod \"e2e-api-b-fc9d5e-openshift-default-d7c7f87fc-qcvv5\" (UID: \"0316ef32-8546-41c6-a32c-0613fff7599a\") " pod="openshift-ingress/e2e-api-b-fc9d5e-openshift-default-d7c7f87fc-qcvv5" Jul 23 22:30:53.521761 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:30:53.521714 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0316ef32-8546-41c6-a32c-0613fff7599a-istiod-ca-cert\") pod \"e2e-api-b-fc9d5e-openshift-default-d7c7f87fc-qcvv5\" (UID: \"0316ef32-8546-41c6-a32c-0613fff7599a\") " pod="openshift-ingress/e2e-api-b-fc9d5e-openshift-default-d7c7f87fc-qcvv5" Jul 23 22:30:53.521815 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:30:53.521783 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0316ef32-8546-41c6-a32c-0613fff7599a-istio-data\") pod \"e2e-api-b-fc9d5e-openshift-default-d7c7f87fc-qcvv5\" (UID: \"0316ef32-8546-41c6-a32c-0613fff7599a\") " pod="openshift-ingress/e2e-api-b-fc9d5e-openshift-default-d7c7f87fc-qcvv5" Jul 23 22:30:53.521815 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:30:53.521785 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0316ef32-8546-41c6-a32c-0613fff7599a-workload-certs\") pod \"e2e-api-b-fc9d5e-openshift-default-d7c7f87fc-qcvv5\" (UID: \"0316ef32-8546-41c6-a32c-0613fff7599a\") " pod="openshift-ingress/e2e-api-b-fc9d5e-openshift-default-d7c7f87fc-qcvv5" Jul 23 22:30:53.523470 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:30:53.523453 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0316ef32-8546-41c6-a32c-0613fff7599a-istio-envoy\") pod \"e2e-api-b-fc9d5e-openshift-default-d7c7f87fc-qcvv5\" (UID: \"0316ef32-8546-41c6-a32c-0613fff7599a\") " pod="openshift-ingress/e2e-api-b-fc9d5e-openshift-default-d7c7f87fc-qcvv5" Jul 23 22:30:53.524284 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:30:53.524268 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0316ef32-8546-41c6-a32c-0613fff7599a-istio-podinfo\") pod \"e2e-api-b-fc9d5e-openshift-default-d7c7f87fc-qcvv5\" (UID: \"0316ef32-8546-41c6-a32c-0613fff7599a\") " pod="openshift-ingress/e2e-api-b-fc9d5e-openshift-default-d7c7f87fc-qcvv5" Jul 23 22:30:53.530144 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:30:53.530116 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0316ef32-8546-41c6-a32c-0613fff7599a-istio-token\") pod \"e2e-api-b-fc9d5e-openshift-default-d7c7f87fc-qcvv5\" (UID: \"0316ef32-8546-41c6-a32c-0613fff7599a\") " pod="openshift-ingress/e2e-api-b-fc9d5e-openshift-default-d7c7f87fc-qcvv5" Jul 23 22:30:53.530283 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:30:53.530268 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-279q9\" (UniqueName: \"kubernetes.io/projected/0316ef32-8546-41c6-a32c-0613fff7599a-kube-api-access-279q9\") pod \"e2e-api-b-fc9d5e-openshift-default-d7c7f87fc-qcvv5\" (UID: \"0316ef32-8546-41c6-a32c-0613fff7599a\") " pod="openshift-ingress/e2e-api-b-fc9d5e-openshift-default-d7c7f87fc-qcvv5" Jul 23 22:30:53.630222 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:30:53.630168 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-fc9d5e-openshift-default-d7c7f87fc-qcvv5" Jul 23 22:30:53.760216 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:30:53.760189 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-fc9d5e-openshift-default-d7c7f87fc-qcvv5"] Jul 23 22:30:53.761379 ip-10-0-136-157 kubenswrapper[2578]: W0723 22:30:53.761356 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0316ef32_8546_41c6_a32c_0613fff7599a.slice/crio-32eba0f2e58b9e58365f39afcfcac2f24fd9e1560bbd60b8ebdbe0716f8593ae WatchSource:0}: Error finding container 32eba0f2e58b9e58365f39afcfcac2f24fd9e1560bbd60b8ebdbe0716f8593ae: Status 404 returned error can't find the container with id 32eba0f2e58b9e58365f39afcfcac2f24fd9e1560bbd60b8ebdbe0716f8593ae Jul 23 22:30:53.763643 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:30:53.763614 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 23 22:30:53.763742 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:30:53.763680 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 23 22:30:53.763742 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:30:53.763728 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 23 22:30:54.514533 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:30:54.514500 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-fc9d5e-openshift-default-d7c7f87fc-qcvv5" event={"ID":"0316ef32-8546-41c6-a32c-0613fff7599a","Type":"ContainerStarted","Data":"a3e1d1cd58c5bfd898191d56ba4fad5f38d609e2de9d0c1515f30603ee7567a2"} Jul 23 22:30:54.514910 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:30:54.514558 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-fc9d5e-openshift-default-d7c7f87fc-qcvv5" event={"ID":"0316ef32-8546-41c6-a32c-0613fff7599a","Type":"ContainerStarted","Data":"32eba0f2e58b9e58365f39afcfcac2f24fd9e1560bbd60b8ebdbe0716f8593ae"} Jul 23 22:30:54.536388 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:30:54.536337 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-fc9d5e-openshift-default-d7c7f87fc-qcvv5" podStartSLOduration=1.53632299 podStartE2EDuration="1.53632299s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 22:30:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 22:30:54.53419192 +0000 UTC m=+2356.776099795" watchObservedRunningTime="2026-07-23 22:30:54.53632299 +0000 UTC m=+2356.778230939" Jul 23 22:30:54.631125 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:30:54.631092 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-fc9d5e-openshift-default-d7c7f87fc-qcvv5" Jul 23 22:30:54.635959 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:30:54.635936 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-fc9d5e-openshift-default-d7c7f87fc-qcvv5" Jul 23 22:30:55.522382 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:30:55.522345 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-fc9d5e-openshift-default-d7c7f87fc-qcvv5" Jul 23 22:30:55.523380 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:30:55.523363 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-fc9d5e-openshift-default-d7c7f87fc-qcvv5" Jul 23 22:31:09.232752 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:31:09.232718 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c3321b-7b9f5"] Jul 23 22:31:09.235915 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:31:09.235896 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c3321b-7b9f5" Jul 23 22:31:09.238792 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:31:09.238769 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lq4mm\"" Jul 23 22:31:09.238921 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:31:09.238792 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 23 22:31:09.252188 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:31:09.252159 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c3321b-7b9f5"] Jul 23 22:31:09.369099 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:31:09.369067 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d4d8ffc1-d0a1-4ee2-badf-ba435136fdb7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-c3321b-7b9f5\" (UID: \"d4d8ffc1-d0a1-4ee2-badf-ba435136fdb7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c3321b-7b9f5" Jul 23 22:31:09.469822 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:31:09.469783 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d4d8ffc1-d0a1-4ee2-badf-ba435136fdb7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-c3321b-7b9f5\" (UID: \"d4d8ffc1-d0a1-4ee2-badf-ba435136fdb7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c3321b-7b9f5" Jul 23 22:31:09.470367 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:31:09.470349 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d4d8ffc1-d0a1-4ee2-badf-ba435136fdb7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-c3321b-7b9f5\" (UID: \"d4d8ffc1-d0a1-4ee2-badf-ba435136fdb7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c3321b-7b9f5" Jul 23 22:31:09.547074 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:31:09.547043 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c3321b-7b9f5" Jul 23 22:31:09.693820 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:31:09.693782 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c3321b-7b9f5"] Jul 23 22:31:09.695419 ip-10-0-136-157 kubenswrapper[2578]: W0723 22:31:09.695387 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4d8ffc1_d0a1_4ee2_badf_ba435136fdb7.slice/crio-3d410fdd41d67876db98c07a53c23e279fcecdfea5a1ff4c71cd3c3b0e49bb5b WatchSource:0}: Error finding container 3d410fdd41d67876db98c07a53c23e279fcecdfea5a1ff4c71cd3c3b0e49bb5b: Status 404 returned error can't find the container with id 3d410fdd41d67876db98c07a53c23e279fcecdfea5a1ff4c71cd3c3b0e49bb5b Jul 23 22:31:10.603920 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:31:10.603852 2578 generic.go:356] "Generic (PLEG): container finished" podID="d4d8ffc1-d0a1-4ee2-badf-ba435136fdb7" containerID="144447995bb84a8f2812718ad334915aa7159ef3abb5003cc56bf6cb6f931aaa" exitCode=0 Jul 23 22:31:10.603920 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:31:10.603897 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c3321b-7b9f5" event={"ID":"d4d8ffc1-d0a1-4ee2-badf-ba435136fdb7","Type":"ContainerDied","Data":"144447995bb84a8f2812718ad334915aa7159ef3abb5003cc56bf6cb6f931aaa"} Jul 23 22:31:10.604393 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:31:10.603992 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c3321b-7b9f5" event={"ID":"d4d8ffc1-d0a1-4ee2-badf-ba435136fdb7","Type":"ContainerStarted","Data":"3d410fdd41d67876db98c07a53c23e279fcecdfea5a1ff4c71cd3c3b0e49bb5b"} Jul 23 22:31:11.563693 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:31:11.563658 2578 patch_prober.go:28] interesting pod/e2e-api-b-fc9d5e-openshift-default-d7c7f87fc-qcvv5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 23 22:31:11.563866 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:31:11.563746 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-fc9d5e-openshift-default-d7c7f87fc-qcvv5" podUID="0316ef32-8546-41c6-a32c-0613fff7599a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 23 22:31:11.739111 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:31:11.739091 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c3321b-7b9f5" Jul 23 22:31:11.894808 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:31:11.894776 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d4d8ffc1-d0a1-4ee2-badf-ba435136fdb7-maas-api-service-ca\") pod \"d4d8ffc1-d0a1-4ee2-badf-ba435136fdb7\" (UID: \"d4d8ffc1-d0a1-4ee2-badf-ba435136fdb7\") " Jul 23 22:31:11.895103 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:31:11.895083 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d4d8ffc1-d0a1-4ee2-badf-ba435136fdb7-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d4d8ffc1-d0a1-4ee2-badf-ba435136fdb7" (UID: "d4d8ffc1-d0a1-4ee2-badf-ba435136fdb7"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 22:31:11.996183 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:31:11.996131 2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d4d8ffc1-d0a1-4ee2-badf-ba435136fdb7-maas-api-service-ca\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\"" Jul 23 22:31:12.617425 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:31:12.617396 2578 generic.go:356] "Generic (PLEG): container finished" podID="0316ef32-8546-41c6-a32c-0613fff7599a" containerID="a3e1d1cd58c5bfd898191d56ba4fad5f38d609e2de9d0c1515f30603ee7567a2" exitCode=0 Jul 23 22:31:12.617624 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:31:12.617468 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-fc9d5e-openshift-default-d7c7f87fc-qcvv5" event={"ID":"0316ef32-8546-41c6-a32c-0613fff7599a","Type":"ContainerDied","Data":"a3e1d1cd58c5bfd898191d56ba4fad5f38d609e2de9d0c1515f30603ee7567a2"} Jul 23 22:31:12.617943 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:31:12.617925 2578 scope.go:117] "RemoveContainer" containerID="a3e1d1cd58c5bfd898191d56ba4fad5f38d609e2de9d0c1515f30603ee7567a2" Jul 23 22:31:12.618943 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:31:12.618924 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c3321b-7b9f5" Jul 23 22:31:12.619043 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:31:12.618941 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c3321b-7b9f5" event={"ID":"d4d8ffc1-d0a1-4ee2-badf-ba435136fdb7","Type":"ContainerDied","Data":"3d410fdd41d67876db98c07a53c23e279fcecdfea5a1ff4c71cd3c3b0e49bb5b"} Jul 23 22:31:12.619043 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:31:12.618969 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 23 22:31:12.619043 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:31:12.619009 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3d410fdd41d67876db98c07a53c23e279fcecdfea5a1ff4c71cd3c3b0e49bb5b" Jul 23 22:31:12.619194 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:31:12.619045 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 23 22:31:12.619194 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:31:12.619077 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 23 22:31:13.625362 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:31:13.625330 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-fc9d5e-openshift-default-d7c7f87fc-qcvv5" event={"ID":"0316ef32-8546-41c6-a32c-0613fff7599a","Type":"ContainerStarted","Data":"c685c80607672253b72fb238c815c85ce95f33888baa745e419b45c04aed4433"} Jul 23 22:31:13.630622 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:31:13.630604 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-fc9d5e-openshift-default-d7c7f87fc-qcvv5" Jul 23 22:31:13.630846 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:31:13.630826 2578 patch_prober.go:28] interesting pod/e2e-api-b-fc9d5e-openshift-default-d7c7f87fc-qcvv5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body= Jul 23 22:31:13.630887 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:31:13.630873 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-fc9d5e-openshift-default-d7c7f87fc-qcvv5" podUID="0316ef32-8546-41c6-a32c-0613fff7599a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" Jul 23 22:31:14.631331 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:31:14.631295 2578 patch_prober.go:28] interesting pod/e2e-api-b-fc9d5e-openshift-default-d7c7f87fc-qcvv5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body= Jul 23 22:31:14.631792 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:31:14.631344 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-fc9d5e-openshift-default-d7c7f87fc-qcvv5" podUID="0316ef32-8546-41c6-a32c-0613fff7599a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" Jul 23 22:31:15.631648 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:31:15.631606 2578 patch_prober.go:28] interesting pod/e2e-api-b-fc9d5e-openshift-default-d7c7f87fc-qcvv5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body= Jul 23 22:31:15.632195 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:31:15.631680 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-fc9d5e-openshift-default-d7c7f87fc-qcvv5" podUID="0316ef32-8546-41c6-a32c-0613fff7599a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" Jul 23 22:31:16.630734 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:31:16.630703 2578 patch_prober.go:28] interesting pod/e2e-api-b-fc9d5e-openshift-default-d7c7f87fc-qcvv5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body= Jul 23 22:31:16.630909 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:31:16.630771 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-fc9d5e-openshift-default-d7c7f87fc-qcvv5" podUID="0316ef32-8546-41c6-a32c-0613fff7599a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" Jul 23 22:31:17.634317 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:31:17.634287 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-fc9d5e-openshift-default-d7c7f87fc-qcvv5" Jul 23 22:31:17.634785 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:31:17.634531 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-fc9d5e-openshift-default-d7c7f87fc-qcvv5" Jul 23 22:31:17.635219 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:31:17.635204 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-fc9d5e-openshift-default-d7c7f87fc-qcvv5" Jul 23 22:31:29.325371 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:31:29.325324 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c3321b-7b9f5"] Jul 23 22:31:29.327483 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:31:29.327463 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-c3321b-7b9f5"] Jul 23 22:31:30.253110 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:31:30.253056 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d4d8ffc1-d0a1-4ee2-badf-ba435136fdb7" path="/var/lib/kubelet/pods/d4d8ffc1-d0a1-4ee2-badf-ba435136fdb7/volumes" Jul 23 22:31:38.762368 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:31:38.762280 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7jkjm_7342a9f6-72b1-4e3c-8d50-0b0326454a86/ovn-acl-logging/0.log" Jul 23 22:31:38.765729 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:31:38.765707 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7jkjm_7342a9f6-72b1-4e3c-8d50-0b0326454a86/ovn-acl-logging/0.log" Jul 23 22:32:03.237091 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:32:03.237003 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-fc9d5e-openshift-default-d7c7f87fc-qcvv5"] Jul 23 22:32:03.237656 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:32:03.237259 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-fc9d5e-openshift-default-d7c7f87fc-qcvv5" podUID="0316ef32-8546-41c6-a32c-0613fff7599a" containerName="istio-proxy" containerID="cri-o://c685c80607672253b72fb238c815c85ce95f33888baa745e419b45c04aed4433" gracePeriod=30 Jul 23 22:32:08.488694 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:32:08.488670 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-fc9d5e-openshift-default-d7c7f87fc-qcvv5" Jul 23 22:32:08.665927 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:32:08.665901 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0316ef32-8546-41c6-a32c-0613fff7599a-istio-envoy\") pod \"0316ef32-8546-41c6-a32c-0613fff7599a\" (UID: \"0316ef32-8546-41c6-a32c-0613fff7599a\") " Jul 23 22:32:08.666054 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:32:08.665933 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0316ef32-8546-41c6-a32c-0613fff7599a-istiod-ca-cert\") pod \"0316ef32-8546-41c6-a32c-0613fff7599a\" (UID: \"0316ef32-8546-41c6-a32c-0613fff7599a\") " Jul 23 22:32:08.666054 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:32:08.665957 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0316ef32-8546-41c6-a32c-0613fff7599a-istio-token\") pod \"0316ef32-8546-41c6-a32c-0613fff7599a\" (UID: \"0316ef32-8546-41c6-a32c-0613fff7599a\") " Jul 23 22:32:08.666054 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:32:08.666003 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0316ef32-8546-41c6-a32c-0613fff7599a-istio-podinfo\") pod \"0316ef32-8546-41c6-a32c-0613fff7599a\" (UID: \"0316ef32-8546-41c6-a32c-0613fff7599a\") " Jul 23 22:32:08.666054 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:32:08.666038 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0316ef32-8546-41c6-a32c-0613fff7599a-istio-data\") pod \"0316ef32-8546-41c6-a32c-0613fff7599a\" (UID: \"0316ef32-8546-41c6-a32c-0613fff7599a\") " Jul 23 22:32:08.666237 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:32:08.666073 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-279q9\" (UniqueName: \"kubernetes.io/projected/0316ef32-8546-41c6-a32c-0613fff7599a-kube-api-access-279q9\") pod \"0316ef32-8546-41c6-a32c-0613fff7599a\" (UID: \"0316ef32-8546-41c6-a32c-0613fff7599a\") " Jul 23 22:32:08.666237 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:32:08.666204 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0316ef32-8546-41c6-a32c-0613fff7599a-workload-certs\") pod \"0316ef32-8546-41c6-a32c-0613fff7599a\" (UID: \"0316ef32-8546-41c6-a32c-0613fff7599a\") " Jul 23 22:32:08.666351 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:32:08.666321 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0316ef32-8546-41c6-a32c-0613fff7599a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "0316ef32-8546-41c6-a32c-0613fff7599a" (UID: "0316ef32-8546-41c6-a32c-0613fff7599a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 22:32:08.666482 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:32:08.666444 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0316ef32-8546-41c6-a32c-0613fff7599a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "0316ef32-8546-41c6-a32c-0613fff7599a" (UID: "0316ef32-8546-41c6-a32c-0613fff7599a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 22:32:08.666482 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:32:08.666470 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0316ef32-8546-41c6-a32c-0613fff7599a-credential-socket\") pod \"0316ef32-8546-41c6-a32c-0613fff7599a\" (UID: \"0316ef32-8546-41c6-a32c-0613fff7599a\") " Jul 23 22:32:08.666693 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:32:08.666500 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0316ef32-8546-41c6-a32c-0613fff7599a-workload-socket\") pod \"0316ef32-8546-41c6-a32c-0613fff7599a\" (UID: \"0316ef32-8546-41c6-a32c-0613fff7599a\") " Jul 23 22:32:08.666793 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:32:08.666767 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0316ef32-8546-41c6-a32c-0613fff7599a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "0316ef32-8546-41c6-a32c-0613fff7599a" (UID: "0316ef32-8546-41c6-a32c-0613fff7599a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 22:32:08.666921 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:32:08.666817 2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0316ef32-8546-41c6-a32c-0613fff7599a-istiod-ca-cert\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\"" Jul 23 22:32:08.666921 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:32:08.666830 2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0316ef32-8546-41c6-a32c-0613fff7599a-workload-certs\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\"" Jul 23 22:32:08.667022 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:32:08.666977 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0316ef32-8546-41c6-a32c-0613fff7599a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "0316ef32-8546-41c6-a32c-0613fff7599a" (UID: "0316ef32-8546-41c6-a32c-0613fff7599a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 22:32:08.667076 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:32:08.667056 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0316ef32-8546-41c6-a32c-0613fff7599a-istio-data" (OuterVolumeSpecName: "istio-data") pod "0316ef32-8546-41c6-a32c-0613fff7599a" (UID: "0316ef32-8546-41c6-a32c-0613fff7599a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 22:32:08.668757 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:32:08.668729 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0316ef32-8546-41c6-a32c-0613fff7599a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "0316ef32-8546-41c6-a32c-0613fff7599a" (UID: "0316ef32-8546-41c6-a32c-0613fff7599a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 23 22:32:08.668857 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:32:08.668759 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0316ef32-8546-41c6-a32c-0613fff7599a-istio-token" (OuterVolumeSpecName: "istio-token") pod "0316ef32-8546-41c6-a32c-0613fff7599a" (UID: "0316ef32-8546-41c6-a32c-0613fff7599a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 22:32:08.668857 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:32:08.668790 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0316ef32-8546-41c6-a32c-0613fff7599a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "0316ef32-8546-41c6-a32c-0613fff7599a" (UID: "0316ef32-8546-41c6-a32c-0613fff7599a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 22:32:08.668857 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:32:08.668796 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0316ef32-8546-41c6-a32c-0613fff7599a-kube-api-access-279q9" (OuterVolumeSpecName: "kube-api-access-279q9") pod "0316ef32-8546-41c6-a32c-0613fff7599a" (UID: "0316ef32-8546-41c6-a32c-0613fff7599a"). InnerVolumeSpecName "kube-api-access-279q9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 22:32:08.767853 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:32:08.767825 2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0316ef32-8546-41c6-a32c-0613fff7599a-istio-envoy\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\"" Jul 23 22:32:08.767853 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:32:08.767850 2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0316ef32-8546-41c6-a32c-0613fff7599a-istio-token\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\"" Jul 23 22:32:08.768027 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:32:08.767859 2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0316ef32-8546-41c6-a32c-0613fff7599a-istio-podinfo\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\"" Jul 23 22:32:08.768027 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:32:08.767866 2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0316ef32-8546-41c6-a32c-0613fff7599a-istio-data\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\"" Jul 23 22:32:08.768027 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:32:08.767876 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-279q9\" (UniqueName: \"kubernetes.io/projected/0316ef32-8546-41c6-a32c-0613fff7599a-kube-api-access-279q9\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\"" Jul 23 22:32:08.768027 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:32:08.767888 2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0316ef32-8546-41c6-a32c-0613fff7599a-credential-socket\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\"" Jul 23 22:32:08.768027 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:32:08.767896 2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0316ef32-8546-41c6-a32c-0613fff7599a-workload-socket\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\"" Jul 23 22:32:08.910461 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:32:08.910428 2578 generic.go:356] "Generic (PLEG): container finished" podID="0316ef32-8546-41c6-a32c-0613fff7599a" containerID="c685c80607672253b72fb238c815c85ce95f33888baa745e419b45c04aed4433" exitCode=0 Jul 23 22:32:08.910628 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:32:08.910476 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-fc9d5e-openshift-default-d7c7f87fc-qcvv5" Jul 23 22:32:08.910628 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:32:08.910493 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-fc9d5e-openshift-default-d7c7f87fc-qcvv5" event={"ID":"0316ef32-8546-41c6-a32c-0613fff7599a","Type":"ContainerDied","Data":"c685c80607672253b72fb238c815c85ce95f33888baa745e419b45c04aed4433"} Jul 23 22:32:08.910628 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:32:08.910557 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-fc9d5e-openshift-default-d7c7f87fc-qcvv5" event={"ID":"0316ef32-8546-41c6-a32c-0613fff7599a","Type":"ContainerDied","Data":"32eba0f2e58b9e58365f39afcfcac2f24fd9e1560bbd60b8ebdbe0716f8593ae"} Jul 23 22:32:08.910628 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:32:08.910579 2578 scope.go:117] "RemoveContainer" containerID="c685c80607672253b72fb238c815c85ce95f33888baa745e419b45c04aed4433" Jul 23 22:32:08.919719 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:32:08.919704 2578 scope.go:117] "RemoveContainer" containerID="a3e1d1cd58c5bfd898191d56ba4fad5f38d609e2de9d0c1515f30603ee7567a2" Jul 23 22:32:08.927612 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:32:08.927594 2578 scope.go:117] "RemoveContainer" containerID="c685c80607672253b72fb238c815c85ce95f33888baa745e419b45c04aed4433" Jul 23 22:32:08.927863 ip-10-0-136-157 kubenswrapper[2578]: E0723 22:32:08.927844 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c685c80607672253b72fb238c815c85ce95f33888baa745e419b45c04aed4433\": container with ID starting with c685c80607672253b72fb238c815c85ce95f33888baa745e419b45c04aed4433 not found: ID does not exist" containerID="c685c80607672253b72fb238c815c85ce95f33888baa745e419b45c04aed4433" Jul 23 22:32:08.927922 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:32:08.927871 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c685c80607672253b72fb238c815c85ce95f33888baa745e419b45c04aed4433"} err="failed to get container status \"c685c80607672253b72fb238c815c85ce95f33888baa745e419b45c04aed4433\": rpc error: code = NotFound desc = could not find container \"c685c80607672253b72fb238c815c85ce95f33888baa745e419b45c04aed4433\": container with ID starting with c685c80607672253b72fb238c815c85ce95f33888baa745e419b45c04aed4433 not found: ID does not exist" Jul 23 22:32:08.927922 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:32:08.927889 2578 scope.go:117] "RemoveContainer" containerID="a3e1d1cd58c5bfd898191d56ba4fad5f38d609e2de9d0c1515f30603ee7567a2" Jul 23 22:32:08.928101 ip-10-0-136-157 kubenswrapper[2578]: E0723 22:32:08.928081 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a3e1d1cd58c5bfd898191d56ba4fad5f38d609e2de9d0c1515f30603ee7567a2\": container with ID starting with a3e1d1cd58c5bfd898191d56ba4fad5f38d609e2de9d0c1515f30603ee7567a2 not found: ID does not exist" containerID="a3e1d1cd58c5bfd898191d56ba4fad5f38d609e2de9d0c1515f30603ee7567a2" Jul 23 22:32:08.928146 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:32:08.928109 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a3e1d1cd58c5bfd898191d56ba4fad5f38d609e2de9d0c1515f30603ee7567a2"} err="failed to get container status \"a3e1d1cd58c5bfd898191d56ba4fad5f38d609e2de9d0c1515f30603ee7567a2\": rpc error: code = NotFound desc = could not find container \"a3e1d1cd58c5bfd898191d56ba4fad5f38d609e2de9d0c1515f30603ee7567a2\": container with ID starting with a3e1d1cd58c5bfd898191d56ba4fad5f38d609e2de9d0c1515f30603ee7567a2 not found: ID does not exist" Jul 23 22:32:08.932988 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:32:08.932963 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-fc9d5e-openshift-default-d7c7f87fc-qcvv5"] Jul 23 22:32:08.936847 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:32:08.936826 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-fc9d5e-openshift-default-d7c7f87fc-qcvv5"] Jul 23 22:32:10.251644 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:32:10.251606 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0316ef32-8546-41c6-a32c-0613fff7599a" path="/var/lib/kubelet/pods/0316ef32-8546-41c6-a32c-0613fff7599a/volumes" Jul 23 22:32:12.510528 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:32:12.510487 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-091ba5-openshift-default-7ffff75dff-xp5hx"] Jul 23 22:32:12.510975 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:32:12.510959 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4d8ffc1-d0a1-4ee2-badf-ba435136fdb7" containerName="revoke-keys" Jul 23 22:32:12.510975 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:32:12.510973 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4d8ffc1-d0a1-4ee2-badf-ba435136fdb7" containerName="revoke-keys" Jul 23 22:32:12.511050 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:32:12.510996 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0316ef32-8546-41c6-a32c-0613fff7599a" containerName="istio-proxy" Jul 23 22:32:12.511050 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:32:12.511004 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="0316ef32-8546-41c6-a32c-0613fff7599a" containerName="istio-proxy" Jul 23 22:32:12.511050 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:32:12.511015 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0316ef32-8546-41c6-a32c-0613fff7599a" containerName="istio-proxy" Jul 23 22:32:12.511050 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:32:12.511024 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="0316ef32-8546-41c6-a32c-0613fff7599a" containerName="istio-proxy" Jul 23 22:32:12.511171 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:32:12.511112 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="d4d8ffc1-d0a1-4ee2-badf-ba435136fdb7" containerName="revoke-keys" Jul 23 22:32:12.511171 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:32:12.511124 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="0316ef32-8546-41c6-a32c-0613fff7599a" containerName="istio-proxy" Jul 23 22:32:12.511171 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:32:12.511133 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="0316ef32-8546-41c6-a32c-0613fff7599a" containerName="istio-proxy" Jul 23 22:32:12.514575 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:32:12.514557 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-091ba5-openshift-default-7ffff75dff-xp5hx" Jul 23 22:32:12.516794 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:32:12.516770 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-091ba5-openshift-default-dockercfg-kkzmk\"" Jul 23 22:32:12.524907 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:32:12.524886 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-091ba5-openshift-default-7ffff75dff-xp5hx"] Jul 23 22:32:12.696406 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:32:12.696367 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/78092b11-2836-4d6e-b331-9a88f80670ba-workload-certs\") pod \"e2e-shared-b-091ba5-openshift-default-7ffff75dff-xp5hx\" (UID: \"78092b11-2836-4d6e-b331-9a88f80670ba\") " pod="openshift-ingress/e2e-shared-b-091ba5-openshift-default-7ffff75dff-xp5hx" Jul 23 22:32:12.696587 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:32:12.696410 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/78092b11-2836-4d6e-b331-9a88f80670ba-istiod-ca-cert\") pod \"e2e-shared-b-091ba5-openshift-default-7ffff75dff-xp5hx\" (UID: \"78092b11-2836-4d6e-b331-9a88f80670ba\") " pod="openshift-ingress/e2e-shared-b-091ba5-openshift-default-7ffff75dff-xp5hx" Jul 23 22:32:12.696587 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:32:12.696439 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kw452\" (UniqueName: \"kubernetes.io/projected/78092b11-2836-4d6e-b331-9a88f80670ba-kube-api-access-kw452\") pod \"e2e-shared-b-091ba5-openshift-default-7ffff75dff-xp5hx\" (UID: \"78092b11-2836-4d6e-b331-9a88f80670ba\") " pod="openshift-ingress/e2e-shared-b-091ba5-openshift-default-7ffff75dff-xp5hx" Jul 23 22:32:12.696587 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:32:12.696502 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/78092b11-2836-4d6e-b331-9a88f80670ba-workload-socket\") pod \"e2e-shared-b-091ba5-openshift-default-7ffff75dff-xp5hx\" (UID: \"78092b11-2836-4d6e-b331-9a88f80670ba\") " pod="openshift-ingress/e2e-shared-b-091ba5-openshift-default-7ffff75dff-xp5hx" Jul 23 22:32:12.696587 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:32:12.696525 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/78092b11-2836-4d6e-b331-9a88f80670ba-credential-socket\") pod \"e2e-shared-b-091ba5-openshift-default-7ffff75dff-xp5hx\" (UID: \"78092b11-2836-4d6e-b331-9a88f80670ba\") " pod="openshift-ingress/e2e-shared-b-091ba5-openshift-default-7ffff75dff-xp5hx" Jul 23 22:32:12.696746 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:32:12.696589 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/78092b11-2836-4d6e-b331-9a88f80670ba-istio-envoy\") pod \"e2e-shared-b-091ba5-openshift-default-7ffff75dff-xp5hx\" (UID: \"78092b11-2836-4d6e-b331-9a88f80670ba\") " pod="openshift-ingress/e2e-shared-b-091ba5-openshift-default-7ffff75dff-xp5hx" Jul 23 22:32:12.696746 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:32:12.696629 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/78092b11-2836-4d6e-b331-9a88f80670ba-istio-podinfo\") pod \"e2e-shared-b-091ba5-openshift-default-7ffff75dff-xp5hx\" (UID: \"78092b11-2836-4d6e-b331-9a88f80670ba\") " pod="openshift-ingress/e2e-shared-b-091ba5-openshift-default-7ffff75dff-xp5hx" Jul 23 22:32:12.696746 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:32:12.696665 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/78092b11-2836-4d6e-b331-9a88f80670ba-istio-token\") pod \"e2e-shared-b-091ba5-openshift-default-7ffff75dff-xp5hx\" (UID: \"78092b11-2836-4d6e-b331-9a88f80670ba\") " pod="openshift-ingress/e2e-shared-b-091ba5-openshift-default-7ffff75dff-xp5hx" Jul 23 22:32:12.696746 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:32:12.696702 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/78092b11-2836-4d6e-b331-9a88f80670ba-istio-data\") pod \"e2e-shared-b-091ba5-openshift-default-7ffff75dff-xp5hx\" (UID: \"78092b11-2836-4d6e-b331-9a88f80670ba\") " pod="openshift-ingress/e2e-shared-b-091ba5-openshift-default-7ffff75dff-xp5hx" Jul 23 22:32:12.797290 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:32:12.797224 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/78092b11-2836-4d6e-b331-9a88f80670ba-istio-token\") pod \"e2e-shared-b-091ba5-openshift-default-7ffff75dff-xp5hx\" (UID: \"78092b11-2836-4d6e-b331-9a88f80670ba\") " pod="openshift-ingress/e2e-shared-b-091ba5-openshift-default-7ffff75dff-xp5hx" Jul 23 22:32:12.797290 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:32:12.797260 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/78092b11-2836-4d6e-b331-9a88f80670ba-istio-data\") pod \"e2e-shared-b-091ba5-openshift-default-7ffff75dff-xp5hx\" (UID: \"78092b11-2836-4d6e-b331-9a88f80670ba\") " pod="openshift-ingress/e2e-shared-b-091ba5-openshift-default-7ffff75dff-xp5hx" Jul 23 22:32:12.797290 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:32:12.797290 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/78092b11-2836-4d6e-b331-9a88f80670ba-workload-certs\") pod \"e2e-shared-b-091ba5-openshift-default-7ffff75dff-xp5hx\" (UID: \"78092b11-2836-4d6e-b331-9a88f80670ba\") " pod="openshift-ingress/e2e-shared-b-091ba5-openshift-default-7ffff75dff-xp5hx" Jul 23 22:32:12.797526 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:32:12.797307 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/78092b11-2836-4d6e-b331-9a88f80670ba-istiod-ca-cert\") pod \"e2e-shared-b-091ba5-openshift-default-7ffff75dff-xp5hx\" (UID: \"78092b11-2836-4d6e-b331-9a88f80670ba\") " pod="openshift-ingress/e2e-shared-b-091ba5-openshift-default-7ffff75dff-xp5hx" Jul 23 22:32:12.797526 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:32:12.797324 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kw452\" (UniqueName: \"kubernetes.io/projected/78092b11-2836-4d6e-b331-9a88f80670ba-kube-api-access-kw452\") pod \"e2e-shared-b-091ba5-openshift-default-7ffff75dff-xp5hx\" (UID: \"78092b11-2836-4d6e-b331-9a88f80670ba\") " pod="openshift-ingress/e2e-shared-b-091ba5-openshift-default-7ffff75dff-xp5hx" Jul 23 22:32:12.797526 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:32:12.797389 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/78092b11-2836-4d6e-b331-9a88f80670ba-workload-socket\") pod \"e2e-shared-b-091ba5-openshift-default-7ffff75dff-xp5hx\" (UID: \"78092b11-2836-4d6e-b331-9a88f80670ba\") " pod="openshift-ingress/e2e-shared-b-091ba5-openshift-default-7ffff75dff-xp5hx" Jul 23 22:32:12.797526 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:32:12.797412 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/78092b11-2836-4d6e-b331-9a88f80670ba-credential-socket\") pod \"e2e-shared-b-091ba5-openshift-default-7ffff75dff-xp5hx\" (UID: \"78092b11-2836-4d6e-b331-9a88f80670ba\") " pod="openshift-ingress/e2e-shared-b-091ba5-openshift-default-7ffff75dff-xp5hx" Jul 23 22:32:12.797526 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:32:12.797435 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/78092b11-2836-4d6e-b331-9a88f80670ba-istio-envoy\") pod \"e2e-shared-b-091ba5-openshift-default-7ffff75dff-xp5hx\" (UID: \"78092b11-2836-4d6e-b331-9a88f80670ba\") " pod="openshift-ingress/e2e-shared-b-091ba5-openshift-default-7ffff75dff-xp5hx" Jul 23 22:32:12.797526 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:32:12.797469 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/78092b11-2836-4d6e-b331-9a88f80670ba-istio-podinfo\") pod \"e2e-shared-b-091ba5-openshift-default-7ffff75dff-xp5hx\" (UID: \"78092b11-2836-4d6e-b331-9a88f80670ba\") " pod="openshift-ingress/e2e-shared-b-091ba5-openshift-default-7ffff75dff-xp5hx" Jul 23 22:32:12.797831 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:32:12.797682 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/78092b11-2836-4d6e-b331-9a88f80670ba-istio-data\") pod \"e2e-shared-b-091ba5-openshift-default-7ffff75dff-xp5hx\" (UID: \"78092b11-2836-4d6e-b331-9a88f80670ba\") " pod="openshift-ingress/e2e-shared-b-091ba5-openshift-default-7ffff75dff-xp5hx" Jul 23 22:32:12.797831 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:32:12.797736 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/78092b11-2836-4d6e-b331-9a88f80670ba-workload-certs\") pod \"e2e-shared-b-091ba5-openshift-default-7ffff75dff-xp5hx\" (UID: \"78092b11-2836-4d6e-b331-9a88f80670ba\") " pod="openshift-ingress/e2e-shared-b-091ba5-openshift-default-7ffff75dff-xp5hx" Jul 23 22:32:12.797831 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:32:12.797788 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/78092b11-2836-4d6e-b331-9a88f80670ba-workload-socket\") pod \"e2e-shared-b-091ba5-openshift-default-7ffff75dff-xp5hx\" (UID: \"78092b11-2836-4d6e-b331-9a88f80670ba\") " pod="openshift-ingress/e2e-shared-b-091ba5-openshift-default-7ffff75dff-xp5hx" Jul 23 22:32:12.797924 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:32:12.797858 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/78092b11-2836-4d6e-b331-9a88f80670ba-credential-socket\") pod \"e2e-shared-b-091ba5-openshift-default-7ffff75dff-xp5hx\" (UID: \"78092b11-2836-4d6e-b331-9a88f80670ba\") " pod="openshift-ingress/e2e-shared-b-091ba5-openshift-default-7ffff75dff-xp5hx" Jul 23 22:32:12.798092 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:32:12.798066 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/78092b11-2836-4d6e-b331-9a88f80670ba-istiod-ca-cert\") pod \"e2e-shared-b-091ba5-openshift-default-7ffff75dff-xp5hx\" (UID: \"78092b11-2836-4d6e-b331-9a88f80670ba\") " pod="openshift-ingress/e2e-shared-b-091ba5-openshift-default-7ffff75dff-xp5hx" Jul 23 22:32:12.799910 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:32:12.799888 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/78092b11-2836-4d6e-b331-9a88f80670ba-istio-envoy\") pod \"e2e-shared-b-091ba5-openshift-default-7ffff75dff-xp5hx\" (UID: \"78092b11-2836-4d6e-b331-9a88f80670ba\") " pod="openshift-ingress/e2e-shared-b-091ba5-openshift-default-7ffff75dff-xp5hx" Jul 23 22:32:12.800133 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:32:12.800114 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/78092b11-2836-4d6e-b331-9a88f80670ba-istio-podinfo\") pod \"e2e-shared-b-091ba5-openshift-default-7ffff75dff-xp5hx\" (UID: \"78092b11-2836-4d6e-b331-9a88f80670ba\") " pod="openshift-ingress/e2e-shared-b-091ba5-openshift-default-7ffff75dff-xp5hx" Jul 23 22:32:12.805854 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:32:12.805828 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/78092b11-2836-4d6e-b331-9a88f80670ba-istio-token\") pod \"e2e-shared-b-091ba5-openshift-default-7ffff75dff-xp5hx\" (UID: \"78092b11-2836-4d6e-b331-9a88f80670ba\") " pod="openshift-ingress/e2e-shared-b-091ba5-openshift-default-7ffff75dff-xp5hx" Jul 23 22:32:12.806085 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:32:12.806069 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kw452\" (UniqueName: \"kubernetes.io/projected/78092b11-2836-4d6e-b331-9a88f80670ba-kube-api-access-kw452\") pod \"e2e-shared-b-091ba5-openshift-default-7ffff75dff-xp5hx\" (UID: \"78092b11-2836-4d6e-b331-9a88f80670ba\") " pod="openshift-ingress/e2e-shared-b-091ba5-openshift-default-7ffff75dff-xp5hx" Jul 23 22:32:12.827668 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:32:12.827644 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-091ba5-openshift-default-7ffff75dff-xp5hx" Jul 23 22:32:12.966202 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:32:12.966176 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-091ba5-openshift-default-7ffff75dff-xp5hx"] Jul 23 22:32:12.967942 ip-10-0-136-157 kubenswrapper[2578]: W0723 22:32:12.967911 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78092b11_2836_4d6e_b331_9a88f80670ba.slice/crio-241984db1ab3dfc470e511527ae68af01519afd2714a29624abecb3b5108bf25 WatchSource:0}: Error finding container 241984db1ab3dfc470e511527ae68af01519afd2714a29624abecb3b5108bf25: Status 404 returned error can't find the container with id 241984db1ab3dfc470e511527ae68af01519afd2714a29624abecb3b5108bf25 Jul 23 22:32:12.969917 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:32:12.969886 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 23 22:32:12.969994 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:32:12.969949 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 23 22:32:12.969994 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:32:12.969978 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 23 22:32:13.939799 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:32:13.939767 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-091ba5-openshift-default-7ffff75dff-xp5hx" event={"ID":"78092b11-2836-4d6e-b331-9a88f80670ba","Type":"ContainerStarted","Data":"72acd13d1ff2fe5cf082610174f390bb4632929775c4b1ff8696deff4c1f559f"} Jul 23 22:32:13.940125 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:32:13.939827 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-091ba5-openshift-default-7ffff75dff-xp5hx" event={"ID":"78092b11-2836-4d6e-b331-9a88f80670ba","Type":"ContainerStarted","Data":"241984db1ab3dfc470e511527ae68af01519afd2714a29624abecb3b5108bf25"} Jul 23 22:32:13.970560 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:32:13.970497 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-091ba5-openshift-default-7ffff75dff-xp5hx" podStartSLOduration=1.9704831889999999 podStartE2EDuration="1.970483189s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 22:32:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 22:32:13.969234401 +0000 UTC m=+2436.211142266" watchObservedRunningTime="2026-07-23 22:32:13.970483189 +0000 UTC m=+2436.212391061" Jul 23 22:32:14.828353 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:32:14.828325 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-091ba5-openshift-default-7ffff75dff-xp5hx" Jul 23 22:32:14.833005 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:32:14.832983 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-091ba5-openshift-default-7ffff75dff-xp5hx" Jul 23 22:32:14.945071 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:32:14.945038 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-091ba5-openshift-default-7ffff75dff-xp5hx" Jul 23 22:32:14.945962 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:32:14.945945 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-091ba5-openshift-default-7ffff75dff-xp5hx" Jul 23 22:32:30.986704 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:32:30.986665 2578 patch_prober.go:28] interesting pod/e2e-shared-b-091ba5-openshift-default-7ffff75dff-xp5hx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 23 22:32:30.987066 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:32:30.986734 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-091ba5-openshift-default-7ffff75dff-xp5hx" podUID="78092b11-2836-4d6e-b331-9a88f80670ba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 23 22:32:32.038246 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:32:32.038217 2578 generic.go:356] "Generic (PLEG): container finished" podID="78092b11-2836-4d6e-b331-9a88f80670ba" containerID="72acd13d1ff2fe5cf082610174f390bb4632929775c4b1ff8696deff4c1f559f" exitCode=0 Jul 23 22:32:32.038644 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:32:32.038299 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-091ba5-openshift-default-7ffff75dff-xp5hx" event={"ID":"78092b11-2836-4d6e-b331-9a88f80670ba","Type":"ContainerDied","Data":"72acd13d1ff2fe5cf082610174f390bb4632929775c4b1ff8696deff4c1f559f"} Jul 23 22:32:32.038716 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:32:32.038699 2578 scope.go:117] "RemoveContainer" containerID="72acd13d1ff2fe5cf082610174f390bb4632929775c4b1ff8696deff4c1f559f" Jul 23 22:32:32.039731 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:32:32.039707 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 23 22:32:32.039800 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:32:32.039773 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 23 22:32:32.039835 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:32:32.039805 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 23 22:32:33.048987 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:32:33.048939 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-091ba5-openshift-default-7ffff75dff-xp5hx" event={"ID":"78092b11-2836-4d6e-b331-9a88f80670ba","Type":"ContainerStarted","Data":"fd208f550533fe8d312addfb0c20932d6bb558fe5de13a08a34b778889b2a23d"} Jul 23 22:32:33.828766 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:32:33.828736 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-091ba5-openshift-default-7ffff75dff-xp5hx" Jul 23 22:32:33.828949 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:32:33.828851 2578 patch_prober.go:28] interesting pod/e2e-shared-b-091ba5-openshift-default-7ffff75dff-xp5hx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body= Jul 23 22:32:33.828949 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:32:33.828905 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-091ba5-openshift-default-7ffff75dff-xp5hx" podUID="78092b11-2836-4d6e-b331-9a88f80670ba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" Jul 23 22:32:34.829015 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:32:34.828983 2578 patch_prober.go:28] interesting pod/e2e-shared-b-091ba5-openshift-default-7ffff75dff-xp5hx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body= Jul 23 22:32:34.829444 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:32:34.829034 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-091ba5-openshift-default-7ffff75dff-xp5hx" podUID="78092b11-2836-4d6e-b331-9a88f80670ba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" Jul 23 22:32:35.828577 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:32:35.828520 2578 patch_prober.go:28] interesting pod/e2e-shared-b-091ba5-openshift-default-7ffff75dff-xp5hx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body= Jul 23 22:32:35.828742 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:32:35.828594 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-091ba5-openshift-default-7ffff75dff-xp5hx" podUID="78092b11-2836-4d6e-b331-9a88f80670ba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" Jul 23 22:32:36.831515 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:32:36.831476 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-091ba5-openshift-default-7ffff75dff-xp5hx" Jul 23 22:32:36.831991 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:32:36.831972 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-091ba5-openshift-default-7ffff75dff-xp5hx" Jul 23 22:32:36.832512 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:32:36.832491 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-091ba5-openshift-default-7ffff75dff-xp5hx" Jul 23 22:36:38.437195 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:36:38.437132 2578 scope.go:117] "RemoveContainer" containerID="d1daff7a09c9c3300e6cfd7572d0115022a592565facbe438b58737f6b177695" Jul 23 22:36:38.796287 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:36:38.796209 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7jkjm_7342a9f6-72b1-4e3c-8d50-0b0326454a86/ovn-acl-logging/0.log" Jul 23 22:36:38.800088 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:36:38.800069 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7jkjm_7342a9f6-72b1-4e3c-8d50-0b0326454a86/ovn-acl-logging/0.log" Jul 23 22:37:38.456581 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:37:38.456525 2578 scope.go:117] "RemoveContainer" containerID="144447995bb84a8f2812718ad334915aa7159ef3abb5003cc56bf6cb6f931aaa" Jul 23 22:38:52.555681 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:38:52.555641 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-a1850d-5lq78"] Jul 23 22:38:52.558735 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:38:52.558716 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-a1850d-5lq78" Jul 23 22:38:52.561133 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:38:52.561113 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 23 22:38:52.561133 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:38:52.561128 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-lq4mm\"" Jul 23 22:38:52.567653 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:38:52.567631 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-a1850d-5lq78"] Jul 23 22:38:52.669559 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:38:52.669515 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9d992d9c-ec8c-4ab3-bdb2-6034ffcf1cfe-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-a1850d-5lq78\" (UID: \"9d992d9c-ec8c-4ab3-bdb2-6034ffcf1cfe\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-a1850d-5lq78" Jul 23 22:38:52.770366 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:38:52.770346 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9d992d9c-ec8c-4ab3-bdb2-6034ffcf1cfe-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-a1850d-5lq78\" (UID: \"9d992d9c-ec8c-4ab3-bdb2-6034ffcf1cfe\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-a1850d-5lq78" Jul 23 22:38:52.770963 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:38:52.770944 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9d992d9c-ec8c-4ab3-bdb2-6034ffcf1cfe-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-a1850d-5lq78\" (UID: \"9d992d9c-ec8c-4ab3-bdb2-6034ffcf1cfe\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-a1850d-5lq78" Jul 23 22:38:52.869332 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:38:52.869281 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-a1850d-5lq78" Jul 23 22:38:53.000093 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:38:53.000070 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-a1850d-5lq78"] Jul 23 22:38:53.001442 ip-10-0-136-157 kubenswrapper[2578]: W0723 22:38:53.001408 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d992d9c_ec8c_4ab3_bdb2_6034ffcf1cfe.slice/crio-140e76c867fb20e823d2369f0f0638964d64c2a302647fb55be95053ab775930 WatchSource:0}: Error finding container 140e76c867fb20e823d2369f0f0638964d64c2a302647fb55be95053ab775930: Status 404 returned error can't find the container with id 140e76c867fb20e823d2369f0f0638964d64c2a302647fb55be95053ab775930 Jul 23 22:38:53.936372 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:38:53.936338 2578 generic.go:356] "Generic (PLEG): container finished" podID="9d992d9c-ec8c-4ab3-bdb2-6034ffcf1cfe" containerID="50dae252fd18a8960c70866cf9834c9a480c83a314008a7c5638a84341ed467c" exitCode=0 Jul 23 22:38:53.936767 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:38:53.936427 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-a1850d-5lq78" event={"ID":"9d992d9c-ec8c-4ab3-bdb2-6034ffcf1cfe","Type":"ContainerDied","Data":"50dae252fd18a8960c70866cf9834c9a480c83a314008a7c5638a84341ed467c"} Jul 23 22:38:53.936767 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:38:53.936480 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-a1850d-5lq78" event={"ID":"9d992d9c-ec8c-4ab3-bdb2-6034ffcf1cfe","Type":"ContainerStarted","Data":"140e76c867fb20e823d2369f0f0638964d64c2a302647fb55be95053ab775930"} Jul 23 22:38:55.067162 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:38:55.067141 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-a1850d-5lq78" Jul 23 22:38:55.192423 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:38:55.192359 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9d992d9c-ec8c-4ab3-bdb2-6034ffcf1cfe-maas-api-service-ca\") pod \"9d992d9c-ec8c-4ab3-bdb2-6034ffcf1cfe\" (UID: \"9d992d9c-ec8c-4ab3-bdb2-6034ffcf1cfe\") " Jul 23 22:38:55.192708 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:38:55.192689 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9d992d9c-ec8c-4ab3-bdb2-6034ffcf1cfe-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9d992d9c-ec8c-4ab3-bdb2-6034ffcf1cfe" (UID: "9d992d9c-ec8c-4ab3-bdb2-6034ffcf1cfe"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 22:38:55.293261 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:38:55.293235 2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9d992d9c-ec8c-4ab3-bdb2-6034ffcf1cfe-maas-api-service-ca\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\"" Jul 23 22:38:55.947498 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:38:55.947466 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-a1850d-5lq78" Jul 23 22:38:55.947714 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:38:55.947497 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-a1850d-5lq78" event={"ID":"9d992d9c-ec8c-4ab3-bdb2-6034ffcf1cfe","Type":"ContainerDied","Data":"140e76c867fb20e823d2369f0f0638964d64c2a302647fb55be95053ab775930"} Jul 23 22:38:55.947714 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:38:55.947573 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="140e76c867fb20e823d2369f0f0638964d64c2a302647fb55be95053ab775930" Jul 23 22:39:07.678200 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:39:07.678169 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-a1850d-5lq78"] Jul 23 22:39:07.683651 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:39:07.683628 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-a1850d-5lq78"] Jul 23 22:39:08.251373 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:39:08.251343 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9d992d9c-ec8c-4ab3-bdb2-6034ffcf1cfe" path="/var/lib/kubelet/pods/9d992d9c-ec8c-4ab3-bdb2-6034ffcf1cfe/volumes" Jul 23 22:39:36.957219 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:39:36.957129 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-091ba5-openshift-default-7ffff75dff-xp5hx"] Jul 23 22:39:36.957729 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:39:36.957342 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-091ba5-openshift-default-7ffff75dff-xp5hx" podUID="78092b11-2836-4d6e-b331-9a88f80670ba" containerName="istio-proxy" containerID="cri-o://fd208f550533fe8d312addfb0c20932d6bb558fe5de13a08a34b778889b2a23d" gracePeriod=30 Jul 23 22:39:42.178857 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:39:42.178826 2578 generic.go:356] "Generic (PLEG): container finished" podID="78092b11-2836-4d6e-b331-9a88f80670ba" containerID="fd208f550533fe8d312addfb0c20932d6bb558fe5de13a08a34b778889b2a23d" exitCode=0 Jul 23 22:39:42.179195 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:39:42.178865 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-091ba5-openshift-default-7ffff75dff-xp5hx" event={"ID":"78092b11-2836-4d6e-b331-9a88f80670ba","Type":"ContainerDied","Data":"fd208f550533fe8d312addfb0c20932d6bb558fe5de13a08a34b778889b2a23d"} Jul 23 22:39:42.179195 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:39:42.178938 2578 scope.go:117] "RemoveContainer" containerID="72acd13d1ff2fe5cf082610174f390bb4632929775c4b1ff8696deff4c1f559f" Jul 23 22:39:42.215011 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:39:42.214990 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-091ba5-openshift-default-7ffff75dff-xp5hx" Jul 23 22:39:42.264731 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:39:42.264677 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/78092b11-2836-4d6e-b331-9a88f80670ba-workload-socket\") pod \"78092b11-2836-4d6e-b331-9a88f80670ba\" (UID: \"78092b11-2836-4d6e-b331-9a88f80670ba\") " Jul 23 22:39:42.264830 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:39:42.264740 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/78092b11-2836-4d6e-b331-9a88f80670ba-credential-socket\") pod \"78092b11-2836-4d6e-b331-9a88f80670ba\" (UID: \"78092b11-2836-4d6e-b331-9a88f80670ba\") " Jul 23 22:39:42.264830 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:39:42.264762 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kw452\" (UniqueName: \"kubernetes.io/projected/78092b11-2836-4d6e-b331-9a88f80670ba-kube-api-access-kw452\") pod \"78092b11-2836-4d6e-b331-9a88f80670ba\" (UID: \"78092b11-2836-4d6e-b331-9a88f80670ba\") " Jul 23 22:39:42.264830 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:39:42.264793 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/78092b11-2836-4d6e-b331-9a88f80670ba-istiod-ca-cert\") pod \"78092b11-2836-4d6e-b331-9a88f80670ba\" (UID: \"78092b11-2836-4d6e-b331-9a88f80670ba\") " Jul 23 22:39:42.264830 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:39:42.264816 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/78092b11-2836-4d6e-b331-9a88f80670ba-workload-certs\") pod \"78092b11-2836-4d6e-b331-9a88f80670ba\" (UID: \"78092b11-2836-4d6e-b331-9a88f80670ba\") " Jul 23 22:39:42.265003 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:39:42.264838 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/78092b11-2836-4d6e-b331-9a88f80670ba-istio-envoy\") pod \"78092b11-2836-4d6e-b331-9a88f80670ba\" (UID: \"78092b11-2836-4d6e-b331-9a88f80670ba\") " Jul 23 22:39:42.265003 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:39:42.264854 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/78092b11-2836-4d6e-b331-9a88f80670ba-istio-token\") pod \"78092b11-2836-4d6e-b331-9a88f80670ba\" (UID: \"78092b11-2836-4d6e-b331-9a88f80670ba\") " Jul 23 22:39:42.265003 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:39:42.264903 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/78092b11-2836-4d6e-b331-9a88f80670ba-istio-podinfo\") pod \"78092b11-2836-4d6e-b331-9a88f80670ba\" (UID: \"78092b11-2836-4d6e-b331-9a88f80670ba\") " Jul 23 22:39:42.265003 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:39:42.264929 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/78092b11-2836-4d6e-b331-9a88f80670ba-istio-data\") pod \"78092b11-2836-4d6e-b331-9a88f80670ba\" (UID: \"78092b11-2836-4d6e-b331-9a88f80670ba\") " Jul 23 22:39:42.265203 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:39:42.265123 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78092b11-2836-4d6e-b331-9a88f80670ba-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "78092b11-2836-4d6e-b331-9a88f80670ba" (UID: "78092b11-2836-4d6e-b331-9a88f80670ba"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 22:39:42.265203 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:39:42.265138 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78092b11-2836-4d6e-b331-9a88f80670ba-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "78092b11-2836-4d6e-b331-9a88f80670ba" (UID: "78092b11-2836-4d6e-b331-9a88f80670ba"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 22:39:42.265203 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:39:42.265187 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/78092b11-2836-4d6e-b331-9a88f80670ba-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "78092b11-2836-4d6e-b331-9a88f80670ba" (UID: "78092b11-2836-4d6e-b331-9a88f80670ba"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 22:39:42.265364 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:39:42.265309 2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/78092b11-2836-4d6e-b331-9a88f80670ba-credential-socket\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\"" Jul 23 22:39:42.265364 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:39:42.265330 2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/78092b11-2836-4d6e-b331-9a88f80670ba-workload-certs\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\"" Jul 23 22:39:42.265364 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:39:42.265355 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78092b11-2836-4d6e-b331-9a88f80670ba-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "78092b11-2836-4d6e-b331-9a88f80670ba" (UID: "78092b11-2836-4d6e-b331-9a88f80670ba"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 22:39:42.265857 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:39:42.265838 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78092b11-2836-4d6e-b331-9a88f80670ba-istio-data" (OuterVolumeSpecName: "istio-data") pod "78092b11-2836-4d6e-b331-9a88f80670ba" (UID: "78092b11-2836-4d6e-b331-9a88f80670ba"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 22:39:42.266989 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:39:42.266971 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/78092b11-2836-4d6e-b331-9a88f80670ba-kube-api-access-kw452" (OuterVolumeSpecName: "kube-api-access-kw452") pod "78092b11-2836-4d6e-b331-9a88f80670ba" (UID: "78092b11-2836-4d6e-b331-9a88f80670ba"). InnerVolumeSpecName "kube-api-access-kw452". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 22:39:42.267126 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:39:42.267111 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78092b11-2836-4d6e-b331-9a88f80670ba-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "78092b11-2836-4d6e-b331-9a88f80670ba" (UID: "78092b11-2836-4d6e-b331-9a88f80670ba"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 22:39:42.267562 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:39:42.267511 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/78092b11-2836-4d6e-b331-9a88f80670ba-istio-token" (OuterVolumeSpecName: "istio-token") pod "78092b11-2836-4d6e-b331-9a88f80670ba" (UID: "78092b11-2836-4d6e-b331-9a88f80670ba"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 22:39:42.267562 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:39:42.267525 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/78092b11-2836-4d6e-b331-9a88f80670ba-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "78092b11-2836-4d6e-b331-9a88f80670ba" (UID: "78092b11-2836-4d6e-b331-9a88f80670ba"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 23 22:39:42.365738 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:39:42.365711 2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/78092b11-2836-4d6e-b331-9a88f80670ba-workload-socket\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\"" Jul 23 22:39:42.365738 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:39:42.365734 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kw452\" (UniqueName: \"kubernetes.io/projected/78092b11-2836-4d6e-b331-9a88f80670ba-kube-api-access-kw452\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\"" Jul 23 22:39:42.365865 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:39:42.365744 2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/78092b11-2836-4d6e-b331-9a88f80670ba-istiod-ca-cert\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\"" Jul 23 22:39:42.365865 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:39:42.365753 2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/78092b11-2836-4d6e-b331-9a88f80670ba-istio-envoy\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\"" Jul 23 22:39:42.365865 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:39:42.365761 2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/78092b11-2836-4d6e-b331-9a88f80670ba-istio-token\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\"" Jul 23 22:39:42.365865 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:39:42.365768 2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/78092b11-2836-4d6e-b331-9a88f80670ba-istio-podinfo\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\"" Jul 23 22:39:42.365865 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:39:42.365777 2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/78092b11-2836-4d6e-b331-9a88f80670ba-istio-data\") on node \"ip-10-0-136-157.ec2.internal\" DevicePath \"\"" Jul 23 22:39:43.184485 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:39:43.184456 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-091ba5-openshift-default-7ffff75dff-xp5hx" Jul 23 22:39:43.184961 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:39:43.184458 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-091ba5-openshift-default-7ffff75dff-xp5hx" event={"ID":"78092b11-2836-4d6e-b331-9a88f80670ba","Type":"ContainerDied","Data":"241984db1ab3dfc470e511527ae68af01519afd2714a29624abecb3b5108bf25"} Jul 23 22:39:43.184961 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:39:43.184609 2578 scope.go:117] "RemoveContainer" containerID="fd208f550533fe8d312addfb0c20932d6bb558fe5de13a08a34b778889b2a23d" Jul 23 22:39:43.206381 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:39:43.206360 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-091ba5-openshift-default-7ffff75dff-xp5hx"] Jul 23 22:39:43.209469 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:39:43.209441 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-091ba5-openshift-default-7ffff75dff-xp5hx"] Jul 23 22:39:43.584322 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:39:43.584298 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-7c697dc6f4-vsxn9_89f2a4a2-1302-470c-90ab-c09dbcd525a4/manager/0.log" Jul 23 22:39:44.251606 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:39:44.251570 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6864c64679-mgzwp_1b35faf5-1558-4b78-8af3-9b8ab44205a1/manager/0.log" Jul 23 22:39:44.251961 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:39:44.251704 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="78092b11-2836-4d6e-b331-9a88f80670ba" path="/var/lib/kubelet/pods/78092b11-2836-4d6e-b331-9a88f80670ba/volumes" Jul 23 22:39:44.258555 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:39:44.258526 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6864c64679-mgzwp_1b35faf5-1558-4b78-8af3-9b8ab44205a1/copy-manifests/0.log" Jul 23 22:39:45.770987 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:39:45.770961 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759p9kwg_21657d68-6f73-4633-ac53-6ec0f2f84890/util/0.log" Jul 23 22:39:45.779895 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:39:45.779870 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759p9kwg_21657d68-6f73-4633-ac53-6ec0f2f84890/pull/0.log" Jul 23 22:39:45.786435 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:39:45.786413 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759p9kwg_21657d68-6f73-4633-ac53-6ec0f2f84890/extract/0.log" Jul 23 22:39:45.904179 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:39:45.904156 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jcc56_1c0bc115-5e71-4a61-b2a9-8cee460568ce/util/0.log" Jul 23 22:39:45.910903 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:39:45.910882 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jcc56_1c0bc115-5e71-4a61-b2a9-8cee460568ce/pull/0.log" Jul 23 22:39:45.917555 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:39:45.917520 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jcc56_1c0bc115-5e71-4a61-b2a9-8cee460568ce/extract/0.log" Jul 23 22:39:46.038211 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:39:46.038133 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732886f_2c72f0f0-99ff-4f6e-a4bc-8e630d97f177/util/0.log" Jul 23 22:39:46.044241 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:39:46.044222 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732886f_2c72f0f0-99ff-4f6e-a4bc-8e630d97f177/pull/0.log" Jul 23 22:39:46.049669 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:39:46.049654 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732886f_2c72f0f0-99ff-4f6e-a4bc-8e630d97f177/extract/0.log" Jul 23 22:39:46.167147 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:39:46.167114 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hppvf_dfa15e28-ab86-4899-8aa8-c7ccabbc2a12/extract/0.log" Jul 23 22:39:46.173388 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:39:46.173362 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hppvf_dfa15e28-ab86-4899-8aa8-c7ccabbc2a12/util/0.log" Jul 23 22:39:46.180085 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:39:46.180063 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hppvf_dfa15e28-ab86-4899-8aa8-c7ccabbc2a12/pull/0.log" Jul 23 22:39:47.573960 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:39:47.573930 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4svqcn_91e17fed-a12b-4a8f-9c0e-6f84e3776d1f/istio-proxy/0.log" Jul 23 22:39:47.698817 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:39:47.698793 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-9pzmh_a7683500-183f-4e5c-bd8d-756d50a6f5e0/discovery/0.log" Jul 23 22:39:48.067821 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:39:48.067785 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-jnrd4_9872c26d-b195-44ba-ba89-05aae0c7ff98/istio-proxy/1.log" Jul 23 22:39:48.194533 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:39:48.194508 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-6mbp8_ec20a4df-f4f0-4817-b844-7f2134c331dc/payload-pre-processing/0.log" Jul 23 22:39:49.062824 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:39:49.062799 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-2sjvq_2173d13e-134c-41da-b2cd-1fea1e82e754/main/0.log" Jul 23 22:39:49.299256 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:39:49.299231 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-7q2mv_0ca6b8e9-c6c5-4d38-a8e8-98a73b38c977/main/0.log" Jul 23 22:39:57.999142 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:39:57.999111 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-26shl_500a30f7-ac15-47da-adf9-d0485f80af93/global-pull-secret-syncer/0.log" Jul 23 22:39:58.204716 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:39:58.204688 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-rpgtk_d76a823d-b6d8-42d4-9060-32fe24a6ab73/konnectivity-agent/0.log" Jul 23 22:39:58.255841 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:39:58.255768 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-136-157.ec2.internal_a9bc0c1f6ccfd7b14a1f3c2bc283e4db/haproxy/0.log" Jul 23 22:40:02.415300 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:40:02.415273 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759p9kwg_21657d68-6f73-4633-ac53-6ec0f2f84890/extract/0.log" Jul 23 22:40:02.440820 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:40:02.440794 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759p9kwg_21657d68-6f73-4633-ac53-6ec0f2f84890/util/0.log" Jul 23 22:40:02.463247 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:40:02.463230 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759p9kwg_21657d68-6f73-4633-ac53-6ec0f2f84890/pull/0.log" Jul 23 22:40:02.494108 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:40:02.494075 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jcc56_1c0bc115-5e71-4a61-b2a9-8cee460568ce/extract/0.log" Jul 23 22:40:02.518838 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:40:02.518816 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jcc56_1c0bc115-5e71-4a61-b2a9-8cee460568ce/util/0.log" Jul 23 22:40:02.541178 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:40:02.541161 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0jcc56_1c0bc115-5e71-4a61-b2a9-8cee460568ce/pull/0.log" Jul 23 22:40:02.568280 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:40:02.568264 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732886f_2c72f0f0-99ff-4f6e-a4bc-8e630d97f177/extract/0.log" Jul 23 22:40:02.591673 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:40:02.591654 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732886f_2c72f0f0-99ff-4f6e-a4bc-8e630d97f177/util/0.log" Jul 23 22:40:02.614489 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:40:02.614471 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732886f_2c72f0f0-99ff-4f6e-a4bc-8e630d97f177/pull/0.log" Jul 23 22:40:02.641467 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:40:02.641441 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hppvf_dfa15e28-ab86-4899-8aa8-c7ccabbc2a12/extract/0.log" Jul 23 22:40:02.663858 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:40:02.663836 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hppvf_dfa15e28-ab86-4899-8aa8-c7ccabbc2a12/util/0.log" Jul 23 22:40:02.685792 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:40:02.685735 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1hppvf_dfa15e28-ab86-4899-8aa8-c7ccabbc2a12/pull/0.log" Jul 23 22:40:05.393651 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:40:05.393613 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-5cb6c9bd46-gwkr9_a82ef530-e2e9-4234-b23f-1230edd4b83c/metrics-server/0.log" Jul 23 22:40:05.463309 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:40:05.463287 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-9szvj_98fdd345-9020-4d2f-9ce2-9a392c19218c/node-exporter/0.log" Jul 23 22:40:05.497534 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:40:05.497501 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-9szvj_98fdd345-9020-4d2f-9ce2-9a392c19218c/kube-rbac-proxy/0.log" Jul 23 22:40:05.526453 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:40:05.526435 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-9szvj_98fdd345-9020-4d2f-9ce2-9a392c19218c/init-textfile/0.log" Jul 23 22:40:06.035877 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:40:06.035842 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-5945b5cfd-j9rv8_7b56052e-4592-4797-ba17-50254d728b8a/prometheus-operator/0.log" Jul 23 22:40:06.063640 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:40:06.063605 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-5945b5cfd-j9rv8_7b56052e-4592-4797-ba17-50254d728b8a/kube-rbac-proxy/0.log" Jul 23 22:40:06.193348 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:40:06.193316 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-55ff89fbc9-t5zhk_034d2dc9-a711-493f-b5bc-6f772cf1641f/thanos-query/0.log" Jul 23 22:40:06.215317 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:40:06.215298 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-55ff89fbc9-t5zhk_034d2dc9-a711-493f-b5bc-6f772cf1641f/kube-rbac-proxy-web/0.log" Jul 23 22:40:06.236986 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:40:06.236972 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-55ff89fbc9-t5zhk_034d2dc9-a711-493f-b5bc-6f772cf1641f/kube-rbac-proxy/0.log" Jul 23 22:40:06.258355 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:40:06.258332 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-55ff89fbc9-t5zhk_034d2dc9-a711-493f-b5bc-6f772cf1641f/prom-label-proxy/0.log" Jul 23 22:40:06.284746 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:40:06.284730 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-55ff89fbc9-t5zhk_034d2dc9-a711-493f-b5bc-6f772cf1641f/kube-rbac-proxy-rules/0.log" Jul 23 22:40:06.309876 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:40:06.309853 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-55ff89fbc9-t5zhk_034d2dc9-a711-493f-b5bc-6f772cf1641f/kube-rbac-proxy-metrics/0.log" Jul 23 22:40:07.678220 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:40:07.678194 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-7bc7489cb5-qkfwc_3dd824e2-8ab1-4a94-8d99-a0eec78aa59d/networking-console-plugin/0.log" Jul 23 22:40:08.161160 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:40:08.161128 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-fvbz2/perf-node-gather-daemonset-wwbsc"] Jul 23 22:40:08.161479 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:40:08.161467 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d992d9c-ec8c-4ab3-bdb2-6034ffcf1cfe" containerName="revoke-keys" Jul 23 22:40:08.161525 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:40:08.161481 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d992d9c-ec8c-4ab3-bdb2-6034ffcf1cfe" containerName="revoke-keys" Jul 23 22:40:08.161525 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:40:08.161499 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="78092b11-2836-4d6e-b331-9a88f80670ba" containerName="istio-proxy" Jul 23 22:40:08.161525 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:40:08.161505 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="78092b11-2836-4d6e-b331-9a88f80670ba" containerName="istio-proxy" Jul 23 22:40:08.161525 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:40:08.161519 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="78092b11-2836-4d6e-b331-9a88f80670ba" containerName="istio-proxy" Jul 23 22:40:08.161525 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:40:08.161525 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="78092b11-2836-4d6e-b331-9a88f80670ba" containerName="istio-proxy" Jul 23 22:40:08.161714 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:40:08.161637 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="78092b11-2836-4d6e-b331-9a88f80670ba" containerName="istio-proxy" Jul 23 22:40:08.161714 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:40:08.161646 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="78092b11-2836-4d6e-b331-9a88f80670ba" containerName="istio-proxy" Jul 23 22:40:08.161714 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:40:08.161655 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="9d992d9c-ec8c-4ab3-bdb2-6034ffcf1cfe" containerName="revoke-keys" Jul 23 22:40:08.164895 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:40:08.164878 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-fvbz2/perf-node-gather-daemonset-wwbsc" Jul 23 22:40:08.167179 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:40:08.167157 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-fvbz2\"/\"default-dockercfg-fnxfx\"" Jul 23 22:40:08.167283 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:40:08.167160 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-fvbz2\"/\"openshift-service-ca.crt\"" Jul 23 22:40:08.168023 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:40:08.168007 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-fvbz2\"/\"kube-root-ca.crt\"" Jul 23 22:40:08.173389 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:40:08.173367 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-fvbz2/perf-node-gather-daemonset-wwbsc"] Jul 23 22:40:08.280284 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:40:08.280261 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nns9b\" (UniqueName: \"kubernetes.io/projected/a5c4e5ef-3975-4239-9419-8a090bda4cfc-kube-api-access-nns9b\") pod \"perf-node-gather-daemonset-wwbsc\" (UID: \"a5c4e5ef-3975-4239-9419-8a090bda4cfc\") " pod="openshift-must-gather-fvbz2/perf-node-gather-daemonset-wwbsc" Jul 23 22:40:08.280414 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:40:08.280323 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a5c4e5ef-3975-4239-9419-8a090bda4cfc-proc\") pod \"perf-node-gather-daemonset-wwbsc\" (UID: \"a5c4e5ef-3975-4239-9419-8a090bda4cfc\") " pod="openshift-must-gather-fvbz2/perf-node-gather-daemonset-wwbsc" Jul 23 22:40:08.280414 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:40:08.280347 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a5c4e5ef-3975-4239-9419-8a090bda4cfc-podres\") pod \"perf-node-gather-daemonset-wwbsc\" (UID: \"a5c4e5ef-3975-4239-9419-8a090bda4cfc\") " pod="openshift-must-gather-fvbz2/perf-node-gather-daemonset-wwbsc" Jul 23 22:40:08.280414 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:40:08.280371 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a5c4e5ef-3975-4239-9419-8a090bda4cfc-sys\") pod \"perf-node-gather-daemonset-wwbsc\" (UID: \"a5c4e5ef-3975-4239-9419-8a090bda4cfc\") " pod="openshift-must-gather-fvbz2/perf-node-gather-daemonset-wwbsc" Jul 23 22:40:08.280414 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:40:08.280394 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a5c4e5ef-3975-4239-9419-8a090bda4cfc-lib-modules\") pod \"perf-node-gather-daemonset-wwbsc\" (UID: \"a5c4e5ef-3975-4239-9419-8a090bda4cfc\") " pod="openshift-must-gather-fvbz2/perf-node-gather-daemonset-wwbsc" Jul 23 22:40:08.381504 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:40:08.381481 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a5c4e5ef-3975-4239-9419-8a090bda4cfc-proc\") pod \"perf-node-gather-daemonset-wwbsc\" (UID: \"a5c4e5ef-3975-4239-9419-8a090bda4cfc\") " pod="openshift-must-gather-fvbz2/perf-node-gather-daemonset-wwbsc" Jul 23 22:40:08.381609 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:40:08.381509 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a5c4e5ef-3975-4239-9419-8a090bda4cfc-podres\") pod \"perf-node-gather-daemonset-wwbsc\" (UID: \"a5c4e5ef-3975-4239-9419-8a090bda4cfc\") " pod="openshift-must-gather-fvbz2/perf-node-gather-daemonset-wwbsc" Jul 23 22:40:08.381609 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:40:08.381564 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a5c4e5ef-3975-4239-9419-8a090bda4cfc-sys\") pod \"perf-node-gather-daemonset-wwbsc\" (UID: \"a5c4e5ef-3975-4239-9419-8a090bda4cfc\") " pod="openshift-must-gather-fvbz2/perf-node-gather-daemonset-wwbsc" Jul 23 22:40:08.381609 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:40:08.381602 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a5c4e5ef-3975-4239-9419-8a090bda4cfc-lib-modules\") pod \"perf-node-gather-daemonset-wwbsc\" (UID: \"a5c4e5ef-3975-4239-9419-8a090bda4cfc\") " pod="openshift-must-gather-fvbz2/perf-node-gather-daemonset-wwbsc" Jul 23 22:40:08.381720 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:40:08.381605 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a5c4e5ef-3975-4239-9419-8a090bda4cfc-proc\") pod \"perf-node-gather-daemonset-wwbsc\" (UID: \"a5c4e5ef-3975-4239-9419-8a090bda4cfc\") " pod="openshift-must-gather-fvbz2/perf-node-gather-daemonset-wwbsc" Jul 23 22:40:08.381720 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:40:08.381657 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a5c4e5ef-3975-4239-9419-8a090bda4cfc-sys\") pod \"perf-node-gather-daemonset-wwbsc\" (UID: \"a5c4e5ef-3975-4239-9419-8a090bda4cfc\") " pod="openshift-must-gather-fvbz2/perf-node-gather-daemonset-wwbsc" Jul 23 22:40:08.381720 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:40:08.381692 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a5c4e5ef-3975-4239-9419-8a090bda4cfc-podres\") pod \"perf-node-gather-daemonset-wwbsc\" (UID: \"a5c4e5ef-3975-4239-9419-8a090bda4cfc\") " pod="openshift-must-gather-fvbz2/perf-node-gather-daemonset-wwbsc" Jul 23 22:40:08.381720 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:40:08.381707 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a5c4e5ef-3975-4239-9419-8a090bda4cfc-lib-modules\") pod \"perf-node-gather-daemonset-wwbsc\" (UID: \"a5c4e5ef-3975-4239-9419-8a090bda4cfc\") " pod="openshift-must-gather-fvbz2/perf-node-gather-daemonset-wwbsc" Jul 23 22:40:08.381863 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:40:08.381709 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nns9b\" (UniqueName: \"kubernetes.io/projected/a5c4e5ef-3975-4239-9419-8a090bda4cfc-kube-api-access-nns9b\") pod \"perf-node-gather-daemonset-wwbsc\" (UID: \"a5c4e5ef-3975-4239-9419-8a090bda4cfc\") " pod="openshift-must-gather-fvbz2/perf-node-gather-daemonset-wwbsc" Jul 23 22:40:08.390364 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:40:08.390342 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nns9b\" (UniqueName: \"kubernetes.io/projected/a5c4e5ef-3975-4239-9419-8a090bda4cfc-kube-api-access-nns9b\") pod \"perf-node-gather-daemonset-wwbsc\" (UID: \"a5c4e5ef-3975-4239-9419-8a090bda4cfc\") " pod="openshift-must-gather-fvbz2/perf-node-gather-daemonset-wwbsc" Jul 23 22:40:08.475968 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:40:08.475914 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-fvbz2/perf-node-gather-daemonset-wwbsc" Jul 23 22:40:08.601577 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:40:08.601553 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-fvbz2/perf-node-gather-daemonset-wwbsc"] Jul 23 22:40:08.603200 ip-10-0-136-157 kubenswrapper[2578]: W0723 22:40:08.603172 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-poda5c4e5ef_3975_4239_9419_8a090bda4cfc.slice/crio-7d01bfd7e7732178eb8b9259a477b2a9674c88841355576f6cc416df262ae202 WatchSource:0}: Error finding container 7d01bfd7e7732178eb8b9259a477b2a9674c88841355576f6cc416df262ae202: Status 404 returned error can't find the container with id 7d01bfd7e7732178eb8b9259a477b2a9674c88841355576f6cc416df262ae202 Jul 23 22:40:08.981279 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:40:08.981256 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-5bc67dcd46-zxc85_ca691a1f-6865-40b2-96fe-9bb8782b7e18/download-server/0.log" Jul 23 22:40:09.310552 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:40:09.310499 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-fvbz2/perf-node-gather-daemonset-wwbsc" event={"ID":"a5c4e5ef-3975-4239-9419-8a090bda4cfc","Type":"ContainerStarted","Data":"5ae56f778cc06246b6d752492ca90a83eeeb407943cbacf896fd9d779b43dfa3"} Jul 23 22:40:09.310744 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:40:09.310591 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-fvbz2/perf-node-gather-daemonset-wwbsc" Jul 23 22:40:09.310744 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:40:09.310615 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-fvbz2/perf-node-gather-daemonset-wwbsc" event={"ID":"a5c4e5ef-3975-4239-9419-8a090bda4cfc","Type":"ContainerStarted","Data":"7d01bfd7e7732178eb8b9259a477b2a9674c88841355576f6cc416df262ae202"} Jul 23 22:40:09.326885 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:40:09.326839 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-fvbz2/perf-node-gather-daemonset-wwbsc" podStartSLOduration=1.3268245539999999 podStartE2EDuration="1.326824554s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 22:40:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 22:40:09.325948012 +0000 UTC m=+2911.567855883" watchObservedRunningTime="2026-07-23 22:40:09.326824554 +0000 UTC m=+2911.568732427" Jul 23 22:40:10.702757 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:40:10.702729 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-6mgzw_c8c008d7-7d6c-42e2-b9c6-4c9e9557c7c4/dns/0.log" Jul 23 22:40:10.723848 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:40:10.723827 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-6mgzw_c8c008d7-7d6c-42e2-b9c6-4c9e9557c7c4/kube-rbac-proxy/0.log" Jul 23 22:40:10.885670 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:40:10.885657 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-w6jzd_4e42ce8b-422d-4281-b12e-6371c9744f8a/dns-node-resolver/0.log" Jul 23 22:40:11.467710 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:40:11.467663 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-5899879c95-5w8d7_0936cb55-c4b0-484c-8ff0-6f457ce7e5b1/registry/0.log" Jul 23 22:40:11.514827 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:40:11.514799 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-nt77r_49757977-27cf-4177-b295-c830e4a71b03/node-ca/0.log" Jul 23 22:40:12.494319 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:40:12.494293 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4svqcn_91e17fed-a12b-4a8f-9c0e-6f84e3776d1f/istio-proxy/0.log" Jul 23 22:40:12.610885 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:40:12.610853 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-9pzmh_a7683500-183f-4e5c-bd8d-756d50a6f5e0/discovery/0.log" Jul 23 22:40:12.697203 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:40:12.697175 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-jnrd4_9872c26d-b195-44ba-ba89-05aae0c7ff98/istio-proxy/0.log" Jul 23 22:40:12.723322 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:40:12.723297 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-jnrd4_9872c26d-b195-44ba-ba89-05aae0c7ff98/istio-proxy/1.log" Jul 23 22:40:12.762175 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:40:12.762114 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-6mbp8_ec20a4df-f4f0-4817-b844-7f2134c331dc/payload-pre-processing/0.log" Jul 23 22:40:13.508695 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:40:13.508670 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-5gh5l_634a7674-21bb-4fd9-8ce8-1d6357c4500d/serve-healthcheck-canary/0.log" Jul 23 22:40:14.225671 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:40:14.225641 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-fkqwr_f1e7deb0-a030-4479-b82a-5dbbc864104b/kube-rbac-proxy/0.log" Jul 23 22:40:14.245573 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:40:14.245549 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-fkqwr_f1e7deb0-a030-4479-b82a-5dbbc864104b/exporter/0.log" Jul 23 22:40:14.265955 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:40:14.265937 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-fkqwr_f1e7deb0-a030-4479-b82a-5dbbc864104b/extractor/0.log" Jul 23 22:40:15.326372 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:40:15.326348 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-fvbz2/perf-node-gather-daemonset-wwbsc" Jul 23 22:40:16.885124 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:40:16.885097 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-7c697dc6f4-vsxn9_89f2a4a2-1302-470c-90ab-c09dbcd525a4/manager/0.log" Jul 23 22:40:17.182963 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:40:17.182937 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6864c64679-mgzwp_1b35faf5-1558-4b78-8af3-9b8ab44205a1/manager/0.log" Jul 23 22:40:17.204102 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:40:17.204083 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6864c64679-mgzwp_1b35faf5-1558-4b78-8af3-9b8ab44205a1/copy-manifests/0.log" Jul 23 22:40:18.607533 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:40:18.607445 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-7575c4d6b8-2z7qz_56562de7-fa5d-4b31-974d-1bb76ab0927f/manager/0.log" Jul 23 22:40:19.056589 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:40:19.056561 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4svqcn_91e17fed-a12b-4a8f-9c0e-6f84e3776d1f/istio-proxy/0.log" Jul 23 22:40:19.168394 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:40:19.168362 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-9pzmh_a7683500-183f-4e5c-bd8d-756d50a6f5e0/discovery/0.log" Jul 23 22:40:19.218849 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:40:19.218826 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-jnrd4_9872c26d-b195-44ba-ba89-05aae0c7ff98/istio-proxy/0.log" Jul 23 22:40:19.244308 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:40:19.244276 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-jnrd4_9872c26d-b195-44ba-ba89-05aae0c7ff98/istio-proxy/1.log" Jul 23 22:40:19.263881 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:40:19.263855 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-6mbp8_ec20a4df-f4f0-4817-b844-7f2134c331dc/payload-pre-processing/0.log" Jul 23 22:40:20.505741 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:40:20.505700 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4svqcn_91e17fed-a12b-4a8f-9c0e-6f84e3776d1f/istio-proxy/0.log" Jul 23 22:40:20.631850 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:40:20.631818 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-9pzmh_a7683500-183f-4e5c-bd8d-756d50a6f5e0/discovery/0.log" Jul 23 22:40:20.689284 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:40:20.689261 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-jnrd4_9872c26d-b195-44ba-ba89-05aae0c7ff98/istio-proxy/0.log" Jul 23 22:40:20.712433 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:40:20.712404 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-jnrd4_9872c26d-b195-44ba-ba89-05aae0c7ff98/istio-proxy/1.log" Jul 23 22:40:20.733013 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:40:20.732986 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-6mbp8_ec20a4df-f4f0-4817-b844-7f2134c331dc/payload-pre-processing/0.log" Jul 23 22:40:23.854343 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:40:23.854316 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7cc9c7ffd7-gz5x4_8fbea949-92d3-469d-b6e1-7bf6c90aa4e8/migrator/0.log" Jul 23 22:40:23.876523 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:40:23.876502 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7cc9c7ffd7-gz5x4_8fbea949-92d3-469d-b6e1-7bf6c90aa4e8/graceful-termination/0.log" Jul 23 22:40:25.392280 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:40:25.392251 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ntjs5_1e27638c-41d9-441b-8cea-dd134bf51985/kube-multus-additional-cni-plugins/0.log" Jul 23 22:40:25.412501 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:40:25.412478 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ntjs5_1e27638c-41d9-441b-8cea-dd134bf51985/egress-router-binary-copy/0.log" Jul 23 22:40:25.433398 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:40:25.433376 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ntjs5_1e27638c-41d9-441b-8cea-dd134bf51985/cni-plugins/0.log" Jul 23 22:40:25.454898 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:40:25.454877 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ntjs5_1e27638c-41d9-441b-8cea-dd134bf51985/bond-cni-plugin/0.log" Jul 23 22:40:25.477274 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:40:25.477257 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ntjs5_1e27638c-41d9-441b-8cea-dd134bf51985/routeoverride-cni/0.log" Jul 23 22:40:25.499550 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:40:25.499515 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ntjs5_1e27638c-41d9-441b-8cea-dd134bf51985/whereabouts-cni-bincopy/0.log" Jul 23 22:40:25.522308 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:40:25.522292 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-ntjs5_1e27638c-41d9-441b-8cea-dd134bf51985/whereabouts-cni/0.log" Jul 23 22:40:25.713265 ip-10-0-136-157 kubenswrapper[2578]: I0723 22:40:25.713201 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-d5lp9_1b8773cc-8521-4192-a3ab-0a397ba6eb62/kube-multus/0.log"