Jul 14 17:32:29.775072 ip-10-0-129-209 systemd[1]: Starting Kubernetes Kubelet... Jul 14 17:32:30.213329 ip-10-0-129-209 kubenswrapper[2562]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 14 17:32:30.213329 ip-10-0-129-209 kubenswrapper[2562]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 14 17:32:30.213329 ip-10-0-129-209 kubenswrapper[2562]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 14 17:32:30.213329 ip-10-0-129-209 kubenswrapper[2562]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 14 17:32:30.213329 ip-10-0-129-209 kubenswrapper[2562]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 14 17:32:30.215309 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.215227 2562 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 14 17:32:30.219895 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.219880 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 14 17:32:30.219895 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.219895 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 14 17:32:30.219955 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.219899 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 14 17:32:30.219955 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.219903 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 14 17:32:30.219955 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.219906 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 14 17:32:30.219955 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.219909 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 14 17:32:30.219955 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.219912 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 14 17:32:30.219955 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.219915 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 14 17:32:30.219955 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.219919 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 14 17:32:30.219955 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.219923 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 14 17:32:30.219955 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.219929 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 14 17:32:30.219955 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.219933 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 14 17:32:30.219955 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.219935 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 14 17:32:30.219955 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.219938 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 14 17:32:30.219955 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.219941 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 14 17:32:30.219955 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.219943 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 14 17:32:30.219955 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.219946 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 14 17:32:30.219955 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.219948 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 14 17:32:30.219955 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.219951 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 14 17:32:30.219955 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.219953 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 14 17:32:30.219955 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.219956 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 14 17:32:30.220414 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.219958 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 14 17:32:30.220414 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.219961 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 14 17:32:30.220414 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.219964 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 14 17:32:30.220414 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.219967 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 14 17:32:30.220414 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.219970 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 14 17:32:30.220414 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.219972 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 14 17:32:30.220414 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.219975 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 14 17:32:30.220414 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.219977 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 14 17:32:30.220414 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.219980 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 14 17:32:30.220414 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.219982 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 14 17:32:30.220414 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.219984 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 14 17:32:30.220414 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.219987 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 14 17:32:30.220414 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.219989 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 14 17:32:30.220414 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.219992 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 14 17:32:30.220414 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.219995 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 14 17:32:30.220414 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.219999 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 14 17:32:30.220414 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.220002 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 14 17:32:30.220414 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.220005 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 14 17:32:30.220414 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.220007 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 14 17:32:30.220414 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.220010 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 14 17:32:30.220944 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.220012 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 14 17:32:30.220944 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.220015 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 14 17:32:30.220944 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.220017 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 14 17:32:30.220944 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.220020 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 14 17:32:30.220944 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.220022 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 14 17:32:30.220944 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.220025 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 14 17:32:30.220944 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.220027 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 14 17:32:30.220944 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.220029 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 14 17:32:30.220944 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.220032 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 14 17:32:30.220944 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.220034 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 14 17:32:30.220944 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.220036 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 14 17:32:30.220944 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.220039 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 14 17:32:30.220944 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.220041 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 14 17:32:30.220944 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.220044 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 14 17:32:30.220944 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.220047 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 14 17:32:30.220944 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.220049 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 14 17:32:30.220944 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.220052 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 14 17:32:30.220944 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.220054 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 14 17:32:30.220944 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.220056 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 14 17:32:30.220944 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.220059 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 14 17:32:30.221428 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.220061 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 14 17:32:30.221428 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.220064 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 14 17:32:30.221428 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.220067 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 14 17:32:30.221428 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.220069 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 14 17:32:30.221428 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.220072 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 14 17:32:30.221428 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.220074 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 14 17:32:30.221428 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.220077 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 14 17:32:30.221428 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.220080 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 14 17:32:30.221428 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.220082 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 14 17:32:30.221428 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.220084 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 14 17:32:30.221428 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.220087 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 14 17:32:30.221428 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.220089 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 14 17:32:30.221428 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.220091 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 14 17:32:30.221428 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.220094 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 14 17:32:30.221428 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.220096 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 14 17:32:30.221428 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.220098 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 14 17:32:30.221428 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.220102 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 14 17:32:30.221428 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.220104 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 14 17:32:30.221428 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.220107 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 14 17:32:30.222009 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.220109 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 14 17:32:30.222009 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.220112 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 14 17:32:30.222009 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.220114 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 14 17:32:30.222009 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.220117 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 14 17:32:30.222009 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.220119 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 14 17:32:30.222009 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.220122 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 14 17:32:30.222009 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.220126 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 14 17:32:30.222009 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.220128 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 14 17:32:30.222009 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.220131 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 14 17:32:30.222009 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.220134 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 14 17:32:30.222009 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.220136 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 14 17:32:30.222009 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.220139 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 14 17:32:30.222009 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.220141 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 14 17:32:30.222009 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.220143 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 14 17:32:30.222009 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.220146 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 14 17:32:30.222009 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.220148 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 14 17:32:30.222009 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.220150 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 14 17:32:30.222009 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.220153 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 14 17:32:30.222009 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.220155 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 14 17:32:30.222009 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.220836 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 14 17:32:30.222009 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.220843 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 14 17:32:30.222729 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.220847 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 14 17:32:30.222729 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.220850 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 14 17:32:30.222729 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.220853 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 14 17:32:30.222729 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.220855 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 14 17:32:30.222729 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.220858 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 14 17:32:30.222729 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.220860 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 14 17:32:30.222729 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.220863 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 14 17:32:30.222729 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.220865 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 14 17:32:30.222729 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.220868 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 14 17:32:30.222729 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.220871 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 14 17:32:30.222729 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.220873 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 14 17:32:30.222729 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.220875 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 14 17:32:30.222729 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.220878 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 14 17:32:30.222729 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.220880 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 14 17:32:30.222729 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.220883 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 14 17:32:30.222729 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.220885 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 14 17:32:30.222729 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.220888 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 14 17:32:30.222729 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.220892 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 14 17:32:30.222729 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.220895 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 14 17:32:30.223229 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.220898 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 14 17:32:30.223229 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.220900 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 14 17:32:30.223229 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.220903 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 14 17:32:30.223229 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.220905 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 14 17:32:30.223229 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.220908 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 14 17:32:30.223229 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.220911 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 14 17:32:30.223229 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.220913 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 14 17:32:30.223229 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.220916 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 14 17:32:30.223229 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.220918 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 14 17:32:30.223229 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.220921 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 14 17:32:30.223229 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.220923 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 14 17:32:30.223229 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.220926 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 14 17:32:30.223229 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.220928 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 14 17:32:30.223229 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.220930 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 14 17:32:30.223229 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.220934 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 14 17:32:30.223229 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.220936 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 14 17:32:30.223229 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.220940 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 14 17:32:30.223229 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.220944 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 14 17:32:30.223229 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.220947 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 14 17:32:30.223719 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.220950 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 14 17:32:30.223719 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.220953 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 14 17:32:30.223719 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.220956 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 14 17:32:30.223719 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.220959 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 14 17:32:30.223719 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.220961 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 14 17:32:30.223719 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.220964 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 14 17:32:30.223719 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.220966 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 14 17:32:30.223719 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.220968 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 14 17:32:30.223719 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.220971 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 14 17:32:30.223719 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.220973 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 14 17:32:30.223719 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.220976 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 14 17:32:30.223719 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.220979 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 14 17:32:30.223719 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.220981 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 14 17:32:30.223719 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.220984 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 14 17:32:30.223719 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.220986 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 14 17:32:30.223719 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.220989 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 14 17:32:30.223719 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.220991 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 14 17:32:30.223719 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.220994 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 14 17:32:30.223719 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.220997 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 14 17:32:30.223719 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.220999 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 14 17:32:30.224227 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221002 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 14 17:32:30.224227 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221004 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 14 17:32:30.224227 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221007 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 14 17:32:30.224227 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221009 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 14 17:32:30.224227 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221012 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 14 17:32:30.224227 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221014 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 14 17:32:30.224227 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221016 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 14 17:32:30.224227 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221019 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 14 17:32:30.224227 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221022 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 14 17:32:30.224227 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221024 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 14 17:32:30.224227 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221027 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 14 17:32:30.224227 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221029 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 14 17:32:30.224227 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221031 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 14 17:32:30.224227 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221034 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 14 17:32:30.224227 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221036 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 14 17:32:30.224227 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221040 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 14 17:32:30.224227 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221043 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 14 17:32:30.224227 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221046 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 14 17:32:30.224227 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221048 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 14 17:32:30.224227 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221050 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 14 17:32:30.224715 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221052 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 14 17:32:30.224715 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221055 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 14 17:32:30.224715 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221057 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 14 17:32:30.224715 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221060 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 14 17:32:30.224715 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221063 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 14 17:32:30.224715 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221065 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 14 17:32:30.224715 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221068 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 14 17:32:30.224715 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221070 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 14 17:32:30.224715 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221073 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 14 17:32:30.224715 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221075 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 14 17:32:30.224715 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221078 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 14 17:32:30.224715 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221080 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 14 17:32:30.224715 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221082 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 14 17:32:30.224715 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221085 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 14 17:32:30.224715 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221087 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 14 17:32:30.224715 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221089 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 14 17:32:30.224715 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221093 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 14 17:32:30.224715 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221096 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 14 17:32:30.224715 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221099 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 14 17:32:30.224715 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221432 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 14 17:32:30.225209 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221437 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 14 17:32:30.225209 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221440 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 14 17:32:30.225209 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221444 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 14 17:32:30.225209 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221447 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 14 17:32:30.225209 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221450 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 14 17:32:30.225209 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221452 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 14 17:32:30.225209 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221455 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 14 17:32:30.225209 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221458 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 14 17:32:30.225209 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221461 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 14 17:32:30.225209 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221464 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 14 17:32:30.225209 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221467 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 14 17:32:30.225209 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221469 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 14 17:32:30.225209 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221472 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 14 17:32:30.225209 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221475 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 14 17:32:30.225209 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221477 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 14 17:32:30.225209 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221482 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 14 17:32:30.225209 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221485 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 14 17:32:30.225209 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221487 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 14 17:32:30.225209 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221490 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 14 17:32:30.225209 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221493 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 14 17:32:30.225691 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221495 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 14 17:32:30.225691 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221498 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 14 17:32:30.225691 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221502 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 14 17:32:30.225691 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221505 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 14 17:32:30.225691 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221508 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 14 17:32:30.225691 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221511 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 14 17:32:30.225691 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221513 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 14 17:32:30.225691 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221516 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 14 17:32:30.225691 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221518 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 14 17:32:30.225691 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221520 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 14 17:32:30.225691 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221524 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 14 17:32:30.225691 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221526 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 14 17:32:30.225691 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221529 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 14 17:32:30.225691 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221532 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 14 17:32:30.225691 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221534 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 14 17:32:30.225691 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221536 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 14 17:32:30.225691 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221539 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 14 17:32:30.225691 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221541 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 14 17:32:30.225691 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221544 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 14 17:32:30.225691 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221546 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 14 17:32:30.226182 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221548 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 14 17:32:30.226182 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221551 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 14 17:32:30.226182 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221553 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 14 17:32:30.226182 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221555 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 14 17:32:30.226182 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221558 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 14 17:32:30.226182 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221560 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 14 17:32:30.226182 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221563 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 14 17:32:30.226182 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221567 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 14 17:32:30.226182 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221570 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 14 17:32:30.226182 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221573 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 14 17:32:30.226182 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221575 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 14 17:32:30.226182 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221577 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 14 17:32:30.226182 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221580 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 14 17:32:30.226182 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221582 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 14 17:32:30.226182 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221585 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 14 17:32:30.226182 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221587 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 14 17:32:30.226182 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221590 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 14 17:32:30.226182 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221592 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 14 17:32:30.226182 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221594 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 14 17:32:30.226182 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221597 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 14 17:32:30.226677 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221599 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 14 17:32:30.226677 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221602 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 14 17:32:30.226677 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221604 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 14 17:32:30.226677 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221606 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 14 17:32:30.226677 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221609 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 14 17:32:30.226677 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221611 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 14 17:32:30.226677 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221614 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 14 17:32:30.226677 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221616 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 14 17:32:30.226677 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221618 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 14 17:32:30.226677 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221621 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 14 17:32:30.226677 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221623 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 14 17:32:30.226677 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221627 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 14 17:32:30.226677 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221630 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 14 17:32:30.226677 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221632 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 14 17:32:30.226677 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221634 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 14 17:32:30.226677 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221637 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 14 17:32:30.226677 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221639 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 14 17:32:30.226677 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221641 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 14 17:32:30.226677 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221644 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 14 17:32:30.226677 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221648 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 14 17:32:30.227174 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221651 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 14 17:32:30.227174 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221653 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 14 17:32:30.227174 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221656 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 14 17:32:30.227174 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221658 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 14 17:32:30.227174 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221660 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 14 17:32:30.227174 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221663 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 14 17:32:30.227174 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221665 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 14 17:32:30.227174 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221668 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 14 17:32:30.227174 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221670 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 14 17:32:30.227174 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221739 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 14 17:32:30.227174 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221745 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 14 17:32:30.227174 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221750 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 14 17:32:30.227174 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221754 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 14 17:32:30.227174 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.221757 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 14 17:32:30.227174 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.223190 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 14 17:32:30.227174 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.223206 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 14 17:32:30.227174 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.223212 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 14 17:32:30.227174 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.223217 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 14 17:32:30.227174 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223426 2562 flags.go:64] FLAG: --address="0.0.0.0" Jul 14 17:32:30.227641 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223440 2562 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 14 17:32:30.227641 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223447 2562 flags.go:64] FLAG: --anonymous-auth="true" Jul 14 17:32:30.227641 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223451 2562 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 14 17:32:30.227641 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223455 2562 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 14 17:32:30.227641 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223458 2562 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 14 17:32:30.227641 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223462 2562 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 14 17:32:30.227641 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223466 2562 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 14 17:32:30.227641 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223469 2562 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 14 17:32:30.227641 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223472 2562 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 14 17:32:30.227641 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223476 2562 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 14 17:32:30.227641 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223479 2562 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 14 17:32:30.227641 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223482 2562 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 14 17:32:30.227641 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223485 2562 flags.go:64] FLAG: --cgroup-root="" Jul 14 17:32:30.227641 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223487 2562 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 14 17:32:30.227641 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223490 2562 flags.go:64] FLAG: --client-ca-file="" Jul 14 17:32:30.227641 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223493 2562 flags.go:64] FLAG: --cloud-provider="external" Jul 14 17:32:30.227641 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223496 2562 flags.go:64] FLAG: --cluster-dns="[]" Jul 14 17:32:30.227641 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223501 2562 flags.go:64] FLAG: --cluster-domain="" Jul 14 17:32:30.227641 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223504 2562 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 14 17:32:30.227641 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223507 2562 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 14 17:32:30.227641 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223510 2562 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 14 17:32:30.227641 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223513 2562 flags.go:64] FLAG: --container-log-max-files="5" Jul 14 17:32:30.227641 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223516 2562 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 14 17:32:30.227641 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223519 2562 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 14 17:32:30.228238 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223522 2562 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 14 17:32:30.228238 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223525 2562 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 14 17:32:30.228238 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223528 2562 flags.go:64] FLAG: --contention-profiling="false" Jul 14 17:32:30.228238 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223530 2562 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 14 17:32:30.228238 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223534 2562 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 14 17:32:30.228238 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223537 2562 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 14 17:32:30.228238 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223540 2562 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 14 17:32:30.228238 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223544 2562 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 14 17:32:30.228238 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223547 2562 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 14 17:32:30.228238 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223550 2562 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 14 17:32:30.228238 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223552 2562 flags.go:64] FLAG: --enable-load-reader="false" Jul 14 17:32:30.228238 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223555 2562 flags.go:64] FLAG: --enable-server="true" Jul 14 17:32:30.228238 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223559 2562 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 14 17:32:30.228238 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223564 2562 flags.go:64] FLAG: --event-burst="100" Jul 14 17:32:30.228238 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223567 2562 flags.go:64] FLAG: --event-qps="50" Jul 14 17:32:30.228238 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223569 2562 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 14 17:32:30.228238 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223573 2562 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 14 17:32:30.228238 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223576 2562 flags.go:64] FLAG: --eviction-hard="" Jul 14 17:32:30.228238 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223579 2562 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 14 17:32:30.228238 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223582 2562 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 14 17:32:30.228238 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223585 2562 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 14 17:32:30.228238 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223587 2562 flags.go:64] FLAG: --eviction-soft="" Jul 14 17:32:30.228238 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223590 2562 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 14 17:32:30.228238 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223593 2562 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 14 17:32:30.228238 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223596 2562 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 14 17:32:30.228894 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223599 2562 flags.go:64] FLAG: --experimental-mounter-path="" Jul 14 17:32:30.228894 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223602 2562 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 14 17:32:30.228894 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223604 2562 flags.go:64] FLAG: --fail-swap-on="true" Jul 14 17:32:30.228894 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223607 2562 flags.go:64] FLAG: --feature-gates="" Jul 14 17:32:30.228894 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223616 2562 flags.go:64] FLAG: --file-check-frequency="20s" Jul 14 17:32:30.228894 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223619 2562 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 14 17:32:30.228894 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223622 2562 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 14 17:32:30.228894 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223625 2562 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 14 17:32:30.228894 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223628 2562 flags.go:64] FLAG: --healthz-port="10248" Jul 14 17:32:30.228894 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223631 2562 flags.go:64] FLAG: --help="false" Jul 14 17:32:30.228894 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223634 2562 flags.go:64] FLAG: --hostname-override="ip-10-0-129-209.ec2.internal" Jul 14 17:32:30.228894 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223637 2562 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 14 17:32:30.228894 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223640 2562 flags.go:64] FLAG: --http-check-frequency="20s" Jul 14 17:32:30.228894 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223642 2562 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 14 17:32:30.228894 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223646 2562 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 14 17:32:30.228894 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223650 2562 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 14 17:32:30.228894 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223652 2562 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 14 17:32:30.228894 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223655 2562 flags.go:64] FLAG: --image-service-endpoint="" Jul 14 17:32:30.228894 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223658 2562 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 14 17:32:30.228894 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223660 2562 flags.go:64] FLAG: --kube-api-burst="100" Jul 14 17:32:30.228894 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223663 2562 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 14 17:32:30.228894 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223666 2562 flags.go:64] FLAG: --kube-api-qps="50" Jul 14 17:32:30.228894 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223669 2562 flags.go:64] FLAG: --kube-reserved="" Jul 14 17:32:30.228894 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223672 2562 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 14 17:32:30.229483 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223675 2562 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 14 17:32:30.229483 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223678 2562 flags.go:64] FLAG: --kubelet-cgroups="" Jul 14 17:32:30.229483 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223680 2562 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 14 17:32:30.229483 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223683 2562 flags.go:64] FLAG: --lock-file="" Jul 14 17:32:30.229483 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223685 2562 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 14 17:32:30.229483 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223688 2562 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 14 17:32:30.229483 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223691 2562 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 14 17:32:30.229483 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223696 2562 flags.go:64] FLAG: --log-json-split-stream="false" Jul 14 17:32:30.229483 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223699 2562 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 14 17:32:30.229483 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223702 2562 flags.go:64] FLAG: --log-text-split-stream="false" Jul 14 17:32:30.229483 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223705 2562 flags.go:64] FLAG: --logging-format="text" Jul 14 17:32:30.229483 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223708 2562 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 14 17:32:30.229483 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223711 2562 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 14 17:32:30.229483 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223714 2562 flags.go:64] FLAG: --manifest-url="" Jul 14 17:32:30.229483 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223716 2562 flags.go:64] FLAG: --manifest-url-header="" Jul 14 17:32:30.229483 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223721 2562 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 14 17:32:30.229483 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223723 2562 flags.go:64] FLAG: --max-open-files="1000000" Jul 14 17:32:30.229483 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223727 2562 flags.go:64] FLAG: --max-pods="110" Jul 14 17:32:30.229483 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223730 2562 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 14 17:32:30.229483 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223734 2562 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 14 17:32:30.229483 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223737 2562 flags.go:64] FLAG: --memory-manager-policy="None" Jul 14 17:32:30.229483 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223740 2562 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 14 17:32:30.229483 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223742 2562 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 14 17:32:30.229483 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223745 2562 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 14 17:32:30.229483 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223748 2562 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 14 17:32:30.230121 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223755 2562 flags.go:64] FLAG: --node-status-max-images="50" Jul 14 17:32:30.230121 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223757 2562 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 14 17:32:30.230121 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223760 2562 flags.go:64] FLAG: --oom-score-adj="-999" Jul 14 17:32:30.230121 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223763 2562 flags.go:64] FLAG: --pod-cidr="" Jul 14 17:32:30.230121 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223766 2562 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926" Jul 14 17:32:30.230121 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223771 2562 flags.go:64] FLAG: --pod-manifest-path="" Jul 14 17:32:30.230121 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223773 2562 flags.go:64] FLAG: --pod-max-pids="-1" Jul 14 17:32:30.230121 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223777 2562 flags.go:64] FLAG: --pods-per-core="0" Jul 14 17:32:30.230121 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223779 2562 flags.go:64] FLAG: --port="10250" Jul 14 17:32:30.230121 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223782 2562 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 14 17:32:30.230121 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223785 2562 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-08b27cd09a207e666" Jul 14 17:32:30.230121 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223788 2562 flags.go:64] FLAG: --qos-reserved="" Jul 14 17:32:30.230121 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223791 2562 flags.go:64] FLAG: --read-only-port="10255" Jul 14 17:32:30.230121 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223794 2562 flags.go:64] FLAG: --register-node="true" Jul 14 17:32:30.230121 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223809 2562 flags.go:64] FLAG: --register-with-taints="" Jul 14 17:32:30.230121 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223813 2562 flags.go:64] FLAG: --registry-burst="10" Jul 14 17:32:30.230121 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223816 2562 flags.go:64] FLAG: --registry-qps="5" Jul 14 17:32:30.230121 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223819 2562 flags.go:64] FLAG: --reserved-cpus="" Jul 14 17:32:30.230121 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223822 2562 flags.go:64] FLAG: --reserved-memory="" Jul 14 17:32:30.230121 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223825 2562 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 14 17:32:30.230121 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223828 2562 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 14 17:32:30.230121 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223831 2562 flags.go:64] FLAG: --rotate-certificates="false" Jul 14 17:32:30.230121 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223833 2562 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 14 17:32:30.230121 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223836 2562 flags.go:64] FLAG: --runonce="false" Jul 14 17:32:30.230121 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223838 2562 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 14 17:32:30.230782 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223841 2562 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 14 17:32:30.230782 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223844 2562 flags.go:64] FLAG: --seccomp-default="false" Jul 14 17:32:30.230782 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223847 2562 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 14 17:32:30.230782 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223849 2562 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 14 17:32:30.230782 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223852 2562 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 14 17:32:30.230782 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223855 2562 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 14 17:32:30.230782 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223858 2562 flags.go:64] FLAG: --storage-driver-password="root" Jul 14 17:32:30.230782 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223860 2562 flags.go:64] FLAG: --storage-driver-secure="false" Jul 14 17:32:30.230782 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223863 2562 flags.go:64] FLAG: --storage-driver-table="stats" Jul 14 17:32:30.230782 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223865 2562 flags.go:64] FLAG: --storage-driver-user="root" Jul 14 17:32:30.230782 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223868 2562 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 14 17:32:30.230782 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223871 2562 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 14 17:32:30.230782 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223874 2562 flags.go:64] FLAG: --system-cgroups="" Jul 14 17:32:30.230782 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223878 2562 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 14 17:32:30.230782 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223884 2562 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 14 17:32:30.230782 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223887 2562 flags.go:64] FLAG: --tls-cert-file="" Jul 14 17:32:30.230782 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223891 2562 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 14 17:32:30.230782 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223895 2562 flags.go:64] FLAG: --tls-min-version="" Jul 14 17:32:30.230782 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223898 2562 flags.go:64] FLAG: --tls-private-key-file="" Jul 14 17:32:30.230782 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223901 2562 flags.go:64] FLAG: --topology-manager-policy="none" Jul 14 17:32:30.230782 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223903 2562 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 14 17:32:30.230782 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223906 2562 flags.go:64] FLAG: --topology-manager-scope="container" Jul 14 17:32:30.230782 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223911 2562 flags.go:64] FLAG: --v="2" Jul 14 17:32:30.230782 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223914 2562 flags.go:64] FLAG: --version="false" Jul 14 17:32:30.230782 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223918 2562 flags.go:64] FLAG: --vmodule="" Jul 14 17:32:30.231387 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223922 2562 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 14 17:32:30.231387 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.223925 2562 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 14 17:32:30.231387 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.224040 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 14 17:32:30.231387 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.224044 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 14 17:32:30.231387 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.224048 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 14 17:32:30.231387 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.224052 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 14 17:32:30.231387 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.224055 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 14 17:32:30.231387 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.224058 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 14 17:32:30.231387 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.224060 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 14 17:32:30.231387 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.224064 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 14 17:32:30.231387 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.224066 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 14 17:32:30.231387 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.224069 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 14 17:32:30.231387 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.224072 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 14 17:32:30.231387 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.224074 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 14 17:32:30.231387 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.224077 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 14 17:32:30.231387 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.224080 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 14 17:32:30.231387 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.224083 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 14 17:32:30.231387 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.224085 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 14 17:32:30.231387 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.224088 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 14 17:32:30.231387 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.224090 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 14 17:32:30.231897 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.224094 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 14 17:32:30.231897 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.224096 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 14 17:32:30.231897 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.224099 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 14 17:32:30.231897 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.224101 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 14 17:32:30.231897 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.224104 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 14 17:32:30.231897 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.224107 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 14 17:32:30.231897 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.224109 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 14 17:32:30.231897 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.224112 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 14 17:32:30.231897 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.224114 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 14 17:32:30.231897 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.224118 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 14 17:32:30.231897 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.224121 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 14 17:32:30.231897 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.224123 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 14 17:32:30.231897 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.224126 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 14 17:32:30.231897 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.224128 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 14 17:32:30.231897 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.224130 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 14 17:32:30.231897 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.224133 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 14 17:32:30.231897 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.224135 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 14 17:32:30.231897 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.224138 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 14 17:32:30.231897 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.224140 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 14 17:32:30.231897 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.224142 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 14 17:32:30.232375 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.224145 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 14 17:32:30.232375 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.224147 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 14 17:32:30.232375 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.224150 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 14 17:32:30.232375 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.224153 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 14 17:32:30.232375 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.224155 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 14 17:32:30.232375 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.224158 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 14 17:32:30.232375 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.224161 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 14 17:32:30.232375 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.224163 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 14 17:32:30.232375 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.224165 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 14 17:32:30.232375 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.224168 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 14 17:32:30.232375 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.224170 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 14 17:32:30.232375 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.224172 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 14 17:32:30.232375 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.224177 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 14 17:32:30.232375 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.224179 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 14 17:32:30.232375 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.224182 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 14 17:32:30.232375 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.224185 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 14 17:32:30.232375 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.224187 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 14 17:32:30.232375 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.224190 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 14 17:32:30.232375 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.224192 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 14 17:32:30.232375 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.224195 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 14 17:32:30.232915 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.224197 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 14 17:32:30.232915 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.224201 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 14 17:32:30.232915 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.224203 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 14 17:32:30.232915 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.224205 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 14 17:32:30.232915 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.224208 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 14 17:32:30.232915 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.224210 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 14 17:32:30.232915 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.224212 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 14 17:32:30.232915 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.224216 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 14 17:32:30.232915 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.224219 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 14 17:32:30.232915 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.224222 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 14 17:32:30.232915 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.224225 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 14 17:32:30.232915 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.224228 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 14 17:32:30.232915 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.224231 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 14 17:32:30.232915 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.224234 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 14 17:32:30.232915 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.224237 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 14 17:32:30.232915 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.224240 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 14 17:32:30.232915 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.224242 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 14 17:32:30.232915 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.224245 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 14 17:32:30.232915 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.224248 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 14 17:32:30.233374 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.224250 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 14 17:32:30.233374 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.224253 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 14 17:32:30.233374 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.224257 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 14 17:32:30.233374 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.224260 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 14 17:32:30.233374 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.224263 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 14 17:32:30.233374 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.224270 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 14 17:32:30.233374 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.224272 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 14 17:32:30.233374 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.224275 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 14 17:32:30.233374 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.224277 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 14 17:32:30.233374 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.224280 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 14 17:32:30.233374 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.224282 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 14 17:32:30.233374 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.224285 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 14 17:32:30.233374 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.224288 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 14 17:32:30.233374 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.224290 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 14 17:32:30.233374 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.224294 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 14 17:32:30.233374 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.224297 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 14 17:32:30.233374 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.224299 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 14 17:32:30.233374 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.224302 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 14 17:32:30.233374 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.224304 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 14 17:32:30.233851 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.224306 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 14 17:32:30.233851 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.224309 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 14 17:32:30.233851 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.224311 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 14 17:32:30.233851 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.224317 2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 14 17:32:30.233851 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.231006 2562 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 14 17:32:30.233851 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.231019 2562 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 14 17:32:30.233851 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.231889 2562 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 14 17:32:30.233851 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.231896 2562 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 14 17:32:30.233851 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.231955 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 14 17:32:30.233851 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.231961 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 14 17:32:30.233851 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.231965 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 14 17:32:30.233851 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.231968 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 14 17:32:30.233851 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.231971 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 14 17:32:30.233851 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.231974 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 14 17:32:30.234195 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.231976 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 14 17:32:30.234195 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.231979 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 14 17:32:30.234195 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.231982 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 14 17:32:30.234195 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.231985 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 14 17:32:30.234195 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.231987 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 14 17:32:30.234195 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.231990 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 14 17:32:30.234195 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.231992 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 14 17:32:30.234195 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.231995 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 14 17:32:30.234195 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.231997 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 14 17:32:30.234195 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232000 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 14 17:32:30.234195 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232003 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 14 17:32:30.234195 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232006 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 14 17:32:30.234195 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232008 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 14 17:32:30.234195 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232011 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 14 17:32:30.234195 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232013 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 14 17:32:30.234195 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232016 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 14 17:32:30.234195 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232018 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 14 17:32:30.234195 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232020 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 14 17:32:30.234195 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232023 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 14 17:32:30.234195 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232025 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 14 17:32:30.234686 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232028 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 14 17:32:30.234686 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232030 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 14 17:32:30.234686 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232033 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 14 17:32:30.234686 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232037 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 14 17:32:30.234686 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232040 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 14 17:32:30.234686 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232042 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 14 17:32:30.234686 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232045 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 14 17:32:30.234686 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232048 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 14 17:32:30.234686 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232052 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 14 17:32:30.234686 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232055 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 14 17:32:30.234686 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232059 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 14 17:32:30.234686 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232062 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 14 17:32:30.234686 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232065 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 14 17:32:30.234686 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232067 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 14 17:32:30.234686 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232070 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 14 17:32:30.234686 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232072 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 14 17:32:30.234686 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232075 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 14 17:32:30.234686 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232077 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 14 17:32:30.234686 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232080 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 14 17:32:30.235196 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232082 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 14 17:32:30.235196 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232085 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 14 17:32:30.235196 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232087 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 14 17:32:30.235196 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232089 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 14 17:32:30.235196 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232092 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 14 17:32:30.235196 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232094 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 14 17:32:30.235196 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232096 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 14 17:32:30.235196 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232099 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 14 17:32:30.235196 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232102 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 14 17:32:30.235196 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232104 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 14 17:32:30.235196 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232106 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 14 17:32:30.235196 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232108 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 14 17:32:30.235196 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232111 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 14 17:32:30.235196 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232113 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 14 17:32:30.235196 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232116 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 14 17:32:30.235196 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232119 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 14 17:32:30.235196 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232122 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 14 17:32:30.235196 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232125 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 14 17:32:30.235196 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232128 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 14 17:32:30.235196 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232130 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 14 17:32:30.235677 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232133 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 14 17:32:30.235677 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232136 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 14 17:32:30.235677 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232138 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 14 17:32:30.235677 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232140 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 14 17:32:30.235677 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232142 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 14 17:32:30.235677 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232145 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 14 17:32:30.235677 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232147 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 14 17:32:30.235677 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232149 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 14 17:32:30.235677 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232152 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 14 17:32:30.235677 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232154 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 14 17:32:30.235677 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232157 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 14 17:32:30.235677 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232159 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 14 17:32:30.235677 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232162 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 14 17:32:30.235677 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232164 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 14 17:32:30.235677 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232166 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 14 17:32:30.235677 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232169 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 14 17:32:30.235677 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232171 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 14 17:32:30.235677 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232174 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 14 17:32:30.235677 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232176 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 14 17:32:30.235677 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232178 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 14 17:32:30.236223 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232181 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 14 17:32:30.236223 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232183 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 14 17:32:30.236223 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232185 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 14 17:32:30.236223 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232188 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 14 17:32:30.236223 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232191 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 14 17:32:30.236223 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232193 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 14 17:32:30.236223 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232197 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 14 17:32:30.236223 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232200 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 14 17:32:30.236223 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232203 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 14 17:32:30.236223 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232205 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 14 17:32:30.236223 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232208 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 14 17:32:30.236223 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232211 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 14 17:32:30.236223 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232213 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 14 17:32:30.236223 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232215 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 14 17:32:30.236223 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.232220 2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 14 17:32:30.236638 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232331 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 14 17:32:30.236638 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232336 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 14 17:32:30.236638 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232339 2562 feature_gate.go:328] unrecognized feature gate: Example Jul 14 17:32:30.236638 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232342 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 14 17:32:30.236638 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232345 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 14 17:32:30.236638 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232347 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 14 17:32:30.236638 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232350 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 14 17:32:30.236638 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232352 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 14 17:32:30.236638 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232354 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 14 17:32:30.236638 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232356 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 14 17:32:30.236638 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232359 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 14 17:32:30.236638 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232362 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 14 17:32:30.236638 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232364 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 14 17:32:30.236638 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232367 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 14 17:32:30.236638 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232369 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 14 17:32:30.236638 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232372 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 14 17:32:30.236638 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232374 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 14 17:32:30.236638 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232377 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 14 17:32:30.236638 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232397 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 14 17:32:30.236638 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232401 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 14 17:32:30.237163 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232404 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 14 17:32:30.237163 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232406 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 14 17:32:30.237163 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232409 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 14 17:32:30.237163 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232412 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 14 17:32:30.237163 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232414 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 14 17:32:30.237163 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232417 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 14 17:32:30.237163 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232419 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 14 17:32:30.237163 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232423 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 14 17:32:30.237163 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232430 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 14 17:32:30.237163 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232433 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 14 17:32:30.237163 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232435 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 14 17:32:30.237163 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232438 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 14 17:32:30.237163 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232440 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 14 17:32:30.237163 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232443 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 14 17:32:30.237163 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232445 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 14 17:32:30.237163 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232447 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 14 17:32:30.237163 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232450 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 14 17:32:30.237163 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232452 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 14 17:32:30.237163 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232454 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 14 17:32:30.237163 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232457 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 14 17:32:30.237991 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232459 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 14 17:32:30.237991 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232461 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jul 14 17:32:30.237991 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232464 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 14 17:32:30.237991 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232466 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 14 17:32:30.237991 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232468 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 14 17:32:30.237991 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232471 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 14 17:32:30.237991 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232473 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 14 17:32:30.237991 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232476 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 14 17:32:30.237991 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232479 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 14 17:32:30.237991 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232482 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 14 17:32:30.237991 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232485 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 14 17:32:30.237991 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232487 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 14 17:32:30.237991 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232490 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 14 17:32:30.237991 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232492 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 14 17:32:30.237991 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232495 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 14 17:32:30.237991 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232497 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 14 17:32:30.237991 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232499 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 14 17:32:30.237991 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232501 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 14 17:32:30.237991 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232506 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 14 17:32:30.238528 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232509 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 14 17:32:30.238528 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232511 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 14 17:32:30.238528 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232514 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 14 17:32:30.238528 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232517 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 14 17:32:30.238528 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232520 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 14 17:32:30.238528 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232522 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 14 17:32:30.238528 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232525 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 14 17:32:30.238528 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232527 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 14 17:32:30.238528 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232530 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 14 17:32:30.238528 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232532 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 14 17:32:30.238528 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232534 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 14 17:32:30.238528 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232537 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 14 17:32:30.238528 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232539 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 14 17:32:30.238528 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232541 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 14 17:32:30.238528 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232543 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 14 17:32:30.238528 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232546 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 14 17:32:30.238528 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232548 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 14 17:32:30.238528 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232550 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 14 17:32:30.238528 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232553 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 14 17:32:30.238528 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232555 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 14 17:32:30.239166 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232558 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 14 17:32:30.239166 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232560 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 14 17:32:30.239166 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232563 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 14 17:32:30.239166 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232565 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 14 17:32:30.239166 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232568 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 14 17:32:30.239166 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232570 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 14 17:32:30.239166 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232572 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 14 17:32:30.239166 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232575 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 14 17:32:30.239166 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232577 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 14 17:32:30.239166 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232579 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 14 17:32:30.239166 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232582 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 14 17:32:30.239166 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232588 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 14 17:32:30.239166 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232591 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 14 17:32:30.239166 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232593 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 14 17:32:30.239166 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232596 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 14 17:32:30.239166 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232598 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 14 17:32:30.239166 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232601 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 14 17:32:30.239166 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232603 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 14 17:32:30.239166 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232605 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 14 17:32:30.239618 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:30.232608 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 14 17:32:30.239618 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.232612 2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 14 17:32:30.239618 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.232710 2562 server.go:962] "Client rotation is on, will bootstrap in background" Jul 14 17:32:30.239618 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.234871 2562 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 14 17:32:30.239618 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.236413 2562 server.go:1019] "Starting client certificate rotation" Jul 14 17:32:30.239618 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.236534 2562 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 14 17:32:30.239618 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.237150 2562 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 14 17:32:30.239618 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.238942 2562 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 14 17:32:30.240031 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.240011 2562 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 14 17:32:30.247869 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.247850 2562 log.go:25] "Validated CRI v1 runtime API" Jul 14 17:32:30.251684 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.251669 2562 log.go:25] "Validated CRI v1 image API" Jul 14 17:32:30.254113 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.254098 2562 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 14 17:32:30.256703 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.256686 2562 fs.go:135] Filesystem UUIDs: map[63e8b3e7-e793-405f-809d-df6a4d6b4f50:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 f69d7fc1-da0a-4d21-9018-2f1671d27ef4:/dev/nvme0n1p4] Jul 14 17:32:30.256746 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.256704 2562 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 14 17:32:30.263385 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.263280 2562 manager.go:216] Machine: {Timestamp:2026-07-14 17:32:30.26104173 +0000 UTC m=+0.360886939 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099604 MemoryCapacity:32812175360 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec28156c89db67b5fba84eb92943408e SystemUUID:ec28156c-89db-67b5-fba8-4eb92943408e BootID:da666070-cab8-4468-aca0-912d401b5ab6 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:59:62:33:19:e9 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:59:62:33:19:e9 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:96:e3:7f:9f:2a:c8 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812175360 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 14 17:32:30.263385 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.263381 2562 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 14 17:32:30.263497 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.263486 2562 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 14 17:32:30.263616 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.263598 2562 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 14 17:32:30.267903 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.267877 2562 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 14 17:32:30.268033 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.267906 2562 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-129-209.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 14 17:32:30.269023 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.269013 2562 topology_manager.go:138] "Creating topology manager with none policy" Jul 14 17:32:30.269054 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.269025 2562 container_manager_linux.go:309] "Creating device plugin manager" Jul 14 17:32:30.269054 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.269034 2562 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 14 17:32:30.270414 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.270404 2562 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 14 17:32:30.271700 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.271690 2562 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 14 17:32:30.275747 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.275736 2562 state_mem.go:36] "Initialized new in-memory state store" Jul 14 17:32:30.275890 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.275880 2562 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 14 17:32:30.275928 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.275911 2562 kubelet.go:486] "Attempting to sync node with API server" Jul 14 17:32:30.275928 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.275920 2562 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 14 17:32:30.275980 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.275933 2562 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 14 17:32:30.275980 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.275942 2562 kubelet.go:398] "Adding apiserver pod source" Jul 14 17:32:30.275980 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.275951 2562 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 14 17:32:30.276296 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.276281 2562 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 14 17:32:30.276338 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.276333 2562 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 14 17:32:30.278812 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.278782 2562 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 14 17:32:30.280707 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.280691 2562 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 14 17:32:30.280791 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.280730 2562 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 14 17:32:30.280943 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.280930 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 14 17:32:30.281002 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.280949 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 14 17:32:30.281002 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.280958 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 14 17:32:30.281002 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.280966 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 14 17:32:30.281002 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.280979 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 14 17:32:30.281002 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.280988 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 14 17:32:30.281673 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.281662 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 14 17:32:30.281721 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.281676 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 14 17:32:30.281721 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.281688 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 14 17:32:30.281721 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.281697 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 14 17:32:30.281721 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.281710 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 14 17:32:30.281721 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.281722 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 14 17:32:30.283482 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.283468 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 14 17:32:30.283482 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.283484 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 14 17:32:30.285315 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:32:30.285289 2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-129-209.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 14 17:32:30.285415 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:32:30.285306 2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 14 17:32:30.285415 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.285351 2562 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-209.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 14 17:32:30.286779 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.286763 2562 server.go:1268] "Started kubelet" Jul 14 17:32:30.286887 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.286857 2562 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 14 17:32:30.286997 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.286860 2562 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 14 17:32:30.287080 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.287066 2562 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 14 17:32:30.287659 ip-10-0-129-209 systemd[1]: Started Kubernetes Kubelet. Jul 14 17:32:30.291218 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.291193 2562 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 14 17:32:30.291741 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.291720 2562 server.go:310] "Adding debug handlers to kubelet server" Jul 14 17:32:30.292686 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:32:30.292665 2562 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 14 17:32:30.294001 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.293979 2562 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-6bk9h" Jul 14 17:32:30.296018 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:32:30.294828 2562 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-209.ec2.internal.18c238575e408b3b default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-209.ec2.internal,UID:ip-10-0-129-209.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-129-209.ec2.internal,},FirstTimestamp:2026-07-14 17:32:30.286736187 +0000 UTC m=+0.386581397,LastTimestamp:2026-07-14 17:32:30.286736187 +0000 UTC m=+0.386581397,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-209.ec2.internal,}" Jul 14 17:32:30.296131 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.296029 2562 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 14 17:32:30.296131 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.296035 2562 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 14 17:32:30.296225 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:32:30.296138 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-209.ec2.internal\" not found" Jul 14 17:32:30.296762 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.296745 2562 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 14 17:32:30.296762 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.296751 2562 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 14 17:32:30.296889 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.296782 2562 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 14 17:32:30.296889 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.296884 2562 reconstruct.go:97] "Volume reconstruction finished" Jul 14 17:32:30.296961 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.296891 2562 reconciler.go:29] "Reconciler: start to sync state" Jul 14 17:32:30.297195 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.297174 2562 factory.go:153] Registering CRI-O factory Jul 14 17:32:30.297240 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.297216 2562 factory.go:223] Registration of the crio container factory successfully Jul 14 17:32:30.297282 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.297271 2562 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 14 17:32:30.297313 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.297285 2562 factory.go:55] Registering systemd factory Jul 14 17:32:30.297313 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.297295 2562 factory.go:223] Registration of the systemd container factory successfully Jul 14 17:32:30.297401 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.297318 2562 factory.go:103] Registering Raw factory Jul 14 17:32:30.297401 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.297332 2562 manager.go:1196] Started watching for new ooms in manager Jul 14 17:32:30.297715 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.297704 2562 manager.go:349] Starting recovery of all containers Jul 14 17:32:30.302450 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.302428 2562 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-6bk9h" Jul 14 17:32:30.304407 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:32:30.304362 2562 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-129-209.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 14 17:32:30.304508 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:32:30.304488 2562 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 14 17:32:30.309206 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.309053 2562 manager.go:354] Recovery completed Jul 14 17:32:30.313486 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.313473 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 17:32:30.315381 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.315367 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-209.ec2.internal" event="NodeHasSufficientMemory" Jul 14 17:32:30.315448 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.315393 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-209.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 17:32:30.315448 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.315408 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-209.ec2.internal" event="NodeHasSufficientPID" Jul 14 17:32:30.315978 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.315965 2562 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 14 17:32:30.316031 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.315987 2562 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 14 17:32:30.316031 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.316005 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 17:32:30.316125 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.316006 2562 state_mem.go:36] "Initialized new in-memory state store" Jul 14 17:32:30.316675 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.316662 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-209.ec2.internal" event="NodeHasSufficientMemory" Jul 14 17:32:30.316717 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.316683 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-209.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 17:32:30.316717 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.316692 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-209.ec2.internal" event="NodeHasSufficientPID" Jul 14 17:32:30.318104 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.318092 2562 policy_none.go:49] "None policy: Start" Jul 14 17:32:30.318155 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.318109 2562 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 14 17:32:30.318155 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.318120 2562 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 14 17:32:30.320118 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.320107 2562 policy_none.go:47] "Start" Jul 14 17:32:30.355316 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.355295 2562 manager.go:337] "Starting Device Plugin manager" Jul 14 17:32:30.355386 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:32:30.355333 2562 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 14 17:32:30.355386 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.355343 2562 server.go:87] "Starting device plugin registration server" Jul 14 17:32:30.355555 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.355544 2562 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 14 17:32:30.355595 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.355556 2562 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 14 17:32:30.355654 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.355640 2562 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 14 17:32:30.355741 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.355730 2562 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 14 17:32:30.355788 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.355741 2562 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 14 17:32:30.356234 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:32:30.356213 2562 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 14 17:32:30.356289 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:32:30.356270 2562 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-129-209.ec2.internal\" not found" Jul 14 17:32:30.456194 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.456172 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 17:32:30.458146 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.456958 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-209.ec2.internal" event="NodeHasSufficientMemory" Jul 14 17:32:30.458146 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.456990 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-209.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 17:32:30.458146 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.457003 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-209.ec2.internal" event="NodeHasSufficientPID" Jul 14 17:32:30.458146 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.457026 2562 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-129-209.ec2.internal" Jul 14 17:32:30.464830 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.464764 2562 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-129-209.ec2.internal" Jul 14 17:32:30.464830 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:32:30.464785 2562 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-129-209.ec2.internal\": node \"ip-10-0-129-209.ec2.internal\" not found" Jul 14 17:32:30.477924 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:32:30.477902 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-209.ec2.internal\" not found" Jul 14 17:32:30.488775 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.488753 2562 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 14 17:32:30.489989 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.489970 2562 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 14 17:32:30.490071 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.490005 2562 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 14 17:32:30.490071 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.490023 2562 kubelet.go:2448] "Starting kubelet main sync loop" Jul 14 17:32:30.490071 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:32:30.490050 2562 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 14 17:32:30.494701 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.494681 2562 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 14 17:32:30.578260 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:32:30.578219 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-209.ec2.internal\" not found" Jul 14 17:32:30.590391 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.590360 2562 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-209.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-129-209.ec2.internal"] Jul 14 17:32:30.590456 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.590437 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 17:32:30.592050 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.592035 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-209.ec2.internal" event="NodeHasSufficientMemory" Jul 14 17:32:30.592110 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.592066 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-209.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 17:32:30.592110 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.592076 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-209.ec2.internal" event="NodeHasSufficientPID" Jul 14 17:32:30.594362 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.594350 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 17:32:30.594478 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.594454 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-209.ec2.internal" Jul 14 17:32:30.594527 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.594492 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 17:32:30.595205 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.595187 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-209.ec2.internal" event="NodeHasSufficientMemory" Jul 14 17:32:30.595291 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.595220 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-209.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 17:32:30.595291 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.595223 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-209.ec2.internal" event="NodeHasSufficientMemory" Jul 14 17:32:30.595291 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.595234 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-209.ec2.internal" event="NodeHasSufficientPID" Jul 14 17:32:30.595291 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.595241 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-209.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 17:32:30.595291 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.595250 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-209.ec2.internal" event="NodeHasSufficientPID" Jul 14 17:32:30.597372 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.597354 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-209.ec2.internal" Jul 14 17:32:30.597448 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.597381 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 14 17:32:30.597636 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.597595 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/0f6dec0287e6513120043e23e8c805f3-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-209.ec2.internal\" (UID: \"0f6dec0287e6513120043e23e8c805f3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-209.ec2.internal" Jul 14 17:32:30.597692 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.597664 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0f6dec0287e6513120043e23e8c805f3-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-209.ec2.internal\" (UID: \"0f6dec0287e6513120043e23e8c805f3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-209.ec2.internal" Jul 14 17:32:30.597742 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.597691 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/8b1a7c81fd72faef90d5a3084bb532e1-config\") pod \"kube-apiserver-proxy-ip-10-0-129-209.ec2.internal\" (UID: \"8b1a7c81fd72faef90d5a3084bb532e1\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-209.ec2.internal" Jul 14 17:32:30.598099 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.598080 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-209.ec2.internal" event="NodeHasSufficientMemory" Jul 14 17:32:30.598220 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.598108 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-209.ec2.internal" event="NodeHasNoDiskPressure" Jul 14 17:32:30.598220 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.598118 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-209.ec2.internal" event="NodeHasSufficientPID" Jul 14 17:32:30.623611 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:32:30.623588 2562 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-209.ec2.internal\" not found" node="ip-10-0-129-209.ec2.internal" Jul 14 17:32:30.627869 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:32:30.627852 2562 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-209.ec2.internal\" not found" node="ip-10-0-129-209.ec2.internal" Jul 14 17:32:30.679248 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:32:30.679230 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-209.ec2.internal\" not found" Jul 14 17:32:30.698563 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.698543 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/0f6dec0287e6513120043e23e8c805f3-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-209.ec2.internal\" (UID: \"0f6dec0287e6513120043e23e8c805f3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-209.ec2.internal" Jul 14 17:32:30.698635 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.698578 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0f6dec0287e6513120043e23e8c805f3-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-209.ec2.internal\" (UID: \"0f6dec0287e6513120043e23e8c805f3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-209.ec2.internal" Jul 14 17:32:30.698635 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.698597 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/8b1a7c81fd72faef90d5a3084bb532e1-config\") pod \"kube-apiserver-proxy-ip-10-0-129-209.ec2.internal\" (UID: \"8b1a7c81fd72faef90d5a3084bb532e1\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-209.ec2.internal" Jul 14 17:32:30.698726 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.698637 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0f6dec0287e6513120043e23e8c805f3-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-209.ec2.internal\" (UID: \"0f6dec0287e6513120043e23e8c805f3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-209.ec2.internal" Jul 14 17:32:30.698726 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.698641 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/0f6dec0287e6513120043e23e8c805f3-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-209.ec2.internal\" (UID: \"0f6dec0287e6513120043e23e8c805f3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-209.ec2.internal" Jul 14 17:32:30.698726 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:30.698697 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/8b1a7c81fd72faef90d5a3084bb532e1-config\") pod \"kube-apiserver-proxy-ip-10-0-129-209.ec2.internal\" (UID: \"8b1a7c81fd72faef90d5a3084bb532e1\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-209.ec2.internal" Jul 14 17:32:30.779999 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:32:30.779947 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-209.ec2.internal\" not found" Jul 14 17:32:30.880422 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:32:30.880397 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-209.ec2.internal\" not found" Jul 14 17:32:31.949331 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:31.949307 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-209.ec2.internal" Jul 14 17:32:31.955068 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:31.955054 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-209.ec2.internal" Jul 14 17:32:32.004826 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:32:32.004805 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-209.ec2.internal\" not found" Jul 14 17:32:32.105346 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:32:32.105308 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-209.ec2.internal\" not found" Jul 14 17:32:32.205795 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:32:32.205773 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-209.ec2.internal\" not found" Jul 14 17:32:32.259279 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:32.259263 2562 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 14 17:32:32.259814 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:32.259418 2562 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 14 17:32:32.306622 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:32:32.306600 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-209.ec2.internal\" not found" Jul 14 17:32:32.318500 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:32.318484 2562 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 14 17:32:32.319508 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:32.319493 2562 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 14 17:32:32.319586 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:32.319517 2562 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-209.ec2.internal" Jul 14 17:32:32.329888 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:32.329862 2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-13 17:27:31 +0000 UTC" deadline="2028-01-19 12:20:34.806834023 +0000 UTC" Jul 14 17:32:32.329888 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:32.329883 2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13290h48m2.476953165s" Jul 14 17:32:32.335353 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:32.335340 2562 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 14 17:32:32.336300 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:32.336285 2562 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 14 17:32:32.337385 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:32.337372 2562 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-209.ec2.internal" Jul 14 17:32:32.349490 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:32.349469 2562 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 14 17:32:32.360751 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:32.360710 2562 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-xk4r4" Jul 14 17:32:32.374808 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:32.374786 2562 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-xk4r4" Jul 14 17:32:32.544464 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:32.544432 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f6dec0287e6513120043e23e8c805f3.slice/crio-aabe73c4375ac99baa92a65b6cafb01e231700c6cfc7edc56491da6614322756 WatchSource:0}: Error finding container aabe73c4375ac99baa92a65b6cafb01e231700c6cfc7edc56491da6614322756: Status 404 returned error can't find the container with id aabe73c4375ac99baa92a65b6cafb01e231700c6cfc7edc56491da6614322756 Jul 14 17:32:32.547686 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:32.547673 2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 14 17:32:32.628466 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:32.628417 2562 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 14 17:32:32.673795 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:32.673773 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b1a7c81fd72faef90d5a3084bb532e1.slice/crio-09b62f1b7c67b3a923e34a41a65bd00195a67505ae7251c26088bb24e5949a68 WatchSource:0}: Error finding container 09b62f1b7c67b3a923e34a41a65bd00195a67505ae7251c26088bb24e5949a68: Status 404 returned error can't find the container with id 09b62f1b7c67b3a923e34a41a65bd00195a67505ae7251c26088bb24e5949a68 Jul 14 17:32:32.812203 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:32.812160 2562 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 14 17:32:33.300816 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.300781 2562 apiserver.go:52] "Watching apiserver" Jul 14 17:32:33.306969 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.306946 2562 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 14 17:32:33.307368 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.307342 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-node-tuning-operator/tuned-9gs5p","openshift-image-registry/node-ca-thnpp","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-209.ec2.internal","openshift-multus/multus-additional-cni-plugins-fjrz4","openshift-multus/network-metrics-daemon-h8t96","openshift-network-diagnostics/network-check-target-9m2f6","openshift-ovn-kubernetes/ovnkube-node-2sbww","kube-system/global-pull-secret-syncer-kv2bk","kube-system/konnectivity-agent-hxbnp","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7t9rj","openshift-dns/node-resolver-7fdn8","openshift-multus/multus-9dcd9","openshift-network-operator/iptables-alerter-prnxg","kube-system/kube-apiserver-proxy-ip-10-0-129-209.ec2.internal"] Jul 14 17:32:33.310102 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.310083 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-2sbww" Jul 14 17:32:33.312408 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.312381 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-prnxg" Jul 14 17:32:33.313390 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.313371 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 14 17:32:33.313678 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.313660 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 14 17:32:33.314553 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.314538 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kv2bk" Jul 14 17:32:33.314677 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:32:33.314651 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kv2bk" podUID="ac89d28e-0c57-4be5-8c3a-f0482ce0ba6f" Jul 14 17:32:33.314891 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.314870 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 14 17:32:33.314969 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.314905 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 14 17:32:33.315115 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.315100 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 14 17:32:33.315201 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.315126 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 14 17:32:33.315345 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.315331 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 14 17:32:33.315426 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.315361 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 14 17:32:33.315426 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.315403 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-g7dzq\"" Jul 14 17:32:33.315693 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.315677 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-mth2q\"" Jul 14 17:32:33.315693 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.315683 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 14 17:32:33.319326 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.316803 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-fjrz4" Jul 14 17:32:33.319326 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.319089 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h8t96" Jul 14 17:32:33.319326 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:32:33.319150 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-h8t96" podUID="8321d9c3-c963-45a9-8a94-ef9911a8d33f" Jul 14 17:32:33.319326 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.319250 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 14 17:32:33.319912 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.319364 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 14 17:32:33.319912 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.319422 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 14 17:32:33.319912 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.319612 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-r8q82\"" Jul 14 17:32:33.319912 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.319634 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 14 17:32:33.319912 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.319819 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 14 17:32:33.321306 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.321178 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9m2f6" Jul 14 17:32:33.321306 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:32:33.321254 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9m2f6" podUID="5b2d2388-62da-4462-af63-9230c6c43715" Jul 14 17:32:33.323444 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.323427 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-hxbnp" Jul 14 17:32:33.325456 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.325436 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 14 17:32:33.325573 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.325520 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-49xgx\"" Jul 14 17:32:33.325896 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.325734 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 14 17:32:33.326477 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.326460 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-9gs5p" Jul 14 17:32:33.328571 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.328396 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 14 17:32:33.328571 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.328412 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 14 17:32:33.328571 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.328464 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-77cpz\"" Jul 14 17:32:33.328728 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.328616 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-thnpp" Jul 14 17:32:33.329569 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.329551 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/dadd3729-1c46-48d9-92c8-bae9cdbbd1c1-cni-binary-copy\") pod \"multus-additional-cni-plugins-fjrz4\" (UID: \"dadd3729-1c46-48d9-92c8-bae9cdbbd1c1\") " pod="openshift-multus/multus-additional-cni-plugins-fjrz4" Jul 14 17:32:33.329639 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.329576 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8321d9c3-c963-45a9-8a94-ef9911a8d33f-metrics-certs\") pod \"network-metrics-daemon-h8t96\" (UID: \"8321d9c3-c963-45a9-8a94-ef9911a8d33f\") " pod="openshift-multus/network-metrics-daemon-h8t96" Jul 14 17:32:33.329639 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.329600 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c3d00b23-d0d5-4d18-a722-464d7b114745-run-systemd\") pod \"ovnkube-node-2sbww\" (UID: \"c3d00b23-d0d5-4d18-a722-464d7b114745\") " pod="openshift-ovn-kubernetes/ovnkube-node-2sbww" Jul 14 17:32:33.329738 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.329644 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c3d00b23-d0d5-4d18-a722-464d7b114745-host-run-ovn-kubernetes\") pod \"ovnkube-node-2sbww\" (UID: \"c3d00b23-d0d5-4d18-a722-464d7b114745\") " pod="openshift-ovn-kubernetes/ovnkube-node-2sbww" Jul 14 17:32:33.329738 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.329685 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c3d00b23-d0d5-4d18-a722-464d7b114745-host-cni-bin\") pod \"ovnkube-node-2sbww\" (UID: \"c3d00b23-d0d5-4d18-a722-464d7b114745\") " pod="openshift-ovn-kubernetes/ovnkube-node-2sbww" Jul 14 17:32:33.329829 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.329772 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c3d00b23-d0d5-4d18-a722-464d7b114745-host-cni-netd\") pod \"ovnkube-node-2sbww\" (UID: \"c3d00b23-d0d5-4d18-a722-464d7b114745\") " pod="openshift-ovn-kubernetes/ovnkube-node-2sbww" Jul 14 17:32:33.329829 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.329800 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c3d00b23-d0d5-4d18-a722-464d7b114745-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-2sbww\" (UID: \"c3d00b23-d0d5-4d18-a722-464d7b114745\") " pod="openshift-ovn-kubernetes/ovnkube-node-2sbww" Jul 14 17:32:33.329924 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.329838 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c3d00b23-d0d5-4d18-a722-464d7b114745-ovnkube-config\") pod \"ovnkube-node-2sbww\" (UID: \"c3d00b23-d0d5-4d18-a722-464d7b114745\") " pod="openshift-ovn-kubernetes/ovnkube-node-2sbww" Jul 14 17:32:33.329924 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.329858 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c3d00b23-d0d5-4d18-a722-464d7b114745-host-slash\") pod \"ovnkube-node-2sbww\" (UID: \"c3d00b23-d0d5-4d18-a722-464d7b114745\") " pod="openshift-ovn-kubernetes/ovnkube-node-2sbww" Jul 14 17:32:33.329924 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.329897 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c3d00b23-d0d5-4d18-a722-464d7b114745-host-run-netns\") pod \"ovnkube-node-2sbww\" (UID: \"c3d00b23-d0d5-4d18-a722-464d7b114745\") " pod="openshift-ovn-kubernetes/ovnkube-node-2sbww" Jul 14 17:32:33.329924 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.329919 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c3d00b23-d0d5-4d18-a722-464d7b114745-etc-openvswitch\") pod \"ovnkube-node-2sbww\" (UID: \"c3d00b23-d0d5-4d18-a722-464d7b114745\") " pod="openshift-ovn-kubernetes/ovnkube-node-2sbww" Jul 14 17:32:33.330111 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.329942 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c3d00b23-d0d5-4d18-a722-464d7b114745-node-log\") pod \"ovnkube-node-2sbww\" (UID: \"c3d00b23-d0d5-4d18-a722-464d7b114745\") " pod="openshift-ovn-kubernetes/ovnkube-node-2sbww" Jul 14 17:32:33.330111 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.329964 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c3d00b23-d0d5-4d18-a722-464d7b114745-ovn-node-metrics-cert\") pod \"ovnkube-node-2sbww\" (UID: \"c3d00b23-d0d5-4d18-a722-464d7b114745\") " pod="openshift-ovn-kubernetes/ovnkube-node-2sbww" Jul 14 17:32:33.330111 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.329992 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c3d00b23-d0d5-4d18-a722-464d7b114745-systemd-units\") pod \"ovnkube-node-2sbww\" (UID: \"c3d00b23-d0d5-4d18-a722-464d7b114745\") " pod="openshift-ovn-kubernetes/ovnkube-node-2sbww" Jul 14 17:32:33.330111 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.330014 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c3d00b23-d0d5-4d18-a722-464d7b114745-var-lib-openvswitch\") pod \"ovnkube-node-2sbww\" (UID: \"c3d00b23-d0d5-4d18-a722-464d7b114745\") " pod="openshift-ovn-kubernetes/ovnkube-node-2sbww" Jul 14 17:32:33.330111 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.330036 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c3d00b23-d0d5-4d18-a722-464d7b114745-run-openvswitch\") pod \"ovnkube-node-2sbww\" (UID: \"c3d00b23-d0d5-4d18-a722-464d7b114745\") " pod="openshift-ovn-kubernetes/ovnkube-node-2sbww" Jul 14 17:32:33.330111 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.330079 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2b670bba-aee6-407b-8f4e-e900528f54e5-host-slash\") pod \"iptables-alerter-prnxg\" (UID: \"2b670bba-aee6-407b-8f4e-e900528f54e5\") " pod="openshift-network-operator/iptables-alerter-prnxg" Jul 14 17:32:33.330111 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.330108 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c3d00b23-d0d5-4d18-a722-464d7b114745-log-socket\") pod \"ovnkube-node-2sbww\" (UID: \"c3d00b23-d0d5-4d18-a722-464d7b114745\") " pod="openshift-ovn-kubernetes/ovnkube-node-2sbww" Jul 14 17:32:33.330453 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.330133 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c3d00b23-d0d5-4d18-a722-464d7b114745-env-overrides\") pod \"ovnkube-node-2sbww\" (UID: \"c3d00b23-d0d5-4d18-a722-464d7b114745\") " pod="openshift-ovn-kubernetes/ovnkube-node-2sbww" Jul 14 17:32:33.330453 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.330170 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/dadd3729-1c46-48d9-92c8-bae9cdbbd1c1-cnibin\") pod \"multus-additional-cni-plugins-fjrz4\" (UID: \"dadd3729-1c46-48d9-92c8-bae9cdbbd1c1\") " pod="openshift-multus/multus-additional-cni-plugins-fjrz4" Jul 14 17:32:33.330453 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.330220 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/dadd3729-1c46-48d9-92c8-bae9cdbbd1c1-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-fjrz4\" (UID: \"dadd3729-1c46-48d9-92c8-bae9cdbbd1c1\") " pod="openshift-multus/multus-additional-cni-plugins-fjrz4" Jul 14 17:32:33.330453 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.330244 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pz6r7\" (UniqueName: \"kubernetes.io/projected/dadd3729-1c46-48d9-92c8-bae9cdbbd1c1-kube-api-access-pz6r7\") pod \"multus-additional-cni-plugins-fjrz4\" (UID: \"dadd3729-1c46-48d9-92c8-bae9cdbbd1c1\") " pod="openshift-multus/multus-additional-cni-plugins-fjrz4" Jul 14 17:32:33.330453 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.330278 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c3d00b23-d0d5-4d18-a722-464d7b114745-ovnkube-script-lib\") pod \"ovnkube-node-2sbww\" (UID: \"c3d00b23-d0d5-4d18-a722-464d7b114745\") " pod="openshift-ovn-kubernetes/ovnkube-node-2sbww" Jul 14 17:32:33.330453 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.330301 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ac89d28e-0c57-4be5-8c3a-f0482ce0ba6f-kubelet-config\") pod \"global-pull-secret-syncer-kv2bk\" (UID: \"ac89d28e-0c57-4be5-8c3a-f0482ce0ba6f\") " pod="kube-system/global-pull-secret-syncer-kv2bk" Jul 14 17:32:33.330453 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.330324 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dadd3729-1c46-48d9-92c8-bae9cdbbd1c1-system-cni-dir\") pod \"multus-additional-cni-plugins-fjrz4\" (UID: \"dadd3729-1c46-48d9-92c8-bae9cdbbd1c1\") " pod="openshift-multus/multus-additional-cni-plugins-fjrz4" Jul 14 17:32:33.330453 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.330347 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/dadd3729-1c46-48d9-92c8-bae9cdbbd1c1-tuning-conf-dir\") pod \"multus-additional-cni-plugins-fjrz4\" (UID: \"dadd3729-1c46-48d9-92c8-bae9cdbbd1c1\") " pod="openshift-multus/multus-additional-cni-plugins-fjrz4" Jul 14 17:32:33.330453 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.330372 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z57kn\" (UniqueName: \"kubernetes.io/projected/2b670bba-aee6-407b-8f4e-e900528f54e5-kube-api-access-z57kn\") pod \"iptables-alerter-prnxg\" (UID: \"2b670bba-aee6-407b-8f4e-e900528f54e5\") " pod="openshift-network-operator/iptables-alerter-prnxg" Jul 14 17:32:33.330453 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.330402 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ac89d28e-0c57-4be5-8c3a-f0482ce0ba6f-dbus\") pod \"global-pull-secret-syncer-kv2bk\" (UID: \"ac89d28e-0c57-4be5-8c3a-f0482ce0ba6f\") " pod="kube-system/global-pull-secret-syncer-kv2bk" Jul 14 17:32:33.331202 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.331003 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qz7zj\" (UniqueName: \"kubernetes.io/projected/8321d9c3-c963-45a9-8a94-ef9911a8d33f-kube-api-access-qz7zj\") pod \"network-metrics-daemon-h8t96\" (UID: \"8321d9c3-c963-45a9-8a94-ef9911a8d33f\") " pod="openshift-multus/network-metrics-daemon-h8t96" Jul 14 17:32:33.331298 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.331251 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5qqfb\" (UniqueName: \"kubernetes.io/projected/5b2d2388-62da-4462-af63-9230c6c43715-kube-api-access-5qqfb\") pod \"network-check-target-9m2f6\" (UID: \"5b2d2388-62da-4462-af63-9230c6c43715\") " pod="openshift-network-diagnostics/network-check-target-9m2f6" Jul 14 17:32:33.331360 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.331294 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c3d00b23-d0d5-4d18-a722-464d7b114745-run-ovn\") pod \"ovnkube-node-2sbww\" (UID: \"c3d00b23-d0d5-4d18-a722-464d7b114745\") " pod="openshift-ovn-kubernetes/ovnkube-node-2sbww" Jul 14 17:32:33.331360 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.331328 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-62qwb\" (UniqueName: \"kubernetes.io/projected/c3d00b23-d0d5-4d18-a722-464d7b114745-kube-api-access-62qwb\") pod \"ovnkube-node-2sbww\" (UID: \"c3d00b23-d0d5-4d18-a722-464d7b114745\") " pod="openshift-ovn-kubernetes/ovnkube-node-2sbww" Jul 14 17:32:33.331463 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.331364 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2b670bba-aee6-407b-8f4e-e900528f54e5-iptables-alerter-script\") pod \"iptables-alerter-prnxg\" (UID: \"2b670bba-aee6-407b-8f4e-e900528f54e5\") " pod="openshift-network-operator/iptables-alerter-prnxg" Jul 14 17:32:33.331463 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.331395 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/dadd3729-1c46-48d9-92c8-bae9cdbbd1c1-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-fjrz4\" (UID: \"dadd3729-1c46-48d9-92c8-bae9cdbbd1c1\") " pod="openshift-multus/multus-additional-cni-plugins-fjrz4" Jul 14 17:32:33.331463 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.331432 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c3d00b23-d0d5-4d18-a722-464d7b114745-host-kubelet\") pod \"ovnkube-node-2sbww\" (UID: \"c3d00b23-d0d5-4d18-a722-464d7b114745\") " pod="openshift-ovn-kubernetes/ovnkube-node-2sbww" Jul 14 17:32:33.331602 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.331464 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ac89d28e-0c57-4be5-8c3a-f0482ce0ba6f-original-pull-secret\") pod \"global-pull-secret-syncer-kv2bk\" (UID: \"ac89d28e-0c57-4be5-8c3a-f0482ce0ba6f\") " pod="kube-system/global-pull-secret-syncer-kv2bk" Jul 14 17:32:33.331602 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.331536 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/dadd3729-1c46-48d9-92c8-bae9cdbbd1c1-os-release\") pod \"multus-additional-cni-plugins-fjrz4\" (UID: \"dadd3729-1c46-48d9-92c8-bae9cdbbd1c1\") " pod="openshift-multus/multus-additional-cni-plugins-fjrz4" Jul 14 17:32:33.331702 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.331667 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7t9rj" Jul 14 17:32:33.331832 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.331805 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 14 17:32:33.332129 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.332108 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-8fsq9\"" Jul 14 17:32:33.332542 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.332524 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 14 17:32:33.333311 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.333283 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 14 17:32:33.334531 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.334510 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 14 17:32:33.335503 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.335484 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-bvqph\"" Jul 14 17:32:33.335588 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.335542 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 14 17:32:33.335588 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.335576 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 14 17:32:33.336694 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.336673 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-7fdn8" Jul 14 17:32:33.338841 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.338823 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-wkdqr\"" Jul 14 17:32:33.338933 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.338831 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 14 17:32:33.339017 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.339002 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-9dcd9" Jul 14 17:32:33.339422 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.339402 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 14 17:32:33.341291 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.341274 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 14 17:32:33.341916 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.341900 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-sqmxw\"" Jul 14 17:32:33.377717 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.377245 2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-13 17:27:32 +0000 UTC" deadline="2028-04-22 04:24:36.793372158 +0000 UTC" Jul 14 17:32:33.377717 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.377287 2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15538h52m3.416088865s" Jul 14 17:32:33.421182 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.421156 2562 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 14 17:32:33.431839 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.431814 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c3d00b23-d0d5-4d18-a722-464d7b114745-host-run-ovn-kubernetes\") pod \"ovnkube-node-2sbww\" (UID: \"c3d00b23-d0d5-4d18-a722-464d7b114745\") " pod="openshift-ovn-kubernetes/ovnkube-node-2sbww" Jul 14 17:32:33.431954 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.431851 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c3d00b23-d0d5-4d18-a722-464d7b114745-host-cni-netd\") pod \"ovnkube-node-2sbww\" (UID: \"c3d00b23-d0d5-4d18-a722-464d7b114745\") " pod="openshift-ovn-kubernetes/ovnkube-node-2sbww" Jul 14 17:32:33.431954 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.431878 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c3d00b23-d0d5-4d18-a722-464d7b114745-ovnkube-config\") pod \"ovnkube-node-2sbww\" (UID: \"c3d00b23-d0d5-4d18-a722-464d7b114745\") " pod="openshift-ovn-kubernetes/ovnkube-node-2sbww" Jul 14 17:32:33.431954 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.431891 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c3d00b23-d0d5-4d18-a722-464d7b114745-host-run-ovn-kubernetes\") pod \"ovnkube-node-2sbww\" (UID: \"c3d00b23-d0d5-4d18-a722-464d7b114745\") " pod="openshift-ovn-kubernetes/ovnkube-node-2sbww" Jul 14 17:32:33.431954 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.431904 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/650b148f-78d8-456f-bc24-1fbe4a75a24a-konnectivity-ca\") pod \"konnectivity-agent-hxbnp\" (UID: \"650b148f-78d8-456f-bc24-1fbe4a75a24a\") " pod="kube-system/konnectivity-agent-hxbnp" Jul 14 17:32:33.431954 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.431929 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/56a1f173-7ec8-4f09-94ce-ee3bde8e4c17-run\") pod \"tuned-9gs5p\" (UID: \"56a1f173-7ec8-4f09-94ce-ee3bde8e4c17\") " pod="openshift-cluster-node-tuning-operator/tuned-9gs5p" Jul 14 17:32:33.431954 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.431946 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c3d00b23-d0d5-4d18-a722-464d7b114745-host-cni-netd\") pod \"ovnkube-node-2sbww\" (UID: \"c3d00b23-d0d5-4d18-a722-464d7b114745\") " pod="openshift-ovn-kubernetes/ovnkube-node-2sbww" Jul 14 17:32:33.431954 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.431953 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/56a1f173-7ec8-4f09-94ce-ee3bde8e4c17-host\") pod \"tuned-9gs5p\" (UID: \"56a1f173-7ec8-4f09-94ce-ee3bde8e4c17\") " pod="openshift-cluster-node-tuning-operator/tuned-9gs5p" Jul 14 17:32:33.432301 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.431984 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/5bca996e-4e3a-4ec6-bd2f-0a5860c9d632-sys-fs\") pod \"aws-ebs-csi-driver-node-7t9rj\" (UID: \"5bca996e-4e3a-4ec6-bd2f-0a5860c9d632\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7t9rj" Jul 14 17:32:33.432301 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.432011 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c3d00b23-d0d5-4d18-a722-464d7b114745-host-run-netns\") pod \"ovnkube-node-2sbww\" (UID: \"c3d00b23-d0d5-4d18-a722-464d7b114745\") " pod="openshift-ovn-kubernetes/ovnkube-node-2sbww" Jul 14 17:32:33.432301 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.432034 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c3d00b23-d0d5-4d18-a722-464d7b114745-etc-openvswitch\") pod \"ovnkube-node-2sbww\" (UID: \"c3d00b23-d0d5-4d18-a722-464d7b114745\") " pod="openshift-ovn-kubernetes/ovnkube-node-2sbww" Jul 14 17:32:33.432301 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.432057 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/56a1f173-7ec8-4f09-94ce-ee3bde8e4c17-lib-modules\") pod \"tuned-9gs5p\" (UID: \"56a1f173-7ec8-4f09-94ce-ee3bde8e4c17\") " pod="openshift-cluster-node-tuning-operator/tuned-9gs5p" Jul 14 17:32:33.432301 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.432081 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c3d00b23-d0d5-4d18-a722-464d7b114745-var-lib-openvswitch\") pod \"ovnkube-node-2sbww\" (UID: \"c3d00b23-d0d5-4d18-a722-464d7b114745\") " pod="openshift-ovn-kubernetes/ovnkube-node-2sbww" Jul 14 17:32:33.432301 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.432105 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2b670bba-aee6-407b-8f4e-e900528f54e5-host-slash\") pod \"iptables-alerter-prnxg\" (UID: \"2b670bba-aee6-407b-8f4e-e900528f54e5\") " pod="openshift-network-operator/iptables-alerter-prnxg" Jul 14 17:32:33.432301 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.432106 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c3d00b23-d0d5-4d18-a722-464d7b114745-host-run-netns\") pod \"ovnkube-node-2sbww\" (UID: \"c3d00b23-d0d5-4d18-a722-464d7b114745\") " pod="openshift-ovn-kubernetes/ovnkube-node-2sbww" Jul 14 17:32:33.432301 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.432129 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m9kld\" (UniqueName: \"kubernetes.io/projected/48bfbea2-4a10-4454-a93d-72c4c8135d79-kube-api-access-m9kld\") pod \"multus-9dcd9\" (UID: \"48bfbea2-4a10-4454-a93d-72c4c8135d79\") " pod="openshift-multus/multus-9dcd9" Jul 14 17:32:33.432301 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.432168 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c3d00b23-d0d5-4d18-a722-464d7b114745-etc-openvswitch\") pod \"ovnkube-node-2sbww\" (UID: \"c3d00b23-d0d5-4d18-a722-464d7b114745\") " pod="openshift-ovn-kubernetes/ovnkube-node-2sbww" Jul 14 17:32:33.432301 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.432170 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/650b148f-78d8-456f-bc24-1fbe4a75a24a-agent-certs\") pod \"konnectivity-agent-hxbnp\" (UID: \"650b148f-78d8-456f-bc24-1fbe4a75a24a\") " pod="kube-system/konnectivity-agent-hxbnp" Jul 14 17:32:33.432301 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.432233 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/56a1f173-7ec8-4f09-94ce-ee3bde8e4c17-sys\") pod \"tuned-9gs5p\" (UID: \"56a1f173-7ec8-4f09-94ce-ee3bde8e4c17\") " pod="openshift-cluster-node-tuning-operator/tuned-9gs5p" Jul 14 17:32:33.432301 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.432235 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c3d00b23-d0d5-4d18-a722-464d7b114745-var-lib-openvswitch\") pod \"ovnkube-node-2sbww\" (UID: \"c3d00b23-d0d5-4d18-a722-464d7b114745\") " pod="openshift-ovn-kubernetes/ovnkube-node-2sbww" Jul 14 17:32:33.432301 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.432279 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c3d00b23-d0d5-4d18-a722-464d7b114745-log-socket\") pod \"ovnkube-node-2sbww\" (UID: \"c3d00b23-d0d5-4d18-a722-464d7b114745\") " pod="openshift-ovn-kubernetes/ovnkube-node-2sbww" Jul 14 17:32:33.432301 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.432301 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2b670bba-aee6-407b-8f4e-e900528f54e5-host-slash\") pod \"iptables-alerter-prnxg\" (UID: \"2b670bba-aee6-407b-8f4e-e900528f54e5\") " pod="openshift-network-operator/iptables-alerter-prnxg" Jul 14 17:32:33.432301 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.432305 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/dadd3729-1c46-48d9-92c8-bae9cdbbd1c1-cnibin\") pod \"multus-additional-cni-plugins-fjrz4\" (UID: \"dadd3729-1c46-48d9-92c8-bae9cdbbd1c1\") " pod="openshift-multus/multus-additional-cni-plugins-fjrz4" Jul 14 17:32:33.432853 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.432348 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pz6r7\" (UniqueName: \"kubernetes.io/projected/dadd3729-1c46-48d9-92c8-bae9cdbbd1c1-kube-api-access-pz6r7\") pod \"multus-additional-cni-plugins-fjrz4\" (UID: \"dadd3729-1c46-48d9-92c8-bae9cdbbd1c1\") " pod="openshift-multus/multus-additional-cni-plugins-fjrz4" Jul 14 17:32:33.432853 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.432350 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c3d00b23-d0d5-4d18-a722-464d7b114745-log-socket\") pod \"ovnkube-node-2sbww\" (UID: \"c3d00b23-d0d5-4d18-a722-464d7b114745\") " pod="openshift-ovn-kubernetes/ovnkube-node-2sbww" Jul 14 17:32:33.432853 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.432381 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/dadd3729-1c46-48d9-92c8-bae9cdbbd1c1-cnibin\") pod \"multus-additional-cni-plugins-fjrz4\" (UID: \"dadd3729-1c46-48d9-92c8-bae9cdbbd1c1\") " pod="openshift-multus/multus-additional-cni-plugins-fjrz4" Jul 14 17:32:33.432853 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.432425 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c3d00b23-d0d5-4d18-a722-464d7b114745-ovnkube-script-lib\") pod \"ovnkube-node-2sbww\" (UID: \"c3d00b23-d0d5-4d18-a722-464d7b114745\") " pod="openshift-ovn-kubernetes/ovnkube-node-2sbww" Jul 14 17:32:33.432853 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.432472 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ac89d28e-0c57-4be5-8c3a-f0482ce0ba6f-kubelet-config\") pod \"global-pull-secret-syncer-kv2bk\" (UID: \"ac89d28e-0c57-4be5-8c3a-f0482ce0ba6f\") " pod="kube-system/global-pull-secret-syncer-kv2bk" Jul 14 17:32:33.432853 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.432587 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dadd3729-1c46-48d9-92c8-bae9cdbbd1c1-system-cni-dir\") pod \"multus-additional-cni-plugins-fjrz4\" (UID: \"dadd3729-1c46-48d9-92c8-bae9cdbbd1c1\") " pod="openshift-multus/multus-additional-cni-plugins-fjrz4" Jul 14 17:32:33.432853 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.432614 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c3d00b23-d0d5-4d18-a722-464d7b114745-ovnkube-config\") pod \"ovnkube-node-2sbww\" (UID: \"c3d00b23-d0d5-4d18-a722-464d7b114745\") " pod="openshift-ovn-kubernetes/ovnkube-node-2sbww" Jul 14 17:32:33.432853 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.432619 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ac89d28e-0c57-4be5-8c3a-f0482ce0ba6f-kubelet-config\") pod \"global-pull-secret-syncer-kv2bk\" (UID: \"ac89d28e-0c57-4be5-8c3a-f0482ce0ba6f\") " pod="kube-system/global-pull-secret-syncer-kv2bk" Jul 14 17:32:33.432853 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.432664 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/48bfbea2-4a10-4454-a93d-72c4c8135d79-system-cni-dir\") pod \"multus-9dcd9\" (UID: \"48bfbea2-4a10-4454-a93d-72c4c8135d79\") " pod="openshift-multus/multus-9dcd9" Jul 14 17:32:33.432853 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.432677 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dadd3729-1c46-48d9-92c8-bae9cdbbd1c1-system-cni-dir\") pod \"multus-additional-cni-plugins-fjrz4\" (UID: \"dadd3729-1c46-48d9-92c8-bae9cdbbd1c1\") " pod="openshift-multus/multus-additional-cni-plugins-fjrz4" Jul 14 17:32:33.432853 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.432722 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/48bfbea2-4a10-4454-a93d-72c4c8135d79-multus-daemon-config\") pod \"multus-9dcd9\" (UID: \"48bfbea2-4a10-4454-a93d-72c4c8135d79\") " pod="openshift-multus/multus-9dcd9" Jul 14 17:32:33.432853 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.432763 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/5bca996e-4e3a-4ec6-bd2f-0a5860c9d632-socket-dir\") pod \"aws-ebs-csi-driver-node-7t9rj\" (UID: \"5bca996e-4e3a-4ec6-bd2f-0a5860c9d632\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7t9rj" Jul 14 17:32:33.432853 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.432793 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b7fkv\" (UniqueName: \"kubernetes.io/projected/5bca996e-4e3a-4ec6-bd2f-0a5860c9d632-kube-api-access-b7fkv\") pod \"aws-ebs-csi-driver-node-7t9rj\" (UID: \"5bca996e-4e3a-4ec6-bd2f-0a5860c9d632\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7t9rj" Jul 14 17:32:33.432853 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.432818 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/48bfbea2-4a10-4454-a93d-72c4c8135d79-multus-conf-dir\") pod \"multus-9dcd9\" (UID: \"48bfbea2-4a10-4454-a93d-72c4c8135d79\") " pod="openshift-multus/multus-9dcd9" Jul 14 17:32:33.432853 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.432844 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z57kn\" (UniqueName: \"kubernetes.io/projected/2b670bba-aee6-407b-8f4e-e900528f54e5-kube-api-access-z57kn\") pod \"iptables-alerter-prnxg\" (UID: \"2b670bba-aee6-407b-8f4e-e900528f54e5\") " pod="openshift-network-operator/iptables-alerter-prnxg" Jul 14 17:32:33.433577 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.432868 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ac89d28e-0c57-4be5-8c3a-f0482ce0ba6f-dbus\") pod \"global-pull-secret-syncer-kv2bk\" (UID: \"ac89d28e-0c57-4be5-8c3a-f0482ce0ba6f\") " pod="kube-system/global-pull-secret-syncer-kv2bk" Jul 14 17:32:33.433577 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.432892 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/48bfbea2-4a10-4454-a93d-72c4c8135d79-host-var-lib-cni-bin\") pod \"multus-9dcd9\" (UID: \"48bfbea2-4a10-4454-a93d-72c4c8135d79\") " pod="openshift-multus/multus-9dcd9" Jul 14 17:32:33.433577 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.432914 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/48bfbea2-4a10-4454-a93d-72c4c8135d79-host-var-lib-cni-multus\") pod \"multus-9dcd9\" (UID: \"48bfbea2-4a10-4454-a93d-72c4c8135d79\") " pod="openshift-multus/multus-9dcd9" Jul 14 17:32:33.433577 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.432939 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/5bca996e-4e3a-4ec6-bd2f-0a5860c9d632-device-dir\") pod \"aws-ebs-csi-driver-node-7t9rj\" (UID: \"5bca996e-4e3a-4ec6-bd2f-0a5860c9d632\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7t9rj" Jul 14 17:32:33.433577 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.432963 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2b670bba-aee6-407b-8f4e-e900528f54e5-iptables-alerter-script\") pod \"iptables-alerter-prnxg\" (UID: \"2b670bba-aee6-407b-8f4e-e900528f54e5\") " pod="openshift-network-operator/iptables-alerter-prnxg" Jul 14 17:32:33.433577 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.432984 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/48bfbea2-4a10-4454-a93d-72c4c8135d79-os-release\") pod \"multus-9dcd9\" (UID: \"48bfbea2-4a10-4454-a93d-72c4c8135d79\") " pod="openshift-multus/multus-9dcd9" Jul 14 17:32:33.433577 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.433006 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/56a1f173-7ec8-4f09-94ce-ee3bde8e4c17-var-lib-kubelet\") pod \"tuned-9gs5p\" (UID: \"56a1f173-7ec8-4f09-94ce-ee3bde8e4c17\") " pod="openshift-cluster-node-tuning-operator/tuned-9gs5p" Jul 14 17:32:33.433577 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.433026 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/17370624-81e7-42ff-b916-e3f323b3e0a5-serviceca\") pod \"node-ca-thnpp\" (UID: \"17370624-81e7-42ff-b916-e3f323b3e0a5\") " pod="openshift-image-registry/node-ca-thnpp" Jul 14 17:32:33.433577 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.433051 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c3d00b23-d0d5-4d18-a722-464d7b114745-host-kubelet\") pod \"ovnkube-node-2sbww\" (UID: \"c3d00b23-d0d5-4d18-a722-464d7b114745\") " pod="openshift-ovn-kubernetes/ovnkube-node-2sbww" Jul 14 17:32:33.433577 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.433027 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c3d00b23-d0d5-4d18-a722-464d7b114745-ovnkube-script-lib\") pod \"ovnkube-node-2sbww\" (UID: \"c3d00b23-d0d5-4d18-a722-464d7b114745\") " pod="openshift-ovn-kubernetes/ovnkube-node-2sbww" Jul 14 17:32:33.433577 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.433108 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/56a1f173-7ec8-4f09-94ce-ee3bde8e4c17-etc-sysctl-d\") pod \"tuned-9gs5p\" (UID: \"56a1f173-7ec8-4f09-94ce-ee3bde8e4c17\") " pod="openshift-cluster-node-tuning-operator/tuned-9gs5p" Jul 14 17:32:33.433577 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.433126 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c3d00b23-d0d5-4d18-a722-464d7b114745-host-kubelet\") pod \"ovnkube-node-2sbww\" (UID: \"c3d00b23-d0d5-4d18-a722-464d7b114745\") " pod="openshift-ovn-kubernetes/ovnkube-node-2sbww" Jul 14 17:32:33.433577 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.433167 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c3d00b23-d0d5-4d18-a722-464d7b114745-run-systemd\") pod \"ovnkube-node-2sbww\" (UID: \"c3d00b23-d0d5-4d18-a722-464d7b114745\") " pod="openshift-ovn-kubernetes/ovnkube-node-2sbww" Jul 14 17:32:33.433577 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.433214 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c3d00b23-d0d5-4d18-a722-464d7b114745-host-cni-bin\") pod \"ovnkube-node-2sbww\" (UID: \"c3d00b23-d0d5-4d18-a722-464d7b114745\") " pod="openshift-ovn-kubernetes/ovnkube-node-2sbww" Jul 14 17:32:33.433577 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.433225 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c3d00b23-d0d5-4d18-a722-464d7b114745-run-systemd\") pod \"ovnkube-node-2sbww\" (UID: \"c3d00b23-d0d5-4d18-a722-464d7b114745\") " pod="openshift-ovn-kubernetes/ovnkube-node-2sbww" Jul 14 17:32:33.433577 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.433257 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c3d00b23-d0d5-4d18-a722-464d7b114745-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-2sbww\" (UID: \"c3d00b23-d0d5-4d18-a722-464d7b114745\") " pod="openshift-ovn-kubernetes/ovnkube-node-2sbww" Jul 14 17:32:33.433577 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.433272 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c3d00b23-d0d5-4d18-a722-464d7b114745-host-cni-bin\") pod \"ovnkube-node-2sbww\" (UID: \"c3d00b23-d0d5-4d18-a722-464d7b114745\") " pod="openshift-ovn-kubernetes/ovnkube-node-2sbww" Jul 14 17:32:33.434346 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.433279 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/91ee1d15-3741-4cf5-821e-d3b216b7e246-hosts-file\") pod \"node-resolver-7fdn8\" (UID: \"91ee1d15-3741-4cf5-821e-d3b216b7e246\") " pod="openshift-dns/node-resolver-7fdn8" Jul 14 17:32:33.434346 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.433300 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/48bfbea2-4a10-4454-a93d-72c4c8135d79-multus-socket-dir-parent\") pod \"multus-9dcd9\" (UID: \"48bfbea2-4a10-4454-a93d-72c4c8135d79\") " pod="openshift-multus/multus-9dcd9" Jul 14 17:32:33.434346 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.433299 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c3d00b23-d0d5-4d18-a722-464d7b114745-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-2sbww\" (UID: \"c3d00b23-d0d5-4d18-a722-464d7b114745\") " pod="openshift-ovn-kubernetes/ovnkube-node-2sbww" Jul 14 17:32:33.434346 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.433316 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/48bfbea2-4a10-4454-a93d-72c4c8135d79-host-run-k8s-cni-cncf-io\") pod \"multus-9dcd9\" (UID: \"48bfbea2-4a10-4454-a93d-72c4c8135d79\") " pod="openshift-multus/multus-9dcd9" Jul 14 17:32:33.434346 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.433345 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/56a1f173-7ec8-4f09-94ce-ee3bde8e4c17-etc-modprobe-d\") pod \"tuned-9gs5p\" (UID: \"56a1f173-7ec8-4f09-94ce-ee3bde8e4c17\") " pod="openshift-cluster-node-tuning-operator/tuned-9gs5p" Jul 14 17:32:33.434346 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.433363 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/5bca996e-4e3a-4ec6-bd2f-0a5860c9d632-registration-dir\") pod \"aws-ebs-csi-driver-node-7t9rj\" (UID: \"5bca996e-4e3a-4ec6-bd2f-0a5860c9d632\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7t9rj" Jul 14 17:32:33.434346 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.433385 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c3d00b23-d0d5-4d18-a722-464d7b114745-host-slash\") pod \"ovnkube-node-2sbww\" (UID: \"c3d00b23-d0d5-4d18-a722-464d7b114745\") " pod="openshift-ovn-kubernetes/ovnkube-node-2sbww" Jul 14 17:32:33.434346 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.433400 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c3d00b23-d0d5-4d18-a722-464d7b114745-node-log\") pod \"ovnkube-node-2sbww\" (UID: \"c3d00b23-d0d5-4d18-a722-464d7b114745\") " pod="openshift-ovn-kubernetes/ovnkube-node-2sbww" Jul 14 17:32:33.434346 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.433416 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c3d00b23-d0d5-4d18-a722-464d7b114745-ovn-node-metrics-cert\") pod \"ovnkube-node-2sbww\" (UID: \"c3d00b23-d0d5-4d18-a722-464d7b114745\") " pod="openshift-ovn-kubernetes/ovnkube-node-2sbww" Jul 14 17:32:33.434346 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.433430 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/48bfbea2-4a10-4454-a93d-72c4c8135d79-host-run-netns\") pod \"multus-9dcd9\" (UID: \"48bfbea2-4a10-4454-a93d-72c4c8135d79\") " pod="openshift-multus/multus-9dcd9" Jul 14 17:32:33.434346 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.433457 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/2b670bba-aee6-407b-8f4e-e900528f54e5-iptables-alerter-script\") pod \"iptables-alerter-prnxg\" (UID: \"2b670bba-aee6-407b-8f4e-e900528f54e5\") " pod="openshift-network-operator/iptables-alerter-prnxg" Jul 14 17:32:33.434346 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.433461 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/48bfbea2-4a10-4454-a93d-72c4c8135d79-host-run-multus-certs\") pod \"multus-9dcd9\" (UID: \"48bfbea2-4a10-4454-a93d-72c4c8135d79\") " pod="openshift-multus/multus-9dcd9" Jul 14 17:32:33.434346 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.433461 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c3d00b23-d0d5-4d18-a722-464d7b114745-node-log\") pod \"ovnkube-node-2sbww\" (UID: \"c3d00b23-d0d5-4d18-a722-464d7b114745\") " pod="openshift-ovn-kubernetes/ovnkube-node-2sbww" Jul 14 17:32:33.434346 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.433508 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c3d00b23-d0d5-4d18-a722-464d7b114745-host-slash\") pod \"ovnkube-node-2sbww\" (UID: \"c3d00b23-d0d5-4d18-a722-464d7b114745\") " pod="openshift-ovn-kubernetes/ovnkube-node-2sbww" Jul 14 17:32:33.434346 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.433543 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/56a1f173-7ec8-4f09-94ce-ee3bde8e4c17-etc-sysctl-conf\") pod \"tuned-9gs5p\" (UID: \"56a1f173-7ec8-4f09-94ce-ee3bde8e4c17\") " pod="openshift-cluster-node-tuning-operator/tuned-9gs5p" Jul 14 17:32:33.434346 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.433558 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ac89d28e-0c57-4be5-8c3a-f0482ce0ba6f-dbus\") pod \"global-pull-secret-syncer-kv2bk\" (UID: \"ac89d28e-0c57-4be5-8c3a-f0482ce0ba6f\") " pod="kube-system/global-pull-secret-syncer-kv2bk" Jul 14 17:32:33.434346 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.433589 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c3d00b23-d0d5-4d18-a722-464d7b114745-systemd-units\") pod \"ovnkube-node-2sbww\" (UID: \"c3d00b23-d0d5-4d18-a722-464d7b114745\") " pod="openshift-ovn-kubernetes/ovnkube-node-2sbww" Jul 14 17:32:33.435229 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.433616 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c3d00b23-d0d5-4d18-a722-464d7b114745-run-openvswitch\") pod \"ovnkube-node-2sbww\" (UID: \"c3d00b23-d0d5-4d18-a722-464d7b114745\") " pod="openshift-ovn-kubernetes/ovnkube-node-2sbww" Jul 14 17:32:33.435229 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.433637 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c3d00b23-d0d5-4d18-a722-464d7b114745-systemd-units\") pod \"ovnkube-node-2sbww\" (UID: \"c3d00b23-d0d5-4d18-a722-464d7b114745\") " pod="openshift-ovn-kubernetes/ovnkube-node-2sbww" Jul 14 17:32:33.435229 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.433639 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/91ee1d15-3741-4cf5-821e-d3b216b7e246-tmp-dir\") pod \"node-resolver-7fdn8\" (UID: \"91ee1d15-3741-4cf5-821e-d3b216b7e246\") " pod="openshift-dns/node-resolver-7fdn8" Jul 14 17:32:33.435229 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.433686 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c3d00b23-d0d5-4d18-a722-464d7b114745-run-openvswitch\") pod \"ovnkube-node-2sbww\" (UID: \"c3d00b23-d0d5-4d18-a722-464d7b114745\") " pod="openshift-ovn-kubernetes/ovnkube-node-2sbww" Jul 14 17:32:33.435229 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.433711 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/56a1f173-7ec8-4f09-94ce-ee3bde8e4c17-etc-systemd\") pod \"tuned-9gs5p\" (UID: \"56a1f173-7ec8-4f09-94ce-ee3bde8e4c17\") " pod="openshift-cluster-node-tuning-operator/tuned-9gs5p" Jul 14 17:32:33.435229 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.433738 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lqqvr\" (UniqueName: \"kubernetes.io/projected/56a1f173-7ec8-4f09-94ce-ee3bde8e4c17-kube-api-access-lqqvr\") pod \"tuned-9gs5p\" (UID: \"56a1f173-7ec8-4f09-94ce-ee3bde8e4c17\") " pod="openshift-cluster-node-tuning-operator/tuned-9gs5p" Jul 14 17:32:33.435229 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.433788 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/5bca996e-4e3a-4ec6-bd2f-0a5860c9d632-kubelet-dir\") pod \"aws-ebs-csi-driver-node-7t9rj\" (UID: \"5bca996e-4e3a-4ec6-bd2f-0a5860c9d632\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7t9rj" Jul 14 17:32:33.435229 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.433848 2562 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 14 17:32:33.435229 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.433864 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/5bca996e-4e3a-4ec6-bd2f-0a5860c9d632-etc-selinux\") pod \"aws-ebs-csi-driver-node-7t9rj\" (UID: \"5bca996e-4e3a-4ec6-bd2f-0a5860c9d632\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7t9rj" Jul 14 17:32:33.435229 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.433882 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c3d00b23-d0d5-4d18-a722-464d7b114745-env-overrides\") pod \"ovnkube-node-2sbww\" (UID: \"c3d00b23-d0d5-4d18-a722-464d7b114745\") " pod="openshift-ovn-kubernetes/ovnkube-node-2sbww" Jul 14 17:32:33.435229 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.433903 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/dadd3729-1c46-48d9-92c8-bae9cdbbd1c1-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-fjrz4\" (UID: \"dadd3729-1c46-48d9-92c8-bae9cdbbd1c1\") " pod="openshift-multus/multus-additional-cni-plugins-fjrz4" Jul 14 17:32:33.435229 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.433923 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/48bfbea2-4a10-4454-a93d-72c4c8135d79-multus-cni-dir\") pod \"multus-9dcd9\" (UID: \"48bfbea2-4a10-4454-a93d-72c4c8135d79\") " pod="openshift-multus/multus-9dcd9" Jul 14 17:32:33.435229 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.433941 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/48bfbea2-4a10-4454-a93d-72c4c8135d79-host-var-lib-kubelet\") pod \"multus-9dcd9\" (UID: \"48bfbea2-4a10-4454-a93d-72c4c8135d79\") " pod="openshift-multus/multus-9dcd9" Jul 14 17:32:33.435229 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.433958 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/56a1f173-7ec8-4f09-94ce-ee3bde8e4c17-etc-tuned\") pod \"tuned-9gs5p\" (UID: \"56a1f173-7ec8-4f09-94ce-ee3bde8e4c17\") " pod="openshift-cluster-node-tuning-operator/tuned-9gs5p" Jul 14 17:32:33.435229 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.433980 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hznl9\" (UniqueName: \"kubernetes.io/projected/17370624-81e7-42ff-b916-e3f323b3e0a5-kube-api-access-hznl9\") pod \"node-ca-thnpp\" (UID: \"17370624-81e7-42ff-b916-e3f323b3e0a5\") " pod="openshift-image-registry/node-ca-thnpp" Jul 14 17:32:33.435229 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.434005 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/dadd3729-1c46-48d9-92c8-bae9cdbbd1c1-tuning-conf-dir\") pod \"multus-additional-cni-plugins-fjrz4\" (UID: \"dadd3729-1c46-48d9-92c8-bae9cdbbd1c1\") " pod="openshift-multus/multus-additional-cni-plugins-fjrz4" Jul 14 17:32:33.435229 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.434027 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/56a1f173-7ec8-4f09-94ce-ee3bde8e4c17-etc-kubernetes\") pod \"tuned-9gs5p\" (UID: \"56a1f173-7ec8-4f09-94ce-ee3bde8e4c17\") " pod="openshift-cluster-node-tuning-operator/tuned-9gs5p" Jul 14 17:32:33.436017 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.434085 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/48bfbea2-4a10-4454-a93d-72c4c8135d79-hostroot\") pod \"multus-9dcd9\" (UID: \"48bfbea2-4a10-4454-a93d-72c4c8135d79\") " pod="openshift-multus/multus-9dcd9" Jul 14 17:32:33.436017 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.434108 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qz7zj\" (UniqueName: \"kubernetes.io/projected/8321d9c3-c963-45a9-8a94-ef9911a8d33f-kube-api-access-qz7zj\") pod \"network-metrics-daemon-h8t96\" (UID: \"8321d9c3-c963-45a9-8a94-ef9911a8d33f\") " pod="openshift-multus/network-metrics-daemon-h8t96" Jul 14 17:32:33.436017 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.434156 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qqfb\" (UniqueName: \"kubernetes.io/projected/5b2d2388-62da-4462-af63-9230c6c43715-kube-api-access-5qqfb\") pod \"network-check-target-9m2f6\" (UID: \"5b2d2388-62da-4462-af63-9230c6c43715\") " pod="openshift-network-diagnostics/network-check-target-9m2f6" Jul 14 17:32:33.436017 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.434180 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/48bfbea2-4a10-4454-a93d-72c4c8135d79-cnibin\") pod \"multus-9dcd9\" (UID: \"48bfbea2-4a10-4454-a93d-72c4c8135d79\") " pod="openshift-multus/multus-9dcd9" Jul 14 17:32:33.436017 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.434203 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/dadd3729-1c46-48d9-92c8-bae9cdbbd1c1-tuning-conf-dir\") pod \"multus-additional-cni-plugins-fjrz4\" (UID: \"dadd3729-1c46-48d9-92c8-bae9cdbbd1c1\") " pod="openshift-multus/multus-additional-cni-plugins-fjrz4" Jul 14 17:32:33.436017 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.434221 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/48bfbea2-4a10-4454-a93d-72c4c8135d79-cni-binary-copy\") pod \"multus-9dcd9\" (UID: \"48bfbea2-4a10-4454-a93d-72c4c8135d79\") " pod="openshift-multus/multus-9dcd9" Jul 14 17:32:33.436017 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.434274 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/17370624-81e7-42ff-b916-e3f323b3e0a5-host\") pod \"node-ca-thnpp\" (UID: \"17370624-81e7-42ff-b916-e3f323b3e0a5\") " pod="openshift-image-registry/node-ca-thnpp" Jul 14 17:32:33.436017 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.434298 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c3d00b23-d0d5-4d18-a722-464d7b114745-env-overrides\") pod \"ovnkube-node-2sbww\" (UID: \"c3d00b23-d0d5-4d18-a722-464d7b114745\") " pod="openshift-ovn-kubernetes/ovnkube-node-2sbww" Jul 14 17:32:33.436017 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.434298 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c3d00b23-d0d5-4d18-a722-464d7b114745-run-ovn\") pod \"ovnkube-node-2sbww\" (UID: \"c3d00b23-d0d5-4d18-a722-464d7b114745\") " pod="openshift-ovn-kubernetes/ovnkube-node-2sbww" Jul 14 17:32:33.436017 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.434344 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-62qwb\" (UniqueName: \"kubernetes.io/projected/c3d00b23-d0d5-4d18-a722-464d7b114745-kube-api-access-62qwb\") pod \"ovnkube-node-2sbww\" (UID: \"c3d00b23-d0d5-4d18-a722-464d7b114745\") " pod="openshift-ovn-kubernetes/ovnkube-node-2sbww" Jul 14 17:32:33.436017 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.434369 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/dadd3729-1c46-48d9-92c8-bae9cdbbd1c1-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-fjrz4\" (UID: \"dadd3729-1c46-48d9-92c8-bae9cdbbd1c1\") " pod="openshift-multus/multus-additional-cni-plugins-fjrz4" Jul 14 17:32:33.436017 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.434393 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5nllr\" (UniqueName: \"kubernetes.io/projected/91ee1d15-3741-4cf5-821e-d3b216b7e246-kube-api-access-5nllr\") pod \"node-resolver-7fdn8\" (UID: \"91ee1d15-3741-4cf5-821e-d3b216b7e246\") " pod="openshift-dns/node-resolver-7fdn8" Jul 14 17:32:33.436017 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.434454 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/56a1f173-7ec8-4f09-94ce-ee3bde8e4c17-etc-sysconfig\") pod \"tuned-9gs5p\" (UID: \"56a1f173-7ec8-4f09-94ce-ee3bde8e4c17\") " pod="openshift-cluster-node-tuning-operator/tuned-9gs5p" Jul 14 17:32:33.436017 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.434467 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/dadd3729-1c46-48d9-92c8-bae9cdbbd1c1-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-fjrz4\" (UID: \"dadd3729-1c46-48d9-92c8-bae9cdbbd1c1\") " pod="openshift-multus/multus-additional-cni-plugins-fjrz4" Jul 14 17:32:33.436017 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.434483 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c3d00b23-d0d5-4d18-a722-464d7b114745-run-ovn\") pod \"ovnkube-node-2sbww\" (UID: \"c3d00b23-d0d5-4d18-a722-464d7b114745\") " pod="openshift-ovn-kubernetes/ovnkube-node-2sbww" Jul 14 17:32:33.436017 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.434477 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/56a1f173-7ec8-4f09-94ce-ee3bde8e4c17-tmp\") pod \"tuned-9gs5p\" (UID: \"56a1f173-7ec8-4f09-94ce-ee3bde8e4c17\") " pod="openshift-cluster-node-tuning-operator/tuned-9gs5p" Jul 14 17:32:33.436017 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.434531 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ac89d28e-0c57-4be5-8c3a-f0482ce0ba6f-original-pull-secret\") pod \"global-pull-secret-syncer-kv2bk\" (UID: \"ac89d28e-0c57-4be5-8c3a-f0482ce0ba6f\") " pod="kube-system/global-pull-secret-syncer-kv2bk" Jul 14 17:32:33.436814 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.434575 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/dadd3729-1c46-48d9-92c8-bae9cdbbd1c1-os-release\") pod \"multus-additional-cni-plugins-fjrz4\" (UID: \"dadd3729-1c46-48d9-92c8-bae9cdbbd1c1\") " pod="openshift-multus/multus-additional-cni-plugins-fjrz4" Jul 14 17:32:33.436814 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.434626 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/dadd3729-1c46-48d9-92c8-bae9cdbbd1c1-os-release\") pod \"multus-additional-cni-plugins-fjrz4\" (UID: \"dadd3729-1c46-48d9-92c8-bae9cdbbd1c1\") " pod="openshift-multus/multus-additional-cni-plugins-fjrz4" Jul 14 17:32:33.436814 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.434664 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/dadd3729-1c46-48d9-92c8-bae9cdbbd1c1-cni-binary-copy\") pod \"multus-additional-cni-plugins-fjrz4\" (UID: \"dadd3729-1c46-48d9-92c8-bae9cdbbd1c1\") " pod="openshift-multus/multus-additional-cni-plugins-fjrz4" Jul 14 17:32:33.436814 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:32:33.434727 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 17:32:33.436814 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.434773 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8321d9c3-c963-45a9-8a94-ef9911a8d33f-metrics-certs\") pod \"network-metrics-daemon-h8t96\" (UID: \"8321d9c3-c963-45a9-8a94-ef9911a8d33f\") " pod="openshift-multus/network-metrics-daemon-h8t96" Jul 14 17:32:33.436814 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.434805 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/48bfbea2-4a10-4454-a93d-72c4c8135d79-etc-kubernetes\") pod \"multus-9dcd9\" (UID: \"48bfbea2-4a10-4454-a93d-72c4c8135d79\") " pod="openshift-multus/multus-9dcd9" Jul 14 17:32:33.436814 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:32:33.434934 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac89d28e-0c57-4be5-8c3a-f0482ce0ba6f-original-pull-secret podName:ac89d28e-0c57-4be5-8c3a-f0482ce0ba6f nodeName:}" failed. No retries permitted until 2026-07-14 17:32:33.934913096 +0000 UTC m=+3.011370630 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ac89d28e-0c57-4be5-8c3a-f0482ce0ba6f-original-pull-secret") pod "global-pull-secret-syncer-kv2bk" (UID: "ac89d28e-0c57-4be5-8c3a-f0482ce0ba6f") : object "kube-system"/"original-pull-secret" not registered Jul 14 17:32:33.436814 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.434939 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/dadd3729-1c46-48d9-92c8-bae9cdbbd1c1-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-fjrz4\" (UID: \"dadd3729-1c46-48d9-92c8-bae9cdbbd1c1\") " pod="openshift-multus/multus-additional-cni-plugins-fjrz4" Jul 14 17:32:33.436814 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:32:33.435015 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 17:32:33.436814 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:32:33.435080 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8321d9c3-c963-45a9-8a94-ef9911a8d33f-metrics-certs podName:8321d9c3-c963-45a9-8a94-ef9911a8d33f nodeName:}" failed. No retries permitted until 2026-07-14 17:32:33.935065304 +0000 UTC m=+3.011522826 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8321d9c3-c963-45a9-8a94-ef9911a8d33f-metrics-certs") pod "network-metrics-daemon-h8t96" (UID: "8321d9c3-c963-45a9-8a94-ef9911a8d33f") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 17:32:33.436814 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.435870 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/dadd3729-1c46-48d9-92c8-bae9cdbbd1c1-cni-binary-copy\") pod \"multus-additional-cni-plugins-fjrz4\" (UID: \"dadd3729-1c46-48d9-92c8-bae9cdbbd1c1\") " pod="openshift-multus/multus-additional-cni-plugins-fjrz4" Jul 14 17:32:33.440333 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:32:33.440313 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 17:32:33.440433 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:32:33.440336 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 17:32:33.440433 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:32:33.440349 2562 projected.go:196] Error preparing data for projected volume kube-api-access-5qqfb for pod openshift-network-diagnostics/network-check-target-9m2f6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 17:32:33.440533 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:32:33.440479 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5b2d2388-62da-4462-af63-9230c6c43715-kube-api-access-5qqfb podName:5b2d2388-62da-4462-af63-9230c6c43715 nodeName:}" failed. No retries permitted until 2026-07-14 17:32:33.940446839 +0000 UTC m=+3.016904379 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-5qqfb" (UniqueName: "kubernetes.io/projected/5b2d2388-62da-4462-af63-9230c6c43715-kube-api-access-5qqfb") pod "network-check-target-9m2f6" (UID: "5b2d2388-62da-4462-af63-9230c6c43715") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 17:32:33.440533 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.440479 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c3d00b23-d0d5-4d18-a722-464d7b114745-ovn-node-metrics-cert\") pod \"ovnkube-node-2sbww\" (UID: \"c3d00b23-d0d5-4d18-a722-464d7b114745\") " pod="openshift-ovn-kubernetes/ovnkube-node-2sbww" Jul 14 17:32:33.441581 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.441549 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z57kn\" (UniqueName: \"kubernetes.io/projected/2b670bba-aee6-407b-8f4e-e900528f54e5-kube-api-access-z57kn\") pod \"iptables-alerter-prnxg\" (UID: \"2b670bba-aee6-407b-8f4e-e900528f54e5\") " pod="openshift-network-operator/iptables-alerter-prnxg" Jul 14 17:32:33.441668 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.441601 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pz6r7\" (UniqueName: \"kubernetes.io/projected/dadd3729-1c46-48d9-92c8-bae9cdbbd1c1-kube-api-access-pz6r7\") pod \"multus-additional-cni-plugins-fjrz4\" (UID: \"dadd3729-1c46-48d9-92c8-bae9cdbbd1c1\") " pod="openshift-multus/multus-additional-cni-plugins-fjrz4" Jul 14 17:32:33.442457 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.442425 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qz7zj\" (UniqueName: \"kubernetes.io/projected/8321d9c3-c963-45a9-8a94-ef9911a8d33f-kube-api-access-qz7zj\") pod \"network-metrics-daemon-h8t96\" (UID: \"8321d9c3-c963-45a9-8a94-ef9911a8d33f\") " pod="openshift-multus/network-metrics-daemon-h8t96" Jul 14 17:32:33.447610 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.447588 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-62qwb\" (UniqueName: \"kubernetes.io/projected/c3d00b23-d0d5-4d18-a722-464d7b114745-kube-api-access-62qwb\") pod \"ovnkube-node-2sbww\" (UID: \"c3d00b23-d0d5-4d18-a722-464d7b114745\") " pod="openshift-ovn-kubernetes/ovnkube-node-2sbww" Jul 14 17:32:33.517919 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.517867 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-209.ec2.internal" event={"ID":"8b1a7c81fd72faef90d5a3084bb532e1","Type":"ContainerStarted","Data":"09b62f1b7c67b3a923e34a41a65bd00195a67505ae7251c26088bb24e5949a68"} Jul 14 17:32:33.519036 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.519008 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-209.ec2.internal" event={"ID":"0f6dec0287e6513120043e23e8c805f3","Type":"ContainerStarted","Data":"aabe73c4375ac99baa92a65b6cafb01e231700c6cfc7edc56491da6614322756"} Jul 14 17:32:33.530640 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.530623 2562 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 14 17:32:33.535622 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.535603 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/48bfbea2-4a10-4454-a93d-72c4c8135d79-hostroot\") pod \"multus-9dcd9\" (UID: \"48bfbea2-4a10-4454-a93d-72c4c8135d79\") " pod="openshift-multus/multus-9dcd9" Jul 14 17:32:33.535726 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.535639 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/48bfbea2-4a10-4454-a93d-72c4c8135d79-cnibin\") pod \"multus-9dcd9\" (UID: \"48bfbea2-4a10-4454-a93d-72c4c8135d79\") " pod="openshift-multus/multus-9dcd9" Jul 14 17:32:33.535726 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.535654 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/48bfbea2-4a10-4454-a93d-72c4c8135d79-cni-binary-copy\") pod \"multus-9dcd9\" (UID: \"48bfbea2-4a10-4454-a93d-72c4c8135d79\") " pod="openshift-multus/multus-9dcd9" Jul 14 17:32:33.535726 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.535671 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/17370624-81e7-42ff-b916-e3f323b3e0a5-host\") pod \"node-ca-thnpp\" (UID: \"17370624-81e7-42ff-b916-e3f323b3e0a5\") " pod="openshift-image-registry/node-ca-thnpp" Jul 14 17:32:33.535726 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.535697 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5nllr\" (UniqueName: \"kubernetes.io/projected/91ee1d15-3741-4cf5-821e-d3b216b7e246-kube-api-access-5nllr\") pod \"node-resolver-7fdn8\" (UID: \"91ee1d15-3741-4cf5-821e-d3b216b7e246\") " pod="openshift-dns/node-resolver-7fdn8" Jul 14 17:32:33.535726 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.535713 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/48bfbea2-4a10-4454-a93d-72c4c8135d79-hostroot\") pod \"multus-9dcd9\" (UID: \"48bfbea2-4a10-4454-a93d-72c4c8135d79\") " pod="openshift-multus/multus-9dcd9" Jul 14 17:32:33.535959 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.535735 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/48bfbea2-4a10-4454-a93d-72c4c8135d79-cnibin\") pod \"multus-9dcd9\" (UID: \"48bfbea2-4a10-4454-a93d-72c4c8135d79\") " pod="openshift-multus/multus-9dcd9" Jul 14 17:32:33.535959 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.535719 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/56a1f173-7ec8-4f09-94ce-ee3bde8e4c17-etc-sysconfig\") pod \"tuned-9gs5p\" (UID: \"56a1f173-7ec8-4f09-94ce-ee3bde8e4c17\") " pod="openshift-cluster-node-tuning-operator/tuned-9gs5p" Jul 14 17:32:33.535959 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.535765 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/56a1f173-7ec8-4f09-94ce-ee3bde8e4c17-etc-sysconfig\") pod \"tuned-9gs5p\" (UID: \"56a1f173-7ec8-4f09-94ce-ee3bde8e4c17\") " pod="openshift-cluster-node-tuning-operator/tuned-9gs5p" Jul 14 17:32:33.535959 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.535777 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/56a1f173-7ec8-4f09-94ce-ee3bde8e4c17-tmp\") pod \"tuned-9gs5p\" (UID: \"56a1f173-7ec8-4f09-94ce-ee3bde8e4c17\") " pod="openshift-cluster-node-tuning-operator/tuned-9gs5p" Jul 14 17:32:33.535959 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.535789 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/17370624-81e7-42ff-b916-e3f323b3e0a5-host\") pod \"node-ca-thnpp\" (UID: \"17370624-81e7-42ff-b916-e3f323b3e0a5\") " pod="openshift-image-registry/node-ca-thnpp" Jul 14 17:32:33.535959 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.535822 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/48bfbea2-4a10-4454-a93d-72c4c8135d79-etc-kubernetes\") pod \"multus-9dcd9\" (UID: \"48bfbea2-4a10-4454-a93d-72c4c8135d79\") " pod="openshift-multus/multus-9dcd9" Jul 14 17:32:33.535959 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.535849 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/650b148f-78d8-456f-bc24-1fbe4a75a24a-konnectivity-ca\") pod \"konnectivity-agent-hxbnp\" (UID: \"650b148f-78d8-456f-bc24-1fbe4a75a24a\") " pod="kube-system/konnectivity-agent-hxbnp" Jul 14 17:32:33.535959 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.535869 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/56a1f173-7ec8-4f09-94ce-ee3bde8e4c17-run\") pod \"tuned-9gs5p\" (UID: \"56a1f173-7ec8-4f09-94ce-ee3bde8e4c17\") " pod="openshift-cluster-node-tuning-operator/tuned-9gs5p" Jul 14 17:32:33.535959 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.535889 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/56a1f173-7ec8-4f09-94ce-ee3bde8e4c17-host\") pod \"tuned-9gs5p\" (UID: \"56a1f173-7ec8-4f09-94ce-ee3bde8e4c17\") " pod="openshift-cluster-node-tuning-operator/tuned-9gs5p" Jul 14 17:32:33.535959 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.535911 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/5bca996e-4e3a-4ec6-bd2f-0a5860c9d632-sys-fs\") pod \"aws-ebs-csi-driver-node-7t9rj\" (UID: \"5bca996e-4e3a-4ec6-bd2f-0a5860c9d632\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7t9rj" Jul 14 17:32:33.535959 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.535934 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/56a1f173-7ec8-4f09-94ce-ee3bde8e4c17-lib-modules\") pod \"tuned-9gs5p\" (UID: \"56a1f173-7ec8-4f09-94ce-ee3bde8e4c17\") " pod="openshift-cluster-node-tuning-operator/tuned-9gs5p" Jul 14 17:32:33.535959 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.535961 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m9kld\" (UniqueName: \"kubernetes.io/projected/48bfbea2-4a10-4454-a93d-72c4c8135d79-kube-api-access-m9kld\") pod \"multus-9dcd9\" (UID: \"48bfbea2-4a10-4454-a93d-72c4c8135d79\") " pod="openshift-multus/multus-9dcd9" Jul 14 17:32:33.536544 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.535986 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/650b148f-78d8-456f-bc24-1fbe4a75a24a-agent-certs\") pod \"konnectivity-agent-hxbnp\" (UID: \"650b148f-78d8-456f-bc24-1fbe4a75a24a\") " pod="kube-system/konnectivity-agent-hxbnp" Jul 14 17:32:33.536544 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.535999 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/48bfbea2-4a10-4454-a93d-72c4c8135d79-etc-kubernetes\") pod \"multus-9dcd9\" (UID: \"48bfbea2-4a10-4454-a93d-72c4c8135d79\") " pod="openshift-multus/multus-9dcd9" Jul 14 17:32:33.536544 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.536008 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/56a1f173-7ec8-4f09-94ce-ee3bde8e4c17-sys\") pod \"tuned-9gs5p\" (UID: \"56a1f173-7ec8-4f09-94ce-ee3bde8e4c17\") " pod="openshift-cluster-node-tuning-operator/tuned-9gs5p" Jul 14 17:32:33.536544 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.536036 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/48bfbea2-4a10-4454-a93d-72c4c8135d79-system-cni-dir\") pod \"multus-9dcd9\" (UID: \"48bfbea2-4a10-4454-a93d-72c4c8135d79\") " pod="openshift-multus/multus-9dcd9" Jul 14 17:32:33.536544 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.536052 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/5bca996e-4e3a-4ec6-bd2f-0a5860c9d632-sys-fs\") pod \"aws-ebs-csi-driver-node-7t9rj\" (UID: \"5bca996e-4e3a-4ec6-bd2f-0a5860c9d632\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7t9rj" Jul 14 17:32:33.536544 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.536060 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/48bfbea2-4a10-4454-a93d-72c4c8135d79-multus-daemon-config\") pod \"multus-9dcd9\" (UID: \"48bfbea2-4a10-4454-a93d-72c4c8135d79\") " pod="openshift-multus/multus-9dcd9" Jul 14 17:32:33.536544 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.536080 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/5bca996e-4e3a-4ec6-bd2f-0a5860c9d632-socket-dir\") pod \"aws-ebs-csi-driver-node-7t9rj\" (UID: \"5bca996e-4e3a-4ec6-bd2f-0a5860c9d632\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7t9rj" Jul 14 17:32:33.536544 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.536097 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/56a1f173-7ec8-4f09-94ce-ee3bde8e4c17-run\") pod \"tuned-9gs5p\" (UID: \"56a1f173-7ec8-4f09-94ce-ee3bde8e4c17\") " pod="openshift-cluster-node-tuning-operator/tuned-9gs5p" Jul 14 17:32:33.536544 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.536109 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b7fkv\" (UniqueName: \"kubernetes.io/projected/5bca996e-4e3a-4ec6-bd2f-0a5860c9d632-kube-api-access-b7fkv\") pod \"aws-ebs-csi-driver-node-7t9rj\" (UID: \"5bca996e-4e3a-4ec6-bd2f-0a5860c9d632\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7t9rj" Jul 14 17:32:33.536544 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.536137 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/48bfbea2-4a10-4454-a93d-72c4c8135d79-multus-conf-dir\") pod \"multus-9dcd9\" (UID: \"48bfbea2-4a10-4454-a93d-72c4c8135d79\") " pod="openshift-multus/multus-9dcd9" Jul 14 17:32:33.536544 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.536140 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/56a1f173-7ec8-4f09-94ce-ee3bde8e4c17-host\") pod \"tuned-9gs5p\" (UID: \"56a1f173-7ec8-4f09-94ce-ee3bde8e4c17\") " pod="openshift-cluster-node-tuning-operator/tuned-9gs5p" Jul 14 17:32:33.536544 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.536161 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/48bfbea2-4a10-4454-a93d-72c4c8135d79-host-var-lib-cni-bin\") pod \"multus-9dcd9\" (UID: \"48bfbea2-4a10-4454-a93d-72c4c8135d79\") " pod="openshift-multus/multus-9dcd9" Jul 14 17:32:33.536544 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.536207 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/48bfbea2-4a10-4454-a93d-72c4c8135d79-host-var-lib-cni-multus\") pod \"multus-9dcd9\" (UID: \"48bfbea2-4a10-4454-a93d-72c4c8135d79\") " pod="openshift-multus/multus-9dcd9" Jul 14 17:32:33.536544 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.536231 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/5bca996e-4e3a-4ec6-bd2f-0a5860c9d632-device-dir\") pod \"aws-ebs-csi-driver-node-7t9rj\" (UID: \"5bca996e-4e3a-4ec6-bd2f-0a5860c9d632\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7t9rj" Jul 14 17:32:33.536544 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.536252 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/48bfbea2-4a10-4454-a93d-72c4c8135d79-os-release\") pod \"multus-9dcd9\" (UID: \"48bfbea2-4a10-4454-a93d-72c4c8135d79\") " pod="openshift-multus/multus-9dcd9" Jul 14 17:32:33.536544 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.536265 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/48bfbea2-4a10-4454-a93d-72c4c8135d79-cni-binary-copy\") pod \"multus-9dcd9\" (UID: \"48bfbea2-4a10-4454-a93d-72c4c8135d79\") " pod="openshift-multus/multus-9dcd9" Jul 14 17:32:33.536544 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.536273 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/56a1f173-7ec8-4f09-94ce-ee3bde8e4c17-var-lib-kubelet\") pod \"tuned-9gs5p\" (UID: \"56a1f173-7ec8-4f09-94ce-ee3bde8e4c17\") " pod="openshift-cluster-node-tuning-operator/tuned-9gs5p" Jul 14 17:32:33.536544 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.536323 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/56a1f173-7ec8-4f09-94ce-ee3bde8e4c17-var-lib-kubelet\") pod \"tuned-9gs5p\" (UID: \"56a1f173-7ec8-4f09-94ce-ee3bde8e4c17\") " pod="openshift-cluster-node-tuning-operator/tuned-9gs5p" Jul 14 17:32:33.537380 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.536325 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/17370624-81e7-42ff-b916-e3f323b3e0a5-serviceca\") pod \"node-ca-thnpp\" (UID: \"17370624-81e7-42ff-b916-e3f323b3e0a5\") " pod="openshift-image-registry/node-ca-thnpp" Jul 14 17:32:33.537380 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.536340 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/56a1f173-7ec8-4f09-94ce-ee3bde8e4c17-lib-modules\") pod \"tuned-9gs5p\" (UID: \"56a1f173-7ec8-4f09-94ce-ee3bde8e4c17\") " pod="openshift-cluster-node-tuning-operator/tuned-9gs5p" Jul 14 17:32:33.537380 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.536355 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/56a1f173-7ec8-4f09-94ce-ee3bde8e4c17-etc-sysctl-d\") pod \"tuned-9gs5p\" (UID: \"56a1f173-7ec8-4f09-94ce-ee3bde8e4c17\") " pod="openshift-cluster-node-tuning-operator/tuned-9gs5p" Jul 14 17:32:33.537380 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.536371 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/650b148f-78d8-456f-bc24-1fbe4a75a24a-konnectivity-ca\") pod \"konnectivity-agent-hxbnp\" (UID: \"650b148f-78d8-456f-bc24-1fbe4a75a24a\") " pod="kube-system/konnectivity-agent-hxbnp" Jul 14 17:32:33.537380 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.536399 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/56a1f173-7ec8-4f09-94ce-ee3bde8e4c17-sys\") pod \"tuned-9gs5p\" (UID: \"56a1f173-7ec8-4f09-94ce-ee3bde8e4c17\") " pod="openshift-cluster-node-tuning-operator/tuned-9gs5p" Jul 14 17:32:33.537380 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.536428 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/48bfbea2-4a10-4454-a93d-72c4c8135d79-system-cni-dir\") pod \"multus-9dcd9\" (UID: \"48bfbea2-4a10-4454-a93d-72c4c8135d79\") " pod="openshift-multus/multus-9dcd9" Jul 14 17:32:33.537380 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.536459 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/48bfbea2-4a10-4454-a93d-72c4c8135d79-host-var-lib-cni-bin\") pod \"multus-9dcd9\" (UID: \"48bfbea2-4a10-4454-a93d-72c4c8135d79\") " pod="openshift-multus/multus-9dcd9" Jul 14 17:32:33.537380 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.536427 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/91ee1d15-3741-4cf5-821e-d3b216b7e246-hosts-file\") pod \"node-resolver-7fdn8\" (UID: \"91ee1d15-3741-4cf5-821e-d3b216b7e246\") " pod="openshift-dns/node-resolver-7fdn8" Jul 14 17:32:33.537380 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.536524 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/5bca996e-4e3a-4ec6-bd2f-0a5860c9d632-socket-dir\") pod \"aws-ebs-csi-driver-node-7t9rj\" (UID: \"5bca996e-4e3a-4ec6-bd2f-0a5860c9d632\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7t9rj" Jul 14 17:32:33.537380 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.536589 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/48bfbea2-4a10-4454-a93d-72c4c8135d79-multus-socket-dir-parent\") pod \"multus-9dcd9\" (UID: \"48bfbea2-4a10-4454-a93d-72c4c8135d79\") " pod="openshift-multus/multus-9dcd9" Jul 14 17:32:33.537380 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.536612 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/48bfbea2-4a10-4454-a93d-72c4c8135d79-host-run-k8s-cni-cncf-io\") pod \"multus-9dcd9\" (UID: \"48bfbea2-4a10-4454-a93d-72c4c8135d79\") " pod="openshift-multus/multus-9dcd9" Jul 14 17:32:33.537380 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.536628 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/48bfbea2-4a10-4454-a93d-72c4c8135d79-multus-daemon-config\") pod \"multus-9dcd9\" (UID: \"48bfbea2-4a10-4454-a93d-72c4c8135d79\") " pod="openshift-multus/multus-9dcd9" Jul 14 17:32:33.537380 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.536637 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/56a1f173-7ec8-4f09-94ce-ee3bde8e4c17-etc-modprobe-d\") pod \"tuned-9gs5p\" (UID: \"56a1f173-7ec8-4f09-94ce-ee3bde8e4c17\") " pod="openshift-cluster-node-tuning-operator/tuned-9gs5p" Jul 14 17:32:33.537380 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.536661 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/5bca996e-4e3a-4ec6-bd2f-0a5860c9d632-registration-dir\") pod \"aws-ebs-csi-driver-node-7t9rj\" (UID: \"5bca996e-4e3a-4ec6-bd2f-0a5860c9d632\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7t9rj" Jul 14 17:32:33.537380 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.536690 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/48bfbea2-4a10-4454-a93d-72c4c8135d79-host-run-netns\") pod \"multus-9dcd9\" (UID: \"48bfbea2-4a10-4454-a93d-72c4c8135d79\") " pod="openshift-multus/multus-9dcd9" Jul 14 17:32:33.537380 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.536705 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/48bfbea2-4a10-4454-a93d-72c4c8135d79-multus-conf-dir\") pod \"multus-9dcd9\" (UID: \"48bfbea2-4a10-4454-a93d-72c4c8135d79\") " pod="openshift-multus/multus-9dcd9" Jul 14 17:32:33.537380 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.536714 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/48bfbea2-4a10-4454-a93d-72c4c8135d79-host-run-multus-certs\") pod \"multus-9dcd9\" (UID: \"48bfbea2-4a10-4454-a93d-72c4c8135d79\") " pod="openshift-multus/multus-9dcd9" Jul 14 17:32:33.537380 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.536730 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/17370624-81e7-42ff-b916-e3f323b3e0a5-serviceca\") pod \"node-ca-thnpp\" (UID: \"17370624-81e7-42ff-b916-e3f323b3e0a5\") " pod="openshift-image-registry/node-ca-thnpp" Jul 14 17:32:33.538012 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.536742 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/48bfbea2-4a10-4454-a93d-72c4c8135d79-host-run-k8s-cni-cncf-io\") pod \"multus-9dcd9\" (UID: \"48bfbea2-4a10-4454-a93d-72c4c8135d79\") " pod="openshift-multus/multus-9dcd9" Jul 14 17:32:33.538012 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.536740 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/56a1f173-7ec8-4f09-94ce-ee3bde8e4c17-etc-sysctl-conf\") pod \"tuned-9gs5p\" (UID: \"56a1f173-7ec8-4f09-94ce-ee3bde8e4c17\") " pod="openshift-cluster-node-tuning-operator/tuned-9gs5p" Jul 14 17:32:33.538012 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.536763 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/56a1f173-7ec8-4f09-94ce-ee3bde8e4c17-etc-sysctl-d\") pod \"tuned-9gs5p\" (UID: \"56a1f173-7ec8-4f09-94ce-ee3bde8e4c17\") " pod="openshift-cluster-node-tuning-operator/tuned-9gs5p" Jul 14 17:32:33.538012 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.536824 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/91ee1d15-3741-4cf5-821e-d3b216b7e246-tmp-dir\") pod \"node-resolver-7fdn8\" (UID: \"91ee1d15-3741-4cf5-821e-d3b216b7e246\") " pod="openshift-dns/node-resolver-7fdn8" Jul 14 17:32:33.538012 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.536838 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/56a1f173-7ec8-4f09-94ce-ee3bde8e4c17-etc-sysctl-conf\") pod \"tuned-9gs5p\" (UID: \"56a1f173-7ec8-4f09-94ce-ee3bde8e4c17\") " pod="openshift-cluster-node-tuning-operator/tuned-9gs5p" Jul 14 17:32:33.538012 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.536842 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/56a1f173-7ec8-4f09-94ce-ee3bde8e4c17-etc-modprobe-d\") pod \"tuned-9gs5p\" (UID: \"56a1f173-7ec8-4f09-94ce-ee3bde8e4c17\") " pod="openshift-cluster-node-tuning-operator/tuned-9gs5p" Jul 14 17:32:33.538012 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.536859 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/56a1f173-7ec8-4f09-94ce-ee3bde8e4c17-etc-systemd\") pod \"tuned-9gs5p\" (UID: \"56a1f173-7ec8-4f09-94ce-ee3bde8e4c17\") " pod="openshift-cluster-node-tuning-operator/tuned-9gs5p" Jul 14 17:32:33.538012 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.536885 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lqqvr\" (UniqueName: \"kubernetes.io/projected/56a1f173-7ec8-4f09-94ce-ee3bde8e4c17-kube-api-access-lqqvr\") pod \"tuned-9gs5p\" (UID: \"56a1f173-7ec8-4f09-94ce-ee3bde8e4c17\") " pod="openshift-cluster-node-tuning-operator/tuned-9gs5p" Jul 14 17:32:33.538012 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.536891 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/48bfbea2-4a10-4454-a93d-72c4c8135d79-host-var-lib-cni-multus\") pod \"multus-9dcd9\" (UID: \"48bfbea2-4a10-4454-a93d-72c4c8135d79\") " pod="openshift-multus/multus-9dcd9" Jul 14 17:32:33.538012 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.536903 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/5bca996e-4e3a-4ec6-bd2f-0a5860c9d632-registration-dir\") pod \"aws-ebs-csi-driver-node-7t9rj\" (UID: \"5bca996e-4e3a-4ec6-bd2f-0a5860c9d632\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7t9rj" Jul 14 17:32:33.538012 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.536913 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/5bca996e-4e3a-4ec6-bd2f-0a5860c9d632-kubelet-dir\") pod \"aws-ebs-csi-driver-node-7t9rj\" (UID: \"5bca996e-4e3a-4ec6-bd2f-0a5860c9d632\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7t9rj" Jul 14 17:32:33.538012 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.536859 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/5bca996e-4e3a-4ec6-bd2f-0a5860c9d632-device-dir\") pod \"aws-ebs-csi-driver-node-7t9rj\" (UID: \"5bca996e-4e3a-4ec6-bd2f-0a5860c9d632\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7t9rj" Jul 14 17:32:33.538012 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.536926 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/48bfbea2-4a10-4454-a93d-72c4c8135d79-host-run-multus-certs\") pod \"multus-9dcd9\" (UID: \"48bfbea2-4a10-4454-a93d-72c4c8135d79\") " pod="openshift-multus/multus-9dcd9" Jul 14 17:32:33.538012 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.536964 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/5bca996e-4e3a-4ec6-bd2f-0a5860c9d632-etc-selinux\") pod \"aws-ebs-csi-driver-node-7t9rj\" (UID: \"5bca996e-4e3a-4ec6-bd2f-0a5860c9d632\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7t9rj" Jul 14 17:32:33.538012 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.536985 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/48bfbea2-4a10-4454-a93d-72c4c8135d79-os-release\") pod \"multus-9dcd9\" (UID: \"48bfbea2-4a10-4454-a93d-72c4c8135d79\") " pod="openshift-multus/multus-9dcd9" Jul 14 17:32:33.538012 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.536994 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/48bfbea2-4a10-4454-a93d-72c4c8135d79-multus-cni-dir\") pod \"multus-9dcd9\" (UID: \"48bfbea2-4a10-4454-a93d-72c4c8135d79\") " pod="openshift-multus/multus-9dcd9" Jul 14 17:32:33.538012 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.537000 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/56a1f173-7ec8-4f09-94ce-ee3bde8e4c17-etc-systemd\") pod \"tuned-9gs5p\" (UID: \"56a1f173-7ec8-4f09-94ce-ee3bde8e4c17\") " pod="openshift-cluster-node-tuning-operator/tuned-9gs5p" Jul 14 17:32:33.538012 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.536967 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/48bfbea2-4a10-4454-a93d-72c4c8135d79-host-run-netns\") pod \"multus-9dcd9\" (UID: \"48bfbea2-4a10-4454-a93d-72c4c8135d79\") " pod="openshift-multus/multus-9dcd9" Jul 14 17:32:33.538709 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.537011 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/91ee1d15-3741-4cf5-821e-d3b216b7e246-tmp-dir\") pod \"node-resolver-7fdn8\" (UID: \"91ee1d15-3741-4cf5-821e-d3b216b7e246\") " pod="openshift-dns/node-resolver-7fdn8" Jul 14 17:32:33.538709 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.537019 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/48bfbea2-4a10-4454-a93d-72c4c8135d79-host-var-lib-kubelet\") pod \"multus-9dcd9\" (UID: \"48bfbea2-4a10-4454-a93d-72c4c8135d79\") " pod="openshift-multus/multus-9dcd9" Jul 14 17:32:33.538709 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.537043 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/56a1f173-7ec8-4f09-94ce-ee3bde8e4c17-etc-tuned\") pod \"tuned-9gs5p\" (UID: \"56a1f173-7ec8-4f09-94ce-ee3bde8e4c17\") " pod="openshift-cluster-node-tuning-operator/tuned-9gs5p" Jul 14 17:32:33.538709 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.537059 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/48bfbea2-4a10-4454-a93d-72c4c8135d79-multus-socket-dir-parent\") pod \"multus-9dcd9\" (UID: \"48bfbea2-4a10-4454-a93d-72c4c8135d79\") " pod="openshift-multus/multus-9dcd9" Jul 14 17:32:33.538709 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.537067 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hznl9\" (UniqueName: \"kubernetes.io/projected/17370624-81e7-42ff-b916-e3f323b3e0a5-kube-api-access-hznl9\") pod \"node-ca-thnpp\" (UID: \"17370624-81e7-42ff-b916-e3f323b3e0a5\") " pod="openshift-image-registry/node-ca-thnpp" Jul 14 17:32:33.538709 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.537083 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/5bca996e-4e3a-4ec6-bd2f-0a5860c9d632-etc-selinux\") pod \"aws-ebs-csi-driver-node-7t9rj\" (UID: \"5bca996e-4e3a-4ec6-bd2f-0a5860c9d632\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7t9rj" Jul 14 17:32:33.538709 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.537103 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/91ee1d15-3741-4cf5-821e-d3b216b7e246-hosts-file\") pod \"node-resolver-7fdn8\" (UID: \"91ee1d15-3741-4cf5-821e-d3b216b7e246\") " pod="openshift-dns/node-resolver-7fdn8" Jul 14 17:32:33.538709 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.537105 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/56a1f173-7ec8-4f09-94ce-ee3bde8e4c17-etc-kubernetes\") pod \"tuned-9gs5p\" (UID: \"56a1f173-7ec8-4f09-94ce-ee3bde8e4c17\") " pod="openshift-cluster-node-tuning-operator/tuned-9gs5p" Jul 14 17:32:33.538709 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.537143 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/56a1f173-7ec8-4f09-94ce-ee3bde8e4c17-etc-kubernetes\") pod \"tuned-9gs5p\" (UID: \"56a1f173-7ec8-4f09-94ce-ee3bde8e4c17\") " pod="openshift-cluster-node-tuning-operator/tuned-9gs5p" Jul 14 17:32:33.538709 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.537150 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/5bca996e-4e3a-4ec6-bd2f-0a5860c9d632-kubelet-dir\") pod \"aws-ebs-csi-driver-node-7t9rj\" (UID: \"5bca996e-4e3a-4ec6-bd2f-0a5860c9d632\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7t9rj" Jul 14 17:32:33.538709 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.537237 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/48bfbea2-4a10-4454-a93d-72c4c8135d79-host-var-lib-kubelet\") pod \"multus-9dcd9\" (UID: \"48bfbea2-4a10-4454-a93d-72c4c8135d79\") " pod="openshift-multus/multus-9dcd9" Jul 14 17:32:33.538709 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.537382 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/48bfbea2-4a10-4454-a93d-72c4c8135d79-multus-cni-dir\") pod \"multus-9dcd9\" (UID: \"48bfbea2-4a10-4454-a93d-72c4c8135d79\") " pod="openshift-multus/multus-9dcd9" Jul 14 17:32:33.538709 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.538346 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/56a1f173-7ec8-4f09-94ce-ee3bde8e4c17-tmp\") pod \"tuned-9gs5p\" (UID: \"56a1f173-7ec8-4f09-94ce-ee3bde8e4c17\") " pod="openshift-cluster-node-tuning-operator/tuned-9gs5p" Jul 14 17:32:33.539176 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.539161 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/56a1f173-7ec8-4f09-94ce-ee3bde8e4c17-etc-tuned\") pod \"tuned-9gs5p\" (UID: \"56a1f173-7ec8-4f09-94ce-ee3bde8e4c17\") " pod="openshift-cluster-node-tuning-operator/tuned-9gs5p" Jul 14 17:32:33.539276 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.539256 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/650b148f-78d8-456f-bc24-1fbe4a75a24a-agent-certs\") pod \"konnectivity-agent-hxbnp\" (UID: \"650b148f-78d8-456f-bc24-1fbe4a75a24a\") " pod="kube-system/konnectivity-agent-hxbnp" Jul 14 17:32:33.549459 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.548738 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5nllr\" (UniqueName: \"kubernetes.io/projected/91ee1d15-3741-4cf5-821e-d3b216b7e246-kube-api-access-5nllr\") pod \"node-resolver-7fdn8\" (UID: \"91ee1d15-3741-4cf5-821e-d3b216b7e246\") " pod="openshift-dns/node-resolver-7fdn8" Jul 14 17:32:33.549459 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.549065 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m9kld\" (UniqueName: \"kubernetes.io/projected/48bfbea2-4a10-4454-a93d-72c4c8135d79-kube-api-access-m9kld\") pod \"multus-9dcd9\" (UID: \"48bfbea2-4a10-4454-a93d-72c4c8135d79\") " pod="openshift-multus/multus-9dcd9" Jul 14 17:32:33.549459 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.549133 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b7fkv\" (UniqueName: \"kubernetes.io/projected/5bca996e-4e3a-4ec6-bd2f-0a5860c9d632-kube-api-access-b7fkv\") pod \"aws-ebs-csi-driver-node-7t9rj\" (UID: \"5bca996e-4e3a-4ec6-bd2f-0a5860c9d632\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7t9rj" Jul 14 17:32:33.549985 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.549964 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lqqvr\" (UniqueName: \"kubernetes.io/projected/56a1f173-7ec8-4f09-94ce-ee3bde8e4c17-kube-api-access-lqqvr\") pod \"tuned-9gs5p\" (UID: \"56a1f173-7ec8-4f09-94ce-ee3bde8e4c17\") " pod="openshift-cluster-node-tuning-operator/tuned-9gs5p" Jul 14 17:32:33.550652 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.550633 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hznl9\" (UniqueName: \"kubernetes.io/projected/17370624-81e7-42ff-b916-e3f323b3e0a5-kube-api-access-hznl9\") pod \"node-ca-thnpp\" (UID: \"17370624-81e7-42ff-b916-e3f323b3e0a5\") " pod="openshift-image-registry/node-ca-thnpp" Jul 14 17:32:33.622968 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.622910 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-2sbww" Jul 14 17:32:33.630438 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.630419 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-prnxg" Jul 14 17:32:33.640428 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.640405 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-fjrz4" Jul 14 17:32:33.645051 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.645033 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-hxbnp" Jul 14 17:32:33.651524 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.651506 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-9gs5p" Jul 14 17:32:33.656928 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.656910 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-thnpp" Jul 14 17:32:33.657335 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:33.657313 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b670bba_aee6_407b_8f4e_e900528f54e5.slice/crio-66052a09457d073d4b9da074b5784d245d7055124a9a7fce9db70a5686f1d4e7 WatchSource:0}: Error finding container 66052a09457d073d4b9da074b5784d245d7055124a9a7fce9db70a5686f1d4e7: Status 404 returned error can't find the container with id 66052a09457d073d4b9da074b5784d245d7055124a9a7fce9db70a5686f1d4e7 Jul 14 17:32:33.657999 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:33.657923 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod650b148f_78d8_456f_bc24_1fbe4a75a24a.slice/crio-35764fd6e65234d21363ad0d05bc6dc524770f323e52f3e2ee38d510aa386201 WatchSource:0}: Error finding container 35764fd6e65234d21363ad0d05bc6dc524770f323e52f3e2ee38d510aa386201: Status 404 returned error can't find the container with id 35764fd6e65234d21363ad0d05bc6dc524770f323e52f3e2ee38d510aa386201 Jul 14 17:32:33.659069 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:33.659049 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddadd3729_1c46_48d9_92c8_bae9cdbbd1c1.slice/crio-d7121650cc6dd10be764ae0ca90206214a8a36d5bbce7a1a33d622a2bf832360 WatchSource:0}: Error finding container d7121650cc6dd10be764ae0ca90206214a8a36d5bbce7a1a33d622a2bf832360: Status 404 returned error can't find the container with id d7121650cc6dd10be764ae0ca90206214a8a36d5bbce7a1a33d622a2bf832360 Jul 14 17:32:33.661095 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:33.661074 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3d00b23_d0d5_4d18_a722_464d7b114745.slice/crio-4aed63735eb55ca48138b04b621c5fe8b984e7d1ac29dde9a59cf7f4c5358df9 WatchSource:0}: Error finding container 4aed63735eb55ca48138b04b621c5fe8b984e7d1ac29dde9a59cf7f4c5358df9: Status 404 returned error can't find the container with id 4aed63735eb55ca48138b04b621c5fe8b984e7d1ac29dde9a59cf7f4c5358df9 Jul 14 17:32:33.663666 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.663647 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7t9rj" Jul 14 17:32:33.665066 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:33.664991 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56a1f173_7ec8_4f09_94ce_ee3bde8e4c17.slice/crio-19a2a46ceb64e16c62da8c5cedea2f26c3a9950a8890800a16c13cda06b6d861 WatchSource:0}: Error finding container 19a2a46ceb64e16c62da8c5cedea2f26c3a9950a8890800a16c13cda06b6d861: Status 404 returned error can't find the container with id 19a2a46ceb64e16c62da8c5cedea2f26c3a9950a8890800a16c13cda06b6d861 Jul 14 17:32:33.670069 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:33.670039 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17370624_81e7_42ff_b916_e3f323b3e0a5.slice/crio-e1d9eda65d6574da16364594943033f616a049c39064ce70036351f64f47471e WatchSource:0}: Error finding container e1d9eda65d6574da16364594943033f616a049c39064ce70036351f64f47471e: Status 404 returned error can't find the container with id e1d9eda65d6574da16364594943033f616a049c39064ce70036351f64f47471e Jul 14 17:32:33.677423 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.677401 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-7fdn8" Jul 14 17:32:33.683671 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.683652 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-9dcd9" Jul 14 17:32:33.699257 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:33.699236 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5bca996e_4e3a_4ec6_bd2f_0a5860c9d632.slice/crio-7f19c7e2ef6088138b8a23f20a111203374016400f25621f01becb1e1deba7c2 WatchSource:0}: Error finding container 7f19c7e2ef6088138b8a23f20a111203374016400f25621f01becb1e1deba7c2: Status 404 returned error can't find the container with id 7f19c7e2ef6088138b8a23f20a111203374016400f25621f01becb1e1deba7c2 Jul 14 17:32:33.699946 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:33.699926 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48bfbea2_4a10_4454_a93d_72c4c8135d79.slice/crio-66e30cb850c17518e147772c60f8d6683983415de176a007256072555e26c080 WatchSource:0}: Error finding container 66e30cb850c17518e147772c60f8d6683983415de176a007256072555e26c080: Status 404 returned error can't find the container with id 66e30cb850c17518e147772c60f8d6683983415de176a007256072555e26c080 Jul 14 17:32:33.700822 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:32:33.700806 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91ee1d15_3741_4cf5_821e_d3b216b7e246.slice/crio-a1f0693a924c83b75e250333d99e6733e2202edc00e2abcd99e7405bf426f853 WatchSource:0}: Error finding container a1f0693a924c83b75e250333d99e6733e2202edc00e2abcd99e7405bf426f853: Status 404 returned error can't find the container with id a1f0693a924c83b75e250333d99e6733e2202edc00e2abcd99e7405bf426f853 Jul 14 17:32:33.940330 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.940179 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ac89d28e-0c57-4be5-8c3a-f0482ce0ba6f-original-pull-secret\") pod \"global-pull-secret-syncer-kv2bk\" (UID: \"ac89d28e-0c57-4be5-8c3a-f0482ce0ba6f\") " pod="kube-system/global-pull-secret-syncer-kv2bk" Jul 14 17:32:33.940451 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:33.940346 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8321d9c3-c963-45a9-8a94-ef9911a8d33f-metrics-certs\") pod \"network-metrics-daemon-h8t96\" (UID: \"8321d9c3-c963-45a9-8a94-ef9911a8d33f\") " pod="openshift-multus/network-metrics-daemon-h8t96" Jul 14 17:32:33.940451 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:32:33.940306 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 17:32:33.940567 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:32:33.940460 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 17:32:33.940567 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:32:33.940471 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac89d28e-0c57-4be5-8c3a-f0482ce0ba6f-original-pull-secret podName:ac89d28e-0c57-4be5-8c3a-f0482ce0ba6f nodeName:}" failed. No retries permitted until 2026-07-14 17:32:34.940448901 +0000 UTC m=+4.016906438 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ac89d28e-0c57-4be5-8c3a-f0482ce0ba6f-original-pull-secret") pod "global-pull-secret-syncer-kv2bk" (UID: "ac89d28e-0c57-4be5-8c3a-f0482ce0ba6f") : object "kube-system"/"original-pull-secret" not registered Jul 14 17:32:33.940567 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:32:33.940510 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8321d9c3-c963-45a9-8a94-ef9911a8d33f-metrics-certs podName:8321d9c3-c963-45a9-8a94-ef9911a8d33f nodeName:}" failed. No retries permitted until 2026-07-14 17:32:34.940497682 +0000 UTC m=+4.016955198 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8321d9c3-c963-45a9-8a94-ef9911a8d33f-metrics-certs") pod "network-metrics-daemon-h8t96" (UID: "8321d9c3-c963-45a9-8a94-ef9911a8d33f") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 17:32:34.040869 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:34.040839 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qqfb\" (UniqueName: \"kubernetes.io/projected/5b2d2388-62da-4462-af63-9230c6c43715-kube-api-access-5qqfb\") pod \"network-check-target-9m2f6\" (UID: \"5b2d2388-62da-4462-af63-9230c6c43715\") " pod="openshift-network-diagnostics/network-check-target-9m2f6" Jul 14 17:32:34.041055 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:32:34.041034 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 17:32:34.041108 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:32:34.041067 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 17:32:34.041108 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:32:34.041080 2562 projected.go:196] Error preparing data for projected volume kube-api-access-5qqfb for pod openshift-network-diagnostics/network-check-target-9m2f6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 17:32:34.041212 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:32:34.041140 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5b2d2388-62da-4462-af63-9230c6c43715-kube-api-access-5qqfb podName:5b2d2388-62da-4462-af63-9230c6c43715 nodeName:}" failed. No retries permitted until 2026-07-14 17:32:35.041121835 +0000 UTC m=+4.117579367 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-5qqfb" (UniqueName: "kubernetes.io/projected/5b2d2388-62da-4462-af63-9230c6c43715-kube-api-access-5qqfb") pod "network-check-target-9m2f6" (UID: "5b2d2388-62da-4462-af63-9230c6c43715") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 17:32:34.378077 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:34.378004 2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-13 17:27:32 +0000 UTC" deadline="2028-02-06 12:03:12.868855093 +0000 UTC" Jul 14 17:32:34.378077 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:34.378069 2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13722h30m38.490790415s" Jul 14 17:32:34.513843 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:34.513770 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9m2f6" Jul 14 17:32:34.514003 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:32:34.513899 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9m2f6" podUID="5b2d2388-62da-4462-af63-9230c6c43715" Jul 14 17:32:34.514526 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:34.514317 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h8t96" Jul 14 17:32:34.514526 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:32:34.514426 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-h8t96" podUID="8321d9c3-c963-45a9-8a94-ef9911a8d33f" Jul 14 17:32:34.524282 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:34.524178 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7t9rj" event={"ID":"5bca996e-4e3a-4ec6-bd2f-0a5860c9d632","Type":"ContainerStarted","Data":"7f19c7e2ef6088138b8a23f20a111203374016400f25621f01becb1e1deba7c2"} Jul 14 17:32:34.526579 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:34.526512 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-thnpp" event={"ID":"17370624-81e7-42ff-b916-e3f323b3e0a5","Type":"ContainerStarted","Data":"e1d9eda65d6574da16364594943033f616a049c39064ce70036351f64f47471e"} Jul 14 17:32:34.531060 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:34.530951 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2sbww" event={"ID":"c3d00b23-d0d5-4d18-a722-464d7b114745","Type":"ContainerStarted","Data":"4aed63735eb55ca48138b04b621c5fe8b984e7d1ac29dde9a59cf7f4c5358df9"} Jul 14 17:32:34.535155 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:34.535095 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-prnxg" event={"ID":"2b670bba-aee6-407b-8f4e-e900528f54e5","Type":"ContainerStarted","Data":"66052a09457d073d4b9da074b5784d245d7055124a9a7fce9db70a5686f1d4e7"} Jul 14 17:32:34.538946 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:34.538874 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-7fdn8" event={"ID":"91ee1d15-3741-4cf5-821e-d3b216b7e246","Type":"ContainerStarted","Data":"a1f0693a924c83b75e250333d99e6733e2202edc00e2abcd99e7405bf426f853"} Jul 14 17:32:34.541407 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:34.541356 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-9dcd9" event={"ID":"48bfbea2-4a10-4454-a93d-72c4c8135d79","Type":"ContainerStarted","Data":"66e30cb850c17518e147772c60f8d6683983415de176a007256072555e26c080"} Jul 14 17:32:34.543261 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:34.543234 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-9gs5p" event={"ID":"56a1f173-7ec8-4f09-94ce-ee3bde8e4c17","Type":"ContainerStarted","Data":"19a2a46ceb64e16c62da8c5cedea2f26c3a9950a8890800a16c13cda06b6d861"} Jul 14 17:32:34.547112 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:34.547086 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-hxbnp" event={"ID":"650b148f-78d8-456f-bc24-1fbe4a75a24a","Type":"ContainerStarted","Data":"35764fd6e65234d21363ad0d05bc6dc524770f323e52f3e2ee38d510aa386201"} Jul 14 17:32:34.549858 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:34.549802 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fjrz4" event={"ID":"dadd3729-1c46-48d9-92c8-bae9cdbbd1c1","Type":"ContainerStarted","Data":"d7121650cc6dd10be764ae0ca90206214a8a36d5bbce7a1a33d622a2bf832360"} Jul 14 17:32:34.566164 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:34.566132 2562 generic.go:356] "Generic (PLEG): container finished" podID="0f6dec0287e6513120043e23e8c805f3" containerID="90086063c51da4a1e061aa30eb84c87fbf223f0612b27b11b73401916128998c" exitCode=0 Jul 14 17:32:34.566286 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:34.566177 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-209.ec2.internal" event={"ID":"0f6dec0287e6513120043e23e8c805f3","Type":"ContainerDied","Data":"90086063c51da4a1e061aa30eb84c87fbf223f0612b27b11b73401916128998c"} Jul 14 17:32:34.947316 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:34.947181 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ac89d28e-0c57-4be5-8c3a-f0482ce0ba6f-original-pull-secret\") pod \"global-pull-secret-syncer-kv2bk\" (UID: \"ac89d28e-0c57-4be5-8c3a-f0482ce0ba6f\") " pod="kube-system/global-pull-secret-syncer-kv2bk" Jul 14 17:32:34.947316 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:34.947255 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8321d9c3-c963-45a9-8a94-ef9911a8d33f-metrics-certs\") pod \"network-metrics-daemon-h8t96\" (UID: \"8321d9c3-c963-45a9-8a94-ef9911a8d33f\") " pod="openshift-multus/network-metrics-daemon-h8t96" Jul 14 17:32:34.947518 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:32:34.947419 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 17:32:34.947518 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:32:34.947483 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8321d9c3-c963-45a9-8a94-ef9911a8d33f-metrics-certs podName:8321d9c3-c963-45a9-8a94-ef9911a8d33f nodeName:}" failed. No retries permitted until 2026-07-14 17:32:36.947462952 +0000 UTC m=+6.023920472 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8321d9c3-c963-45a9-8a94-ef9911a8d33f-metrics-certs") pod "network-metrics-daemon-h8t96" (UID: "8321d9c3-c963-45a9-8a94-ef9911a8d33f") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 17:32:34.947952 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:32:34.947922 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 17:32:34.948016 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:32:34.947984 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac89d28e-0c57-4be5-8c3a-f0482ce0ba6f-original-pull-secret podName:ac89d28e-0c57-4be5-8c3a-f0482ce0ba6f nodeName:}" failed. No retries permitted until 2026-07-14 17:32:36.947967476 +0000 UTC m=+6.024424996 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ac89d28e-0c57-4be5-8c3a-f0482ce0ba6f-original-pull-secret") pod "global-pull-secret-syncer-kv2bk" (UID: "ac89d28e-0c57-4be5-8c3a-f0482ce0ba6f") : object "kube-system"/"original-pull-secret" not registered Jul 14 17:32:35.048468 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:35.048432 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qqfb\" (UniqueName: \"kubernetes.io/projected/5b2d2388-62da-4462-af63-9230c6c43715-kube-api-access-5qqfb\") pod \"network-check-target-9m2f6\" (UID: \"5b2d2388-62da-4462-af63-9230c6c43715\") " pod="openshift-network-diagnostics/network-check-target-9m2f6" Jul 14 17:32:35.048647 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:32:35.048633 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 17:32:35.048712 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:32:35.048656 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 17:32:35.048712 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:32:35.048678 2562 projected.go:196] Error preparing data for projected volume kube-api-access-5qqfb for pod openshift-network-diagnostics/network-check-target-9m2f6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 17:32:35.048804 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:32:35.048732 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5b2d2388-62da-4462-af63-9230c6c43715-kube-api-access-5qqfb podName:5b2d2388-62da-4462-af63-9230c6c43715 nodeName:}" failed. No retries permitted until 2026-07-14 17:32:37.048715175 +0000 UTC m=+6.125172706 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-5qqfb" (UniqueName: "kubernetes.io/projected/5b2d2388-62da-4462-af63-9230c6c43715-kube-api-access-5qqfb") pod "network-check-target-9m2f6" (UID: "5b2d2388-62da-4462-af63-9230c6c43715") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 17:32:35.521149 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:35.518671 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kv2bk" Jul 14 17:32:35.521149 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:32:35.518799 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kv2bk" podUID="ac89d28e-0c57-4be5-8c3a-f0482ce0ba6f" Jul 14 17:32:36.514142 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:36.514104 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9m2f6" Jul 14 17:32:36.514327 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:32:36.514244 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9m2f6" podUID="5b2d2388-62da-4462-af63-9230c6c43715" Jul 14 17:32:36.514668 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:36.514646 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h8t96" Jul 14 17:32:36.514787 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:32:36.514767 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-h8t96" podUID="8321d9c3-c963-45a9-8a94-ef9911a8d33f" Jul 14 17:32:36.964028 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:36.963326 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ac89d28e-0c57-4be5-8c3a-f0482ce0ba6f-original-pull-secret\") pod \"global-pull-secret-syncer-kv2bk\" (UID: \"ac89d28e-0c57-4be5-8c3a-f0482ce0ba6f\") " pod="kube-system/global-pull-secret-syncer-kv2bk" Jul 14 17:32:36.964028 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:36.963378 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8321d9c3-c963-45a9-8a94-ef9911a8d33f-metrics-certs\") pod \"network-metrics-daemon-h8t96\" (UID: \"8321d9c3-c963-45a9-8a94-ef9911a8d33f\") " pod="openshift-multus/network-metrics-daemon-h8t96" Jul 14 17:32:36.964028 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:32:36.963501 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 17:32:36.964028 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:32:36.963561 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8321d9c3-c963-45a9-8a94-ef9911a8d33f-metrics-certs podName:8321d9c3-c963-45a9-8a94-ef9911a8d33f nodeName:}" failed. No retries permitted until 2026-07-14 17:32:40.963541963 +0000 UTC m=+10.039999484 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8321d9c3-c963-45a9-8a94-ef9911a8d33f-metrics-certs") pod "network-metrics-daemon-h8t96" (UID: "8321d9c3-c963-45a9-8a94-ef9911a8d33f") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 17:32:36.964028 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:32:36.963623 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 17:32:36.964028 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:32:36.963656 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac89d28e-0c57-4be5-8c3a-f0482ce0ba6f-original-pull-secret podName:ac89d28e-0c57-4be5-8c3a-f0482ce0ba6f nodeName:}" failed. No retries permitted until 2026-07-14 17:32:40.963645101 +0000 UTC m=+10.040102620 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ac89d28e-0c57-4be5-8c3a-f0482ce0ba6f-original-pull-secret") pod "global-pull-secret-syncer-kv2bk" (UID: "ac89d28e-0c57-4be5-8c3a-f0482ce0ba6f") : object "kube-system"/"original-pull-secret" not registered Jul 14 17:32:37.065259 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:37.064619 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qqfb\" (UniqueName: \"kubernetes.io/projected/5b2d2388-62da-4462-af63-9230c6c43715-kube-api-access-5qqfb\") pod \"network-check-target-9m2f6\" (UID: \"5b2d2388-62da-4462-af63-9230c6c43715\") " pod="openshift-network-diagnostics/network-check-target-9m2f6" Jul 14 17:32:37.065259 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:32:37.064816 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 17:32:37.065259 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:32:37.064840 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 17:32:37.065259 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:32:37.064853 2562 projected.go:196] Error preparing data for projected volume kube-api-access-5qqfb for pod openshift-network-diagnostics/network-check-target-9m2f6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 17:32:37.065259 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:32:37.064909 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5b2d2388-62da-4462-af63-9230c6c43715-kube-api-access-5qqfb podName:5b2d2388-62da-4462-af63-9230c6c43715 nodeName:}" failed. No retries permitted until 2026-07-14 17:32:41.064890251 +0000 UTC m=+10.141347789 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-5qqfb" (UniqueName: "kubernetes.io/projected/5b2d2388-62da-4462-af63-9230c6c43715-kube-api-access-5qqfb") pod "network-check-target-9m2f6" (UID: "5b2d2388-62da-4462-af63-9230c6c43715") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 17:32:37.515303 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:37.514531 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kv2bk" Jul 14 17:32:37.515303 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:32:37.514654 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kv2bk" podUID="ac89d28e-0c57-4be5-8c3a-f0482ce0ba6f" Jul 14 17:32:38.513948 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:38.513913 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9m2f6" Jul 14 17:32:38.514436 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:32:38.514043 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9m2f6" podUID="5b2d2388-62da-4462-af63-9230c6c43715" Jul 14 17:32:38.514489 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:38.514464 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h8t96" Jul 14 17:32:38.514599 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:32:38.514566 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-h8t96" podUID="8321d9c3-c963-45a9-8a94-ef9911a8d33f" Jul 14 17:32:39.515701 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:39.515624 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kv2bk" Jul 14 17:32:39.516167 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:32:39.515748 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kv2bk" podUID="ac89d28e-0c57-4be5-8c3a-f0482ce0ba6f" Jul 14 17:32:40.514603 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:40.514572 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h8t96" Jul 14 17:32:40.514786 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:40.514620 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9m2f6" Jul 14 17:32:40.514786 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:32:40.514712 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-h8t96" podUID="8321d9c3-c963-45a9-8a94-ef9911a8d33f" Jul 14 17:32:40.514899 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:32:40.514842 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9m2f6" podUID="5b2d2388-62da-4462-af63-9230c6c43715" Jul 14 17:32:40.993447 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:40.993403 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ac89d28e-0c57-4be5-8c3a-f0482ce0ba6f-original-pull-secret\") pod \"global-pull-secret-syncer-kv2bk\" (UID: \"ac89d28e-0c57-4be5-8c3a-f0482ce0ba6f\") " pod="kube-system/global-pull-secret-syncer-kv2bk" Jul 14 17:32:40.993447 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:40.993450 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8321d9c3-c963-45a9-8a94-ef9911a8d33f-metrics-certs\") pod \"network-metrics-daemon-h8t96\" (UID: \"8321d9c3-c963-45a9-8a94-ef9911a8d33f\") " pod="openshift-multus/network-metrics-daemon-h8t96" Jul 14 17:32:40.993934 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:32:40.993568 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 17:32:40.993934 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:32:40.993578 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 17:32:40.993934 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:32:40.993629 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac89d28e-0c57-4be5-8c3a-f0482ce0ba6f-original-pull-secret podName:ac89d28e-0c57-4be5-8c3a-f0482ce0ba6f nodeName:}" failed. No retries permitted until 2026-07-14 17:32:48.993610938 +0000 UTC m=+18.070068455 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ac89d28e-0c57-4be5-8c3a-f0482ce0ba6f-original-pull-secret") pod "global-pull-secret-syncer-kv2bk" (UID: "ac89d28e-0c57-4be5-8c3a-f0482ce0ba6f") : object "kube-system"/"original-pull-secret" not registered Jul 14 17:32:40.993934 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:32:40.993647 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8321d9c3-c963-45a9-8a94-ef9911a8d33f-metrics-certs podName:8321d9c3-c963-45a9-8a94-ef9911a8d33f nodeName:}" failed. No retries permitted until 2026-07-14 17:32:48.993637912 +0000 UTC m=+18.070095433 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8321d9c3-c963-45a9-8a94-ef9911a8d33f-metrics-certs") pod "network-metrics-daemon-h8t96" (UID: "8321d9c3-c963-45a9-8a94-ef9911a8d33f") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 17:32:41.094247 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:41.094212 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qqfb\" (UniqueName: \"kubernetes.io/projected/5b2d2388-62da-4462-af63-9230c6c43715-kube-api-access-5qqfb\") pod \"network-check-target-9m2f6\" (UID: \"5b2d2388-62da-4462-af63-9230c6c43715\") " pod="openshift-network-diagnostics/network-check-target-9m2f6" Jul 14 17:32:41.094436 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:32:41.094399 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 17:32:41.094436 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:32:41.094421 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 17:32:41.094436 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:32:41.094435 2562 projected.go:196] Error preparing data for projected volume kube-api-access-5qqfb for pod openshift-network-diagnostics/network-check-target-9m2f6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 17:32:41.094591 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:32:41.094489 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5b2d2388-62da-4462-af63-9230c6c43715-kube-api-access-5qqfb podName:5b2d2388-62da-4462-af63-9230c6c43715 nodeName:}" failed. No retries permitted until 2026-07-14 17:32:49.094471468 +0000 UTC m=+18.170928989 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-5qqfb" (UniqueName: "kubernetes.io/projected/5b2d2388-62da-4462-af63-9230c6c43715-kube-api-access-5qqfb") pod "network-check-target-9m2f6" (UID: "5b2d2388-62da-4462-af63-9230c6c43715") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 17:32:41.515173 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:41.515141 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kv2bk" Jul 14 17:32:41.515365 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:32:41.515279 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kv2bk" podUID="ac89d28e-0c57-4be5-8c3a-f0482ce0ba6f" Jul 14 17:32:42.514356 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:42.514322 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9m2f6" Jul 14 17:32:42.514760 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:32:42.514447 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9m2f6" podUID="5b2d2388-62da-4462-af63-9230c6c43715" Jul 14 17:32:42.514760 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:42.514716 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h8t96" Jul 14 17:32:42.514865 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:32:42.514785 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-h8t96" podUID="8321d9c3-c963-45a9-8a94-ef9911a8d33f" Jul 14 17:32:43.514568 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:43.514530 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kv2bk" Jul 14 17:32:43.514993 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:32:43.514674 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kv2bk" podUID="ac89d28e-0c57-4be5-8c3a-f0482ce0ba6f" Jul 14 17:32:44.513751 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:44.513719 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9m2f6" Jul 14 17:32:44.513900 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:44.513720 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h8t96" Jul 14 17:32:44.513900 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:32:44.513817 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9m2f6" podUID="5b2d2388-62da-4462-af63-9230c6c43715" Jul 14 17:32:44.513963 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:32:44.513898 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-h8t96" podUID="8321d9c3-c963-45a9-8a94-ef9911a8d33f" Jul 14 17:32:45.513894 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:45.513855 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kv2bk" Jul 14 17:32:45.514388 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:32:45.513983 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kv2bk" podUID="ac89d28e-0c57-4be5-8c3a-f0482ce0ba6f" Jul 14 17:32:46.514422 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:46.514394 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9m2f6" Jul 14 17:32:46.514850 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:46.514395 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h8t96" Jul 14 17:32:46.514850 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:32:46.514517 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9m2f6" podUID="5b2d2388-62da-4462-af63-9230c6c43715" Jul 14 17:32:46.514850 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:32:46.514595 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-h8t96" podUID="8321d9c3-c963-45a9-8a94-ef9911a8d33f" Jul 14 17:32:47.514061 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:47.514015 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kv2bk" Jul 14 17:32:47.514248 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:32:47.514125 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kv2bk" podUID="ac89d28e-0c57-4be5-8c3a-f0482ce0ba6f" Jul 14 17:32:48.513979 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:48.513946 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9m2f6" Jul 14 17:32:48.514498 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:32:48.514074 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9m2f6" podUID="5b2d2388-62da-4462-af63-9230c6c43715" Jul 14 17:32:48.514498 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:48.514475 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h8t96" Jul 14 17:32:48.514628 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:32:48.514609 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-h8t96" podUID="8321d9c3-c963-45a9-8a94-ef9911a8d33f" Jul 14 17:32:49.054768 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:49.054740 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ac89d28e-0c57-4be5-8c3a-f0482ce0ba6f-original-pull-secret\") pod \"global-pull-secret-syncer-kv2bk\" (UID: \"ac89d28e-0c57-4be5-8c3a-f0482ce0ba6f\") " pod="kube-system/global-pull-secret-syncer-kv2bk" Jul 14 17:32:49.054768 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:49.054773 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8321d9c3-c963-45a9-8a94-ef9911a8d33f-metrics-certs\") pod \"network-metrics-daemon-h8t96\" (UID: \"8321d9c3-c963-45a9-8a94-ef9911a8d33f\") " pod="openshift-multus/network-metrics-daemon-h8t96" Jul 14 17:32:49.054999 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:32:49.054862 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 17:32:49.054999 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:32:49.054893 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 17:32:49.054999 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:32:49.054914 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8321d9c3-c963-45a9-8a94-ef9911a8d33f-metrics-certs podName:8321d9c3-c963-45a9-8a94-ef9911a8d33f nodeName:}" failed. No retries permitted until 2026-07-14 17:33:05.05490158 +0000 UTC m=+34.131359098 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8321d9c3-c963-45a9-8a94-ef9911a8d33f-metrics-certs") pod "network-metrics-daemon-h8t96" (UID: "8321d9c3-c963-45a9-8a94-ef9911a8d33f") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 17:32:49.054999 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:32:49.054950 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac89d28e-0c57-4be5-8c3a-f0482ce0ba6f-original-pull-secret podName:ac89d28e-0c57-4be5-8c3a-f0482ce0ba6f nodeName:}" failed. No retries permitted until 2026-07-14 17:33:05.054932101 +0000 UTC m=+34.131389617 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ac89d28e-0c57-4be5-8c3a-f0482ce0ba6f-original-pull-secret") pod "global-pull-secret-syncer-kv2bk" (UID: "ac89d28e-0c57-4be5-8c3a-f0482ce0ba6f") : object "kube-system"/"original-pull-secret" not registered Jul 14 17:32:49.155949 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:49.155921 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qqfb\" (UniqueName: \"kubernetes.io/projected/5b2d2388-62da-4462-af63-9230c6c43715-kube-api-access-5qqfb\") pod \"network-check-target-9m2f6\" (UID: \"5b2d2388-62da-4462-af63-9230c6c43715\") " pod="openshift-network-diagnostics/network-check-target-9m2f6" Jul 14 17:32:49.156080 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:32:49.156065 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 17:32:49.156128 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:32:49.156084 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 17:32:49.156128 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:32:49.156094 2562 projected.go:196] Error preparing data for projected volume kube-api-access-5qqfb for pod openshift-network-diagnostics/network-check-target-9m2f6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 17:32:49.156212 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:32:49.156134 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5b2d2388-62da-4462-af63-9230c6c43715-kube-api-access-5qqfb podName:5b2d2388-62da-4462-af63-9230c6c43715 nodeName:}" failed. No retries permitted until 2026-07-14 17:33:05.156121988 +0000 UTC m=+34.232579504 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-5qqfb" (UniqueName: "kubernetes.io/projected/5b2d2388-62da-4462-af63-9230c6c43715-kube-api-access-5qqfb") pod "network-check-target-9m2f6" (UID: "5b2d2388-62da-4462-af63-9230c6c43715") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 17:32:49.514347 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:49.514275 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kv2bk" Jul 14 17:32:49.514772 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:32:49.514394 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kv2bk" podUID="ac89d28e-0c57-4be5-8c3a-f0482ce0ba6f" Jul 14 17:32:50.514651 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:50.514614 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h8t96" Jul 14 17:32:50.514651 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:50.514649 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9m2f6" Jul 14 17:32:50.515118 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:32:50.514760 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-h8t96" podUID="8321d9c3-c963-45a9-8a94-ef9911a8d33f" Jul 14 17:32:50.515118 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:32:50.514898 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9m2f6" podUID="5b2d2388-62da-4462-af63-9230c6c43715" Jul 14 17:32:51.514829 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:51.514797 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kv2bk" Jul 14 17:32:51.515401 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:32:51.514894 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kv2bk" podUID="ac89d28e-0c57-4be5-8c3a-f0482ce0ba6f" Jul 14 17:32:52.514376 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:52.514342 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9m2f6" Jul 14 17:32:52.514549 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:52.514341 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h8t96" Jul 14 17:32:52.514549 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:32:52.514469 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9m2f6" podUID="5b2d2388-62da-4462-af63-9230c6c43715" Jul 14 17:32:52.514674 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:32:52.514546 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-h8t96" podUID="8321d9c3-c963-45a9-8a94-ef9911a8d33f" Jul 14 17:32:53.514238 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:53.514062 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kv2bk" Jul 14 17:32:53.514800 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:32:53.514316 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kv2bk" podUID="ac89d28e-0c57-4be5-8c3a-f0482ce0ba6f" Jul 14 17:32:53.608261 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:53.608220 2562 generic.go:356] "Generic (PLEG): container finished" podID="dadd3729-1c46-48d9-92c8-bae9cdbbd1c1" containerID="615113d40dd848a044bcdaf37832ea367b7eb553a712ce6dc42f40cc349626b0" exitCode=0 Jul 14 17:32:53.608364 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:53.608298 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fjrz4" event={"ID":"dadd3729-1c46-48d9-92c8-bae9cdbbd1c1","Type":"ContainerDied","Data":"615113d40dd848a044bcdaf37832ea367b7eb553a712ce6dc42f40cc349626b0"} Jul 14 17:32:53.612223 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:53.612177 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-209.ec2.internal" event={"ID":"0f6dec0287e6513120043e23e8c805f3","Type":"ContainerStarted","Data":"1a8ee468cff106d1ef79fc80c97317ee134eeed9af907d75ced3beee4ed07039"} Jul 14 17:32:53.614002 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:53.613980 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7t9rj" event={"ID":"5bca996e-4e3a-4ec6-bd2f-0a5860c9d632","Type":"ContainerStarted","Data":"a7d2fa9e903666fa7fdd48a1652b5d0c071b10ce5b0d8ad170a6174e2e769d19"} Jul 14 17:32:53.615513 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:53.615488 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-thnpp" event={"ID":"17370624-81e7-42ff-b916-e3f323b3e0a5","Type":"ContainerStarted","Data":"080e5bdf6c85eb66e8e31557ad49c554a4203b75cb78906141006adbc3662afd"} Jul 14 17:32:53.617214 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:53.617179 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2sbww_c3d00b23-d0d5-4d18-a722-464d7b114745/ovn-acl-logging/0.log" Jul 14 17:32:53.617550 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:53.617527 2562 generic.go:356] "Generic (PLEG): container finished" podID="c3d00b23-d0d5-4d18-a722-464d7b114745" containerID="22892fd468203608c1f145f7618d540cef929183db47f62f630801f7413cdbd0" exitCode=1 Jul 14 17:32:53.617627 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:53.617603 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2sbww" event={"ID":"c3d00b23-d0d5-4d18-a722-464d7b114745","Type":"ContainerStarted","Data":"4562507cf6defe7af77409b61986a71fcc30ba3f6b86a88876460daa3f604dec"} Jul 14 17:32:53.617693 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:53.617678 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2sbww" event={"ID":"c3d00b23-d0d5-4d18-a722-464d7b114745","Type":"ContainerDied","Data":"22892fd468203608c1f145f7618d540cef929183db47f62f630801f7413cdbd0"} Jul 14 17:32:53.617750 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:53.617738 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2sbww" event={"ID":"c3d00b23-d0d5-4d18-a722-464d7b114745","Type":"ContainerStarted","Data":"70fd67d70b2c99a0575f8363e16daa722685c4f9df5a7a90433492dba2d3888c"} Jul 14 17:32:53.618791 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:53.618773 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-209.ec2.internal" event={"ID":"8b1a7c81fd72faef90d5a3084bb532e1","Type":"ContainerStarted","Data":"faefd7696cfe93a1a167ae4b3c6bc1ad800676702cf82c9640f6df048306bf6a"} Jul 14 17:32:53.619944 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:53.619924 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-7fdn8" event={"ID":"91ee1d15-3741-4cf5-821e-d3b216b7e246","Type":"ContainerStarted","Data":"64969af2b9c477cc4d267355da715fff98d9f610ce02d6ba307438f6ae54e5a2"} Jul 14 17:32:53.621264 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:53.621243 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-9gs5p" event={"ID":"56a1f173-7ec8-4f09-94ce-ee3bde8e4c17","Type":"ContainerStarted","Data":"f89580c269b55d8b5a798d647e5092b1842c2aba1c77ec312f8086417a0de221"} Jul 14 17:32:53.622496 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:53.622477 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-hxbnp" event={"ID":"650b148f-78d8-456f-bc24-1fbe4a75a24a","Type":"ContainerStarted","Data":"71800c33c785eade61d954aaf5480b303bb8c4f365a9d3d3df48f6a550b2d536"} Jul 14 17:32:53.655952 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:53.655908 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-209.ec2.internal" podStartSLOduration=21.655896111 podStartE2EDuration="21.655896111s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 17:32:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:32:53.655475428 +0000 UTC m=+22.731932966" watchObservedRunningTime="2026-07-14 17:32:53.655896111 +0000 UTC m=+22.732353651" Jul 14 17:32:53.691178 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:53.691007 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-209.ec2.internal" podStartSLOduration=21.690986076 podStartE2EDuration="21.690986076s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 17:32:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:32:53.671950208 +0000 UTC m=+22.748407740" watchObservedRunningTime="2026-07-14 17:32:53.690986076 +0000 UTC m=+22.767443614" Jul 14 17:32:53.691410 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:53.691371 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-thnpp" podStartSLOduration=3.31995786 podStartE2EDuration="22.69135998s" totalImagesPullingTime="19.37140212s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 17:32:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:32:53.691233626 +0000 UTC m=+22.767691166" watchObservedRunningTime="2026-07-14 17:32:53.69135998 +0000 UTC m=+22.767817523" Jul 14 17:32:53.707163 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:53.707122 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-hxbnp" podStartSLOduration=3.40459481 podStartE2EDuration="22.707111061s" totalImagesPullingTime="19.302516251s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:32:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:32:53.706715585 +0000 UTC m=+22.783173123" watchObservedRunningTime="2026-07-14 17:32:53.707111061 +0000 UTC m=+22.783568599" Jul 14 17:32:53.723593 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:53.723563 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-9gs5p" podStartSLOduration=3.327899657 podStartE2EDuration="22.723550873s" totalImagesPullingTime="19.395651216s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 17:32:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:32:53.723442059 +0000 UTC m=+22.799899599" watchObservedRunningTime="2026-07-14 17:32:53.723550873 +0000 UTC m=+22.800008411" Jul 14 17:32:53.738237 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:53.738174 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-7fdn8" podStartSLOduration=3.377695665 podStartE2EDuration="22.738166306s" totalImagesPullingTime="19.360470641s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 17:32:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:32:53.738114498 +0000 UTC m=+22.814572038" watchObservedRunningTime="2026-07-14 17:32:53.738166306 +0000 UTC m=+22.814623845" Jul 14 17:32:54.514031 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:54.514006 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9m2f6" Jul 14 17:32:54.514144 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:32:54.514126 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9m2f6" podUID="5b2d2388-62da-4462-af63-9230c6c43715" Jul 14 17:32:54.514494 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:54.514479 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h8t96" Jul 14 17:32:54.515004 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:32:54.514576 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-h8t96" podUID="8321d9c3-c963-45a9-8a94-ef9911a8d33f" Jul 14 17:32:54.626360 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:54.626320 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-9dcd9" event={"ID":"48bfbea2-4a10-4454-a93d-72c4c8135d79","Type":"ContainerStarted","Data":"8c1d8b9fa92c78690e59d7e3e390a31f6109f2ac09b2b8a120f958861a88c87d"} Jul 14 17:32:54.629365 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:54.629346 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2sbww_c3d00b23-d0d5-4d18-a722-464d7b114745/ovn-acl-logging/0.log" Jul 14 17:32:54.629749 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:54.629728 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2sbww" event={"ID":"c3d00b23-d0d5-4d18-a722-464d7b114745","Type":"ContainerStarted","Data":"1e0a3189d2a0312c06dd40daf416e7c05e063e3f836bcd5e8a7dc6f8e7653b00"} Jul 14 17:32:54.629814 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:54.629794 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2sbww" event={"ID":"c3d00b23-d0d5-4d18-a722-464d7b114745","Type":"ContainerStarted","Data":"66f4b1aed859df36d56f572451b3e9fd84fb3e70789f94e27b3ddaa299faed5e"} Jul 14 17:32:54.629870 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:54.629825 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2sbww" event={"ID":"c3d00b23-d0d5-4d18-a722-464d7b114745","Type":"ContainerStarted","Data":"80033046f584f0dab41868d7bdca34cbfe77317c845474a695a0d7c6ed8af86d"} Jul 14 17:32:54.631381 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:54.631357 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-prnxg" event={"ID":"2b670bba-aee6-407b-8f4e-e900528f54e5","Type":"ContainerStarted","Data":"49912f7e30f909b4abc66fa4768ba877ee2bd3492cd047668ebb49182aabf934"} Jul 14 17:32:54.646882 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:54.646840 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-9dcd9" podStartSLOduration=3.627790331 podStartE2EDuration="23.646826981s" totalImagesPullingTime="20.01903665s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 17:32:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:32:54.645632965 +0000 UTC m=+23.722090508" watchObservedRunningTime="2026-07-14 17:32:54.646826981 +0000 UTC m=+23.723284519" Jul 14 17:32:54.661515 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:54.661474 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-prnxg" podStartSLOduration=4.254044032 podStartE2EDuration="23.661460099s" totalImagesPullingTime="19.407416067s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 17:32:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:32:54.66133932 +0000 UTC m=+23.737796858" watchObservedRunningTime="2026-07-14 17:32:54.661460099 +0000 UTC m=+23.737917637" Jul 14 17:32:55.329519 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:55.329455 2562 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 14 17:32:55.392872 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:55.392751 2562 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-14T17:32:55.329471059Z","UUID":"1a7cf2d2-d156-44fd-9472-dc148257aef5","Handler":null,"Name":"","Endpoint":""} Jul 14 17:32:55.394321 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:55.394297 2562 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 14 17:32:55.394419 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:55.394344 2562 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 14 17:32:55.513969 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:55.513887 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kv2bk" Jul 14 17:32:55.514147 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:32:55.514010 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kv2bk" podUID="ac89d28e-0c57-4be5-8c3a-f0482ce0ba6f" Jul 14 17:32:55.575998 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:55.575968 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-hxbnp" Jul 14 17:32:55.576487 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:55.576472 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-hxbnp" Jul 14 17:32:55.635637 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:55.635604 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7t9rj" event={"ID":"5bca996e-4e3a-4ec6-bd2f-0a5860c9d632","Type":"ContainerStarted","Data":"0b31e139e9246cfd9dd8075c58718b413545dc15281b810528073018bad364b5"} Jul 14 17:32:55.635828 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:55.635663 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-hxbnp" Jul 14 17:32:55.636245 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:55.636217 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-hxbnp" Jul 14 17:32:56.514421 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:56.514251 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9m2f6" Jul 14 17:32:56.514628 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:56.514279 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h8t96" Jul 14 17:32:56.514628 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:32:56.514507 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9m2f6" podUID="5b2d2388-62da-4462-af63-9230c6c43715" Jul 14 17:32:56.514628 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:32:56.514607 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-h8t96" podUID="8321d9c3-c963-45a9-8a94-ef9911a8d33f" Jul 14 17:32:56.641174 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:56.641142 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2sbww_c3d00b23-d0d5-4d18-a722-464d7b114745/ovn-acl-logging/0.log" Jul 14 17:32:56.641675 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:56.641643 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2sbww" event={"ID":"c3d00b23-d0d5-4d18-a722-464d7b114745","Type":"ContainerStarted","Data":"8a0127ccf6c9221d2c7b3c78a697802c6875ac486d1f9cd8293ad69f975e0081"} Jul 14 17:32:57.514167 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:57.514135 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kv2bk" Jul 14 17:32:57.514404 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:32:57.514288 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kv2bk" podUID="ac89d28e-0c57-4be5-8c3a-f0482ce0ba6f" Jul 14 17:32:58.513866 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:58.513712 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9m2f6" Jul 14 17:32:58.514510 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:58.513773 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h8t96" Jul 14 17:32:58.514510 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:32:58.513936 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9m2f6" podUID="5b2d2388-62da-4462-af63-9230c6c43715" Jul 14 17:32:58.514510 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:32:58.514027 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-h8t96" podUID="8321d9c3-c963-45a9-8a94-ef9911a8d33f" Jul 14 17:32:58.647157 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:58.647068 2562 generic.go:356] "Generic (PLEG): container finished" podID="dadd3729-1c46-48d9-92c8-bae9cdbbd1c1" containerID="dac7a3dd583f6e4aaca3b947d7fa67a56de5d7c3f518e530f7d0464afa844220" exitCode=0 Jul 14 17:32:58.647157 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:58.647141 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fjrz4" event={"ID":"dadd3729-1c46-48d9-92c8-bae9cdbbd1c1","Type":"ContainerDied","Data":"dac7a3dd583f6e4aaca3b947d7fa67a56de5d7c3f518e530f7d0464afa844220"} Jul 14 17:32:58.649327 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:58.649301 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7t9rj" event={"ID":"5bca996e-4e3a-4ec6-bd2f-0a5860c9d632","Type":"ContainerStarted","Data":"266d6d8d6a31701d2562b41d8f17b4ac996407e96204b51053f2a5a27175e61f"} Jul 14 17:32:58.654335 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:58.654318 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2sbww_c3d00b23-d0d5-4d18-a722-464d7b114745/ovn-acl-logging/0.log" Jul 14 17:32:58.654817 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:58.654795 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2sbww" event={"ID":"c3d00b23-d0d5-4d18-a722-464d7b114745","Type":"ContainerStarted","Data":"4d4cacb8cee35c0c6cc975bdd326339ea9833d2bf5dfffdeeff28a0231903e41"} Jul 14 17:32:58.655066 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:58.655035 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-2sbww" Jul 14 17:32:58.655153 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:58.655103 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-2sbww" Jul 14 17:32:58.655153 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:58.655139 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-2sbww" Jul 14 17:32:58.655281 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:58.655180 2562 scope.go:117] "RemoveContainer" containerID="22892fd468203608c1f145f7618d540cef929183db47f62f630801f7413cdbd0" Jul 14 17:32:58.671485 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:58.671458 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-2sbww" Jul 14 17:32:58.671803 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:58.671785 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-2sbww" Jul 14 17:32:58.726166 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:58.726125 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7t9rj" podStartSLOduration=3.672550328 podStartE2EDuration="27.726111703s" totalImagesPullingTime="24.053561375s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 17:32:31 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:32:58.725957931 +0000 UTC m=+27.802415469" watchObservedRunningTime="2026-07-14 17:32:58.726111703 +0000 UTC m=+27.802569241" Jul 14 17:32:59.513870 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:59.513803 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kv2bk" Jul 14 17:32:59.513996 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:32:59.513908 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kv2bk" podUID="ac89d28e-0c57-4be5-8c3a-f0482ce0ba6f" Jul 14 17:32:59.660095 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:59.660072 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2sbww_c3d00b23-d0d5-4d18-a722-464d7b114745/ovn-acl-logging/0.log" Jul 14 17:32:59.660464 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:59.660439 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2sbww" event={"ID":"c3d00b23-d0d5-4d18-a722-464d7b114745","Type":"ContainerStarted","Data":"94706b343f1e40ae5a2f80045d1688f03d0bcff06c5ed0da749e1b8477321efe"} Jul 14 17:32:59.693978 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:32:59.693940 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-2sbww" podStartSLOduration=9.062537045 podStartE2EDuration="28.69392686s" totalImagesPullingTime="19.631389815s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 17:32:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:32:59.693102931 +0000 UTC m=+28.769560481" watchObservedRunningTime="2026-07-14 17:32:59.69392686 +0000 UTC m=+28.770384395" Jul 14 17:33:00.514128 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:00.514104 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9m2f6" Jul 14 17:33:00.514513 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:33:00.514228 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9m2f6" podUID="5b2d2388-62da-4462-af63-9230c6c43715" Jul 14 17:33:00.514513 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:00.514278 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h8t96" Jul 14 17:33:00.514513 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:33:00.514405 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-h8t96" podUID="8321d9c3-c963-45a9-8a94-ef9911a8d33f" Jul 14 17:33:00.664579 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:00.664523 2562 generic.go:356] "Generic (PLEG): container finished" podID="dadd3729-1c46-48d9-92c8-bae9cdbbd1c1" containerID="993a635f6823b8c252622ff0ebf03fd6d666d6a8e0c6ad039e2091ac529904c4" exitCode=0 Jul 14 17:33:00.664674 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:00.664609 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fjrz4" event={"ID":"dadd3729-1c46-48d9-92c8-bae9cdbbd1c1","Type":"ContainerDied","Data":"993a635f6823b8c252622ff0ebf03fd6d666d6a8e0c6ad039e2091ac529904c4"} Jul 14 17:33:01.515227 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:01.515178 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kv2bk" Jul 14 17:33:01.515588 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:33:01.515275 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kv2bk" podUID="ac89d28e-0c57-4be5-8c3a-f0482ce0ba6f" Jul 14 17:33:02.513682 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:02.513656 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9m2f6" Jul 14 17:33:02.513829 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:33:02.513741 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9m2f6" podUID="5b2d2388-62da-4462-af63-9230c6c43715" Jul 14 17:33:02.513829 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:02.513661 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h8t96" Jul 14 17:33:02.513829 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:33:02.513810 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-h8t96" podUID="8321d9c3-c963-45a9-8a94-ef9911a8d33f" Jul 14 17:33:03.514420 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:03.514390 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kv2bk" Jul 14 17:33:03.514961 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:33:03.514485 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kv2bk" podUID="ac89d28e-0c57-4be5-8c3a-f0482ce0ba6f" Jul 14 17:33:03.672841 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:03.672807 2562 generic.go:356] "Generic (PLEG): container finished" podID="dadd3729-1c46-48d9-92c8-bae9cdbbd1c1" containerID="d160ab82f2f75026e67293e7c9e61e027089cdbd756e331710ca397ebf1d977c" exitCode=0 Jul 14 17:33:03.672967 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:03.672853 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fjrz4" event={"ID":"dadd3729-1c46-48d9-92c8-bae9cdbbd1c1","Type":"ContainerDied","Data":"d160ab82f2f75026e67293e7c9e61e027089cdbd756e331710ca397ebf1d977c"} Jul 14 17:33:04.514580 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:04.514548 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h8t96" Jul 14 17:33:04.514937 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:04.514586 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9m2f6" Jul 14 17:33:04.514937 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:33:04.514661 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-h8t96" podUID="8321d9c3-c963-45a9-8a94-ef9911a8d33f" Jul 14 17:33:04.514937 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:33:04.514782 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9m2f6" podUID="5b2d2388-62da-4462-af63-9230c6c43715" Jul 14 17:33:05.071395 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:05.071366 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ac89d28e-0c57-4be5-8c3a-f0482ce0ba6f-original-pull-secret\") pod \"global-pull-secret-syncer-kv2bk\" (UID: \"ac89d28e-0c57-4be5-8c3a-f0482ce0ba6f\") " pod="kube-system/global-pull-secret-syncer-kv2bk" Jul 14 17:33:05.071395 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:05.071397 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8321d9c3-c963-45a9-8a94-ef9911a8d33f-metrics-certs\") pod \"network-metrics-daemon-h8t96\" (UID: \"8321d9c3-c963-45a9-8a94-ef9911a8d33f\") " pod="openshift-multus/network-metrics-daemon-h8t96" Jul 14 17:33:05.071623 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:33:05.071500 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 17:33:05.071623 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:33:05.071521 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 14 17:33:05.071623 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:33:05.071563 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8321d9c3-c963-45a9-8a94-ef9911a8d33f-metrics-certs podName:8321d9c3-c963-45a9-8a94-ef9911a8d33f nodeName:}" failed. No retries permitted until 2026-07-14 17:33:37.071544248 +0000 UTC m=+66.148001766 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8321d9c3-c963-45a9-8a94-ef9911a8d33f-metrics-certs") pod "network-metrics-daemon-h8t96" (UID: "8321d9c3-c963-45a9-8a94-ef9911a8d33f") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 14 17:33:05.071623 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:33:05.071583 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ac89d28e-0c57-4be5-8c3a-f0482ce0ba6f-original-pull-secret podName:ac89d28e-0c57-4be5-8c3a-f0482ce0ba6f nodeName:}" failed. No retries permitted until 2026-07-14 17:33:37.071573028 +0000 UTC m=+66.148030550 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ac89d28e-0c57-4be5-8c3a-f0482ce0ba6f-original-pull-secret") pod "global-pull-secret-syncer-kv2bk" (UID: "ac89d28e-0c57-4be5-8c3a-f0482ce0ba6f") : object "kube-system"/"original-pull-secret" not registered Jul 14 17:33:05.172147 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:05.172118 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qqfb\" (UniqueName: \"kubernetes.io/projected/5b2d2388-62da-4462-af63-9230c6c43715-kube-api-access-5qqfb\") pod \"network-check-target-9m2f6\" (UID: \"5b2d2388-62da-4462-af63-9230c6c43715\") " pod="openshift-network-diagnostics/network-check-target-9m2f6" Jul 14 17:33:05.172330 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:33:05.172286 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 14 17:33:05.172330 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:33:05.172305 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 14 17:33:05.172330 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:33:05.172314 2562 projected.go:196] Error preparing data for projected volume kube-api-access-5qqfb for pod openshift-network-diagnostics/network-check-target-9m2f6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 17:33:05.172455 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:33:05.172366 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/5b2d2388-62da-4462-af63-9230c6c43715-kube-api-access-5qqfb podName:5b2d2388-62da-4462-af63-9230c6c43715 nodeName:}" failed. No retries permitted until 2026-07-14 17:33:37.172346931 +0000 UTC m=+66.248804457 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-5qqfb" (UniqueName: "kubernetes.io/projected/5b2d2388-62da-4462-af63-9230c6c43715-kube-api-access-5qqfb") pod "network-check-target-9m2f6" (UID: "5b2d2388-62da-4462-af63-9230c6c43715") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 14 17:33:05.517285 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:05.517209 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kv2bk" Jul 14 17:33:05.517620 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:33:05.517312 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kv2bk" podUID="ac89d28e-0c57-4be5-8c3a-f0482ce0ba6f" Jul 14 17:33:06.513870 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:06.513842 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9m2f6" Jul 14 17:33:06.514057 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:06.513843 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h8t96" Jul 14 17:33:06.514057 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:33:06.513946 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9m2f6" podUID="5b2d2388-62da-4462-af63-9230c6c43715" Jul 14 17:33:06.514057 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:33:06.514013 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-h8t96" podUID="8321d9c3-c963-45a9-8a94-ef9911a8d33f" Jul 14 17:33:07.514092 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:07.514053 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kv2bk" Jul 14 17:33:07.514544 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:33:07.514205 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kv2bk" podUID="ac89d28e-0c57-4be5-8c3a-f0482ce0ba6f" Jul 14 17:33:08.514545 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:08.514509 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9m2f6" Jul 14 17:33:08.514959 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:08.514551 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h8t96" Jul 14 17:33:08.514959 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:33:08.514636 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9m2f6" podUID="5b2d2388-62da-4462-af63-9230c6c43715" Jul 14 17:33:08.514959 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:33:08.514782 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-h8t96" podUID="8321d9c3-c963-45a9-8a94-ef9911a8d33f" Jul 14 17:33:09.514276 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:09.514102 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kv2bk" Jul 14 17:33:09.514410 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:33:09.514389 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kv2bk" podUID="ac89d28e-0c57-4be5-8c3a-f0482ce0ba6f" Jul 14 17:33:09.688959 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:09.688888 2562 generic.go:356] "Generic (PLEG): container finished" podID="dadd3729-1c46-48d9-92c8-bae9cdbbd1c1" containerID="6e7ee6d0eb4ba51f25f554c579a26b3483d147e98f1ca7cd73b73a8bf6977ca5" exitCode=0 Jul 14 17:33:09.688959 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:09.688941 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fjrz4" event={"ID":"dadd3729-1c46-48d9-92c8-bae9cdbbd1c1","Type":"ContainerDied","Data":"6e7ee6d0eb4ba51f25f554c579a26b3483d147e98f1ca7cd73b73a8bf6977ca5"} Jul 14 17:33:10.514203 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:10.514170 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9m2f6" Jul 14 17:33:10.514354 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:10.514175 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h8t96" Jul 14 17:33:10.514354 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:33:10.514271 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9m2f6" podUID="5b2d2388-62da-4462-af63-9230c6c43715" Jul 14 17:33:10.514427 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:33:10.514358 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-h8t96" podUID="8321d9c3-c963-45a9-8a94-ef9911a8d33f" Jul 14 17:33:10.694440 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:10.694413 2562 generic.go:356] "Generic (PLEG): container finished" podID="dadd3729-1c46-48d9-92c8-bae9cdbbd1c1" containerID="a1cec79eec430e32c4d63cf313845b76e22c51e769173cd356f91c98ad43dbb8" exitCode=0 Jul 14 17:33:10.694791 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:10.694480 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fjrz4" event={"ID":"dadd3729-1c46-48d9-92c8-bae9cdbbd1c1","Type":"ContainerDied","Data":"a1cec79eec430e32c4d63cf313845b76e22c51e769173cd356f91c98ad43dbb8"} Jul 14 17:33:11.517201 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:11.517158 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kv2bk" Jul 14 17:33:11.517408 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:33:11.517266 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kv2bk" podUID="ac89d28e-0c57-4be5-8c3a-f0482ce0ba6f" Jul 14 17:33:11.699471 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:11.699436 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-fjrz4" event={"ID":"dadd3729-1c46-48d9-92c8-bae9cdbbd1c1","Type":"ContainerStarted","Data":"7f92740a5b46bc782228595130268c20e698a723a124fca98ae62152ae12cbbe"} Jul 14 17:33:11.733579 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:11.733538 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-fjrz4" podStartSLOduration=7.33483078 podStartE2EDuration="40.733524333s" totalImagesPullingTime="33.398693553s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 17:32:31 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:33:11.732931829 +0000 UTC m=+40.809389367" watchObservedRunningTime="2026-07-14 17:33:11.733524333 +0000 UTC m=+40.809981872" Jul 14 17:33:12.514211 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:12.514158 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9m2f6" Jul 14 17:33:12.514431 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:12.514164 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h8t96" Jul 14 17:33:12.514431 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:33:12.514313 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9m2f6" podUID="5b2d2388-62da-4462-af63-9230c6c43715" Jul 14 17:33:12.514431 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:33:12.514401 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-h8t96" podUID="8321d9c3-c963-45a9-8a94-ef9911a8d33f" Jul 14 17:33:13.514144 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:13.514118 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kv2bk" Jul 14 17:33:13.514704 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:33:13.514251 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kv2bk" podUID="ac89d28e-0c57-4be5-8c3a-f0482ce0ba6f" Jul 14 17:33:13.613630 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:13.613592 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-kv2bk"] Jul 14 17:33:13.616907 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:13.616885 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-h8t96"] Jul 14 17:33:13.617049 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:13.617012 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h8t96" Jul 14 17:33:13.617304 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:33:13.617203 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-h8t96" podUID="8321d9c3-c963-45a9-8a94-ef9911a8d33f" Jul 14 17:33:13.617632 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:13.617609 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-9m2f6"] Jul 14 17:33:13.617733 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:13.617716 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9m2f6" Jul 14 17:33:13.618747 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:33:13.617815 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9m2f6" podUID="5b2d2388-62da-4462-af63-9230c6c43715" Jul 14 17:33:13.704681 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:13.704613 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kv2bk" Jul 14 17:33:13.704799 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:33:13.704706 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kv2bk" podUID="ac89d28e-0c57-4be5-8c3a-f0482ce0ba6f" Jul 14 17:33:15.513751 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:15.513723 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h8t96" Jul 14 17:33:15.514214 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:33:15.513821 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-h8t96" podUID="8321d9c3-c963-45a9-8a94-ef9911a8d33f" Jul 14 17:33:15.514214 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:15.513728 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9m2f6" Jul 14 17:33:15.514214 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:33:15.513881 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-9m2f6" podUID="5b2d2388-62da-4462-af63-9230c6c43715" Jul 14 17:33:15.514214 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:15.513897 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kv2bk" Jul 14 17:33:15.514214 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:33:15.513937 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kv2bk" podUID="ac89d28e-0c57-4be5-8c3a-f0482ce0ba6f" Jul 14 17:33:17.221391 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:17.221212 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-209.ec2.internal" event="NodeReady" Jul 14 17:33:17.221787 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:17.221487 2562 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 14 17:33:17.320354 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:17.320288 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-z4vr6"] Jul 14 17:33:17.335965 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:17.335864 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-z4vr6" Jul 14 17:33:17.336686 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:17.336654 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-z4vr6"] Jul 14 17:33:17.338144 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:17.338125 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-wwng2\"" Jul 14 17:33:17.339167 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:17.339133 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 14 17:33:17.339282 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:17.339259 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 14 17:33:17.342131 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:17.342110 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 14 17:33:17.361642 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:17.361622 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-wjkk8"] Jul 14 17:33:17.375892 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:17.375874 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-wjkk8" Jul 14 17:33:17.375892 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:17.375885 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-8r6jg"] Jul 14 17:33:17.380597 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:17.380572 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 14 17:33:17.380597 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:17.380579 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-wv6hh\"" Jul 14 17:33:17.380786 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:17.380761 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 14 17:33:17.390102 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:17.390031 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-wjkk8"] Jul 14 17:33:17.390222 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:17.390169 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-8r6jg" Jul 14 17:33:17.390831 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:17.390810 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-8r6jg"] Jul 14 17:33:17.392606 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:17.392565 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 14 17:33:17.393359 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:17.393344 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 14 17:33:17.393553 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:17.393527 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 14 17:33:17.393635 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:17.393593 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-tw9bp\"" Jul 14 17:33:17.393700 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:17.393688 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 14 17:33:17.460885 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:17.460860 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mw7dh\" (UniqueName: \"kubernetes.io/projected/ebe882b9-ebc1-4be5-acba-06f13a445bd2-kube-api-access-mw7dh\") pod \"ingress-canary-z4vr6\" (UID: \"ebe882b9-ebc1-4be5-acba-06f13a445bd2\") " pod="openshift-ingress-canary/ingress-canary-z4vr6" Jul 14 17:33:17.460977 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:17.460898 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/95b1cf03-081d-49cb-a74d-9c6e5e165e7d-tmp-dir\") pod \"dns-default-wjkk8\" (UID: \"95b1cf03-081d-49cb-a74d-9c6e5e165e7d\") " pod="openshift-dns/dns-default-wjkk8" Jul 14 17:33:17.460977 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:17.460918 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/95b1cf03-081d-49cb-a74d-9c6e5e165e7d-config-volume\") pod \"dns-default-wjkk8\" (UID: \"95b1cf03-081d-49cb-a74d-9c6e5e165e7d\") " pod="openshift-dns/dns-default-wjkk8" Jul 14 17:33:17.460977 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:17.460951 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lq5lv\" (UniqueName: \"kubernetes.io/projected/95b1cf03-081d-49cb-a74d-9c6e5e165e7d-kube-api-access-lq5lv\") pod \"dns-default-wjkk8\" (UID: \"95b1cf03-081d-49cb-a74d-9c6e5e165e7d\") " pod="openshift-dns/dns-default-wjkk8" Jul 14 17:33:17.461076 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:17.460978 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/95b1cf03-081d-49cb-a74d-9c6e5e165e7d-metrics-tls\") pod \"dns-default-wjkk8\" (UID: \"95b1cf03-081d-49cb-a74d-9c6e5e165e7d\") " pod="openshift-dns/dns-default-wjkk8" Jul 14 17:33:17.461076 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:17.461023 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ebe882b9-ebc1-4be5-acba-06f13a445bd2-cert\") pod \"ingress-canary-z4vr6\" (UID: \"ebe882b9-ebc1-4be5-acba-06f13a445bd2\") " pod="openshift-ingress-canary/ingress-canary-z4vr6" Jul 14 17:33:17.513848 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:17.513823 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h8t96" Jul 14 17:33:17.513941 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:17.513825 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kv2bk" Jul 14 17:33:17.514085 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:17.513822 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9m2f6" Jul 14 17:33:17.516594 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:17.516577 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-79b6j\"" Jul 14 17:33:17.516685 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:17.516603 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 14 17:33:17.516685 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:17.516603 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-q7745\"" Jul 14 17:33:17.516800 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:17.516593 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 14 17:33:17.517155 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:17.517133 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 14 17:33:17.517342 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:17.517303 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 14 17:33:17.561655 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:17.561633 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ebe882b9-ebc1-4be5-acba-06f13a445bd2-cert\") pod \"ingress-canary-z4vr6\" (UID: \"ebe882b9-ebc1-4be5-acba-06f13a445bd2\") " pod="openshift-ingress-canary/ingress-canary-z4vr6" Jul 14 17:33:17.561749 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:17.561689 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cgzgc\" (UniqueName: \"kubernetes.io/projected/7635e71d-e195-4ef9-8eff-980969d9f360-kube-api-access-cgzgc\") pod \"insights-runtime-extractor-8r6jg\" (UID: \"7635e71d-e195-4ef9-8eff-980969d9f360\") " pod="openshift-insights/insights-runtime-extractor-8r6jg" Jul 14 17:33:17.561749 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:17.561737 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7635e71d-e195-4ef9-8eff-980969d9f360-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8r6jg\" (UID: \"7635e71d-e195-4ef9-8eff-980969d9f360\") " pod="openshift-insights/insights-runtime-extractor-8r6jg" Jul 14 17:33:17.561857 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:17.561840 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mw7dh\" (UniqueName: \"kubernetes.io/projected/ebe882b9-ebc1-4be5-acba-06f13a445bd2-kube-api-access-mw7dh\") pod \"ingress-canary-z4vr6\" (UID: \"ebe882b9-ebc1-4be5-acba-06f13a445bd2\") " pod="openshift-ingress-canary/ingress-canary-z4vr6" Jul 14 17:33:17.561913 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:17.561881 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7635e71d-e195-4ef9-8eff-980969d9f360-crio-socket\") pod \"insights-runtime-extractor-8r6jg\" (UID: \"7635e71d-e195-4ef9-8eff-980969d9f360\") " pod="openshift-insights/insights-runtime-extractor-8r6jg" Jul 14 17:33:17.561965 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:17.561915 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7635e71d-e195-4ef9-8eff-980969d9f360-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8r6jg\" (UID: \"7635e71d-e195-4ef9-8eff-980969d9f360\") " pod="openshift-insights/insights-runtime-extractor-8r6jg" Jul 14 17:33:17.561965 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:17.561954 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/95b1cf03-081d-49cb-a74d-9c6e5e165e7d-tmp-dir\") pod \"dns-default-wjkk8\" (UID: \"95b1cf03-081d-49cb-a74d-9c6e5e165e7d\") " pod="openshift-dns/dns-default-wjkk8" Jul 14 17:33:17.562076 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:17.561985 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/95b1cf03-081d-49cb-a74d-9c6e5e165e7d-config-volume\") pod \"dns-default-wjkk8\" (UID: \"95b1cf03-081d-49cb-a74d-9c6e5e165e7d\") " pod="openshift-dns/dns-default-wjkk8" Jul 14 17:33:17.562076 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:17.562015 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lq5lv\" (UniqueName: \"kubernetes.io/projected/95b1cf03-081d-49cb-a74d-9c6e5e165e7d-kube-api-access-lq5lv\") pod \"dns-default-wjkk8\" (UID: \"95b1cf03-081d-49cb-a74d-9c6e5e165e7d\") " pod="openshift-dns/dns-default-wjkk8" Jul 14 17:33:17.562076 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:17.562047 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7635e71d-e195-4ef9-8eff-980969d9f360-data-volume\") pod \"insights-runtime-extractor-8r6jg\" (UID: \"7635e71d-e195-4ef9-8eff-980969d9f360\") " pod="openshift-insights/insights-runtime-extractor-8r6jg" Jul 14 17:33:17.562235 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:17.562073 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/95b1cf03-081d-49cb-a74d-9c6e5e165e7d-metrics-tls\") pod \"dns-default-wjkk8\" (UID: \"95b1cf03-081d-49cb-a74d-9c6e5e165e7d\") " pod="openshift-dns/dns-default-wjkk8" Jul 14 17:33:17.562294 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:17.562272 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/95b1cf03-081d-49cb-a74d-9c6e5e165e7d-tmp-dir\") pod \"dns-default-wjkk8\" (UID: \"95b1cf03-081d-49cb-a74d-9c6e5e165e7d\") " pod="openshift-dns/dns-default-wjkk8" Jul 14 17:33:17.562547 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:17.562529 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/95b1cf03-081d-49cb-a74d-9c6e5e165e7d-config-volume\") pod \"dns-default-wjkk8\" (UID: \"95b1cf03-081d-49cb-a74d-9c6e5e165e7d\") " pod="openshift-dns/dns-default-wjkk8" Jul 14 17:33:17.565823 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:17.565804 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/95b1cf03-081d-49cb-a74d-9c6e5e165e7d-metrics-tls\") pod \"dns-default-wjkk8\" (UID: \"95b1cf03-081d-49cb-a74d-9c6e5e165e7d\") " pod="openshift-dns/dns-default-wjkk8" Jul 14 17:33:17.565915 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:17.565850 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ebe882b9-ebc1-4be5-acba-06f13a445bd2-cert\") pod \"ingress-canary-z4vr6\" (UID: \"ebe882b9-ebc1-4be5-acba-06f13a445bd2\") " pod="openshift-ingress-canary/ingress-canary-z4vr6" Jul 14 17:33:17.570687 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:17.570614 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lq5lv\" (UniqueName: \"kubernetes.io/projected/95b1cf03-081d-49cb-a74d-9c6e5e165e7d-kube-api-access-lq5lv\") pod \"dns-default-wjkk8\" (UID: \"95b1cf03-081d-49cb-a74d-9c6e5e165e7d\") " pod="openshift-dns/dns-default-wjkk8" Jul 14 17:33:17.570908 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:17.570890 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mw7dh\" (UniqueName: \"kubernetes.io/projected/ebe882b9-ebc1-4be5-acba-06f13a445bd2-kube-api-access-mw7dh\") pod \"ingress-canary-z4vr6\" (UID: \"ebe882b9-ebc1-4be5-acba-06f13a445bd2\") " pod="openshift-ingress-canary/ingress-canary-z4vr6" Jul 14 17:33:17.646848 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:17.646823 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-z4vr6" Jul 14 17:33:17.663163 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:17.663139 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7635e71d-e195-4ef9-8eff-980969d9f360-data-volume\") pod \"insights-runtime-extractor-8r6jg\" (UID: \"7635e71d-e195-4ef9-8eff-980969d9f360\") " pod="openshift-insights/insights-runtime-extractor-8r6jg" Jul 14 17:33:17.663278 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:17.663224 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cgzgc\" (UniqueName: \"kubernetes.io/projected/7635e71d-e195-4ef9-8eff-980969d9f360-kube-api-access-cgzgc\") pod \"insights-runtime-extractor-8r6jg\" (UID: \"7635e71d-e195-4ef9-8eff-980969d9f360\") " pod="openshift-insights/insights-runtime-extractor-8r6jg" Jul 14 17:33:17.663278 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:17.663256 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7635e71d-e195-4ef9-8eff-980969d9f360-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8r6jg\" (UID: \"7635e71d-e195-4ef9-8eff-980969d9f360\") " pod="openshift-insights/insights-runtime-extractor-8r6jg" Jul 14 17:33:17.663384 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:17.663280 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7635e71d-e195-4ef9-8eff-980969d9f360-crio-socket\") pod \"insights-runtime-extractor-8r6jg\" (UID: \"7635e71d-e195-4ef9-8eff-980969d9f360\") " pod="openshift-insights/insights-runtime-extractor-8r6jg" Jul 14 17:33:17.663384 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:17.663299 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7635e71d-e195-4ef9-8eff-980969d9f360-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8r6jg\" (UID: \"7635e71d-e195-4ef9-8eff-980969d9f360\") " pod="openshift-insights/insights-runtime-extractor-8r6jg" Jul 14 17:33:17.663531 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:17.663505 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7635e71d-e195-4ef9-8eff-980969d9f360-crio-socket\") pod \"insights-runtime-extractor-8r6jg\" (UID: \"7635e71d-e195-4ef9-8eff-980969d9f360\") " pod="openshift-insights/insights-runtime-extractor-8r6jg" Jul 14 17:33:17.663586 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:17.663539 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7635e71d-e195-4ef9-8eff-980969d9f360-data-volume\") pod \"insights-runtime-extractor-8r6jg\" (UID: \"7635e71d-e195-4ef9-8eff-980969d9f360\") " pod="openshift-insights/insights-runtime-extractor-8r6jg" Jul 14 17:33:17.663816 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:17.663798 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7635e71d-e195-4ef9-8eff-980969d9f360-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8r6jg\" (UID: \"7635e71d-e195-4ef9-8eff-980969d9f360\") " pod="openshift-insights/insights-runtime-extractor-8r6jg" Jul 14 17:33:17.665675 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:17.665656 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7635e71d-e195-4ef9-8eff-980969d9f360-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8r6jg\" (UID: \"7635e71d-e195-4ef9-8eff-980969d9f360\") " pod="openshift-insights/insights-runtime-extractor-8r6jg" Jul 14 17:33:17.671633 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:17.671605 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cgzgc\" (UniqueName: \"kubernetes.io/projected/7635e71d-e195-4ef9-8eff-980969d9f360-kube-api-access-cgzgc\") pod \"insights-runtime-extractor-8r6jg\" (UID: \"7635e71d-e195-4ef9-8eff-980969d9f360\") " pod="openshift-insights/insights-runtime-extractor-8r6jg" Jul 14 17:33:17.684511 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:17.684476 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-wjkk8" Jul 14 17:33:17.698798 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:17.698771 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-8r6jg" Jul 14 17:33:17.854049 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:17.854014 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-wjkk8"] Jul 14 17:33:17.856295 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:17.856272 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-z4vr6"] Jul 14 17:33:17.857910 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:33:17.857867 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95b1cf03_081d_49cb_a74d_9c6e5e165e7d.slice/crio-8f2759413a5f27a46601ce0d2df7fbd9f916e26171046741984a20923027c124 WatchSource:0}: Error finding container 8f2759413a5f27a46601ce0d2df7fbd9f916e26171046741984a20923027c124: Status 404 returned error can't find the container with id 8f2759413a5f27a46601ce0d2df7fbd9f916e26171046741984a20923027c124 Jul 14 17:33:17.861278 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:33:17.861254 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podebe882b9_ebc1_4be5_acba_06f13a445bd2.slice/crio-2a19be540d500900c309d148c97b7ed3190ea332a6543ea2c7b4a4bfe026f6c5 WatchSource:0}: Error finding container 2a19be540d500900c309d148c97b7ed3190ea332a6543ea2c7b4a4bfe026f6c5: Status 404 returned error can't find the container with id 2a19be540d500900c309d148c97b7ed3190ea332a6543ea2c7b4a4bfe026f6c5 Jul 14 17:33:17.874598 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:17.874573 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-8r6jg"] Jul 14 17:33:17.879248 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:33:17.879215 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7635e71d_e195_4ef9_8eff_980969d9f360.slice/crio-80710d807b4e6353027f4892e3c96ec213ea63ca516fd7236ae544cbe8f9ae3c WatchSource:0}: Error finding container 80710d807b4e6353027f4892e3c96ec213ea63ca516fd7236ae544cbe8f9ae3c: Status 404 returned error can't find the container with id 80710d807b4e6353027f4892e3c96ec213ea63ca516fd7236ae544cbe8f9ae3c Jul 14 17:33:18.491744 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:18.490747 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-8669f6cd78-rjntn"] Jul 14 17:33:18.509078 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:18.509037 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8669f6cd78-rjntn"] Jul 14 17:33:18.509268 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:18.509152 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8669f6cd78-rjntn" Jul 14 17:33:18.512005 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:18.511984 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 14 17:33:18.512005 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:18.511995 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 14 17:33:18.512450 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:18.512269 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 14 17:33:18.514351 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:18.513753 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 14 17:33:18.514351 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:18.513800 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 14 17:33:18.514351 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:18.513992 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 14 17:33:18.514351 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:18.514322 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 14 17:33:18.514604 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:18.514593 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-krthk\"" Jul 14 17:33:18.673550 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:18.673168 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b56ef57c-4634-40df-a68d-e4ae43d07c3d-console-oauth-config\") pod \"console-8669f6cd78-rjntn\" (UID: \"b56ef57c-4634-40df-a68d-e4ae43d07c3d\") " pod="openshift-console/console-8669f6cd78-rjntn" Jul 14 17:33:18.673550 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:18.673235 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b56ef57c-4634-40df-a68d-e4ae43d07c3d-console-config\") pod \"console-8669f6cd78-rjntn\" (UID: \"b56ef57c-4634-40df-a68d-e4ae43d07c3d\") " pod="openshift-console/console-8669f6cd78-rjntn" Jul 14 17:33:18.673550 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:18.673260 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b56ef57c-4634-40df-a68d-e4ae43d07c3d-service-ca\") pod \"console-8669f6cd78-rjntn\" (UID: \"b56ef57c-4634-40df-a68d-e4ae43d07c3d\") " pod="openshift-console/console-8669f6cd78-rjntn" Jul 14 17:33:18.673550 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:18.673285 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b56ef57c-4634-40df-a68d-e4ae43d07c3d-oauth-serving-cert\") pod \"console-8669f6cd78-rjntn\" (UID: \"b56ef57c-4634-40df-a68d-e4ae43d07c3d\") " pod="openshift-console/console-8669f6cd78-rjntn" Jul 14 17:33:18.673550 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:18.673348 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b56ef57c-4634-40df-a68d-e4ae43d07c3d-console-serving-cert\") pod \"console-8669f6cd78-rjntn\" (UID: \"b56ef57c-4634-40df-a68d-e4ae43d07c3d\") " pod="openshift-console/console-8669f6cd78-rjntn" Jul 14 17:33:18.673550 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:18.673389 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7dlbm\" (UniqueName: \"kubernetes.io/projected/b56ef57c-4634-40df-a68d-e4ae43d07c3d-kube-api-access-7dlbm\") pod \"console-8669f6cd78-rjntn\" (UID: \"b56ef57c-4634-40df-a68d-e4ae43d07c3d\") " pod="openshift-console/console-8669f6cd78-rjntn" Jul 14 17:33:18.717681 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:18.717586 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8r6jg" event={"ID":"7635e71d-e195-4ef9-8eff-980969d9f360","Type":"ContainerStarted","Data":"80710d807b4e6353027f4892e3c96ec213ea63ca516fd7236ae544cbe8f9ae3c"} Jul 14 17:33:18.718956 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:18.718888 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-wjkk8" event={"ID":"95b1cf03-081d-49cb-a74d-9c6e5e165e7d","Type":"ContainerStarted","Data":"8f2759413a5f27a46601ce0d2df7fbd9f916e26171046741984a20923027c124"} Jul 14 17:33:18.722541 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:18.722414 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-z4vr6" event={"ID":"ebe882b9-ebc1-4be5-acba-06f13a445bd2","Type":"ContainerStarted","Data":"2a19be540d500900c309d148c97b7ed3190ea332a6543ea2c7b4a4bfe026f6c5"} Jul 14 17:33:18.774531 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:18.774455 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b56ef57c-4634-40df-a68d-e4ae43d07c3d-console-oauth-config\") pod \"console-8669f6cd78-rjntn\" (UID: \"b56ef57c-4634-40df-a68d-e4ae43d07c3d\") " pod="openshift-console/console-8669f6cd78-rjntn" Jul 14 17:33:18.774531 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:18.774517 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b56ef57c-4634-40df-a68d-e4ae43d07c3d-console-config\") pod \"console-8669f6cd78-rjntn\" (UID: \"b56ef57c-4634-40df-a68d-e4ae43d07c3d\") " pod="openshift-console/console-8669f6cd78-rjntn" Jul 14 17:33:18.774855 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:18.774540 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b56ef57c-4634-40df-a68d-e4ae43d07c3d-service-ca\") pod \"console-8669f6cd78-rjntn\" (UID: \"b56ef57c-4634-40df-a68d-e4ae43d07c3d\") " pod="openshift-console/console-8669f6cd78-rjntn" Jul 14 17:33:18.774855 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:18.774566 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b56ef57c-4634-40df-a68d-e4ae43d07c3d-oauth-serving-cert\") pod \"console-8669f6cd78-rjntn\" (UID: \"b56ef57c-4634-40df-a68d-e4ae43d07c3d\") " pod="openshift-console/console-8669f6cd78-rjntn" Jul 14 17:33:18.774855 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:18.774628 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b56ef57c-4634-40df-a68d-e4ae43d07c3d-console-serving-cert\") pod \"console-8669f6cd78-rjntn\" (UID: \"b56ef57c-4634-40df-a68d-e4ae43d07c3d\") " pod="openshift-console/console-8669f6cd78-rjntn" Jul 14 17:33:18.774855 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:18.774664 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7dlbm\" (UniqueName: \"kubernetes.io/projected/b56ef57c-4634-40df-a68d-e4ae43d07c3d-kube-api-access-7dlbm\") pod \"console-8669f6cd78-rjntn\" (UID: \"b56ef57c-4634-40df-a68d-e4ae43d07c3d\") " pod="openshift-console/console-8669f6cd78-rjntn" Jul 14 17:33:18.776302 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:18.776276 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b56ef57c-4634-40df-a68d-e4ae43d07c3d-service-ca\") pod \"console-8669f6cd78-rjntn\" (UID: \"b56ef57c-4634-40df-a68d-e4ae43d07c3d\") " pod="openshift-console/console-8669f6cd78-rjntn" Jul 14 17:33:18.777033 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:18.777013 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b56ef57c-4634-40df-a68d-e4ae43d07c3d-console-config\") pod \"console-8669f6cd78-rjntn\" (UID: \"b56ef57c-4634-40df-a68d-e4ae43d07c3d\") " pod="openshift-console/console-8669f6cd78-rjntn" Jul 14 17:33:18.777922 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:18.777577 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b56ef57c-4634-40df-a68d-e4ae43d07c3d-oauth-serving-cert\") pod \"console-8669f6cd78-rjntn\" (UID: \"b56ef57c-4634-40df-a68d-e4ae43d07c3d\") " pod="openshift-console/console-8669f6cd78-rjntn" Jul 14 17:33:18.787215 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:18.787147 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b56ef57c-4634-40df-a68d-e4ae43d07c3d-console-oauth-config\") pod \"console-8669f6cd78-rjntn\" (UID: \"b56ef57c-4634-40df-a68d-e4ae43d07c3d\") " pod="openshift-console/console-8669f6cd78-rjntn" Jul 14 17:33:18.788218 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:18.788143 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b56ef57c-4634-40df-a68d-e4ae43d07c3d-console-serving-cert\") pod \"console-8669f6cd78-rjntn\" (UID: \"b56ef57c-4634-40df-a68d-e4ae43d07c3d\") " pod="openshift-console/console-8669f6cd78-rjntn" Jul 14 17:33:18.792401 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:18.792351 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7dlbm\" (UniqueName: \"kubernetes.io/projected/b56ef57c-4634-40df-a68d-e4ae43d07c3d-kube-api-access-7dlbm\") pod \"console-8669f6cd78-rjntn\" (UID: \"b56ef57c-4634-40df-a68d-e4ae43d07c3d\") " pod="openshift-console/console-8669f6cd78-rjntn" Jul 14 17:33:18.820960 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:18.820788 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8669f6cd78-rjntn" Jul 14 17:33:19.058798 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:19.058684 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8669f6cd78-rjntn"] Jul 14 17:33:19.065066 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:33:19.065029 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb56ef57c_4634_40df_a68d_e4ae43d07c3d.slice/crio-a2f12cc32c4904ada497dae43f4f2b4a40a0273f5a5b566d47b033a9ea04c155 WatchSource:0}: Error finding container a2f12cc32c4904ada497dae43f4f2b4a40a0273f5a5b566d47b033a9ea04c155: Status 404 returned error can't find the container with id a2f12cc32c4904ada497dae43f4f2b4a40a0273f5a5b566d47b033a9ea04c155 Jul 14 17:33:19.727569 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:19.727275 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8669f6cd78-rjntn" event={"ID":"b56ef57c-4634-40df-a68d-e4ae43d07c3d","Type":"ContainerStarted","Data":"a2f12cc32c4904ada497dae43f4f2b4a40a0273f5a5b566d47b033a9ea04c155"} Jul 14 17:33:20.744550 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:20.744524 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7944586585-9b8sh"] Jul 14 17:33:20.771899 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:20.771869 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7944586585-9b8sh"] Jul 14 17:33:20.772067 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:20.771998 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7944586585-9b8sh" Jul 14 17:33:20.799304 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:20.799277 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 14 17:33:20.893790 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:20.893750 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-djx6r\" (UniqueName: \"kubernetes.io/projected/1c51ee65-7247-4790-885c-304ae178695c-kube-api-access-djx6r\") pod \"console-7944586585-9b8sh\" (UID: \"1c51ee65-7247-4790-885c-304ae178695c\") " pod="openshift-console/console-7944586585-9b8sh" Jul 14 17:33:20.893947 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:20.893810 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1c51ee65-7247-4790-885c-304ae178695c-oauth-serving-cert\") pod \"console-7944586585-9b8sh\" (UID: \"1c51ee65-7247-4790-885c-304ae178695c\") " pod="openshift-console/console-7944586585-9b8sh" Jul 14 17:33:20.893947 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:20.893889 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1c51ee65-7247-4790-885c-304ae178695c-trusted-ca-bundle\") pod \"console-7944586585-9b8sh\" (UID: \"1c51ee65-7247-4790-885c-304ae178695c\") " pod="openshift-console/console-7944586585-9b8sh" Jul 14 17:33:20.893947 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:20.893928 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1c51ee65-7247-4790-885c-304ae178695c-console-serving-cert\") pod \"console-7944586585-9b8sh\" (UID: \"1c51ee65-7247-4790-885c-304ae178695c\") " pod="openshift-console/console-7944586585-9b8sh" Jul 14 17:33:20.893947 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:20.893947 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1c51ee65-7247-4790-885c-304ae178695c-console-oauth-config\") pod \"console-7944586585-9b8sh\" (UID: \"1c51ee65-7247-4790-885c-304ae178695c\") " pod="openshift-console/console-7944586585-9b8sh" Jul 14 17:33:20.894204 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:20.894011 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1c51ee65-7247-4790-885c-304ae178695c-console-config\") pod \"console-7944586585-9b8sh\" (UID: \"1c51ee65-7247-4790-885c-304ae178695c\") " pod="openshift-console/console-7944586585-9b8sh" Jul 14 17:33:20.894204 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:20.894072 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1c51ee65-7247-4790-885c-304ae178695c-service-ca\") pod \"console-7944586585-9b8sh\" (UID: \"1c51ee65-7247-4790-885c-304ae178695c\") " pod="openshift-console/console-7944586585-9b8sh" Jul 14 17:33:20.995151 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:20.995063 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1c51ee65-7247-4790-885c-304ae178695c-service-ca\") pod \"console-7944586585-9b8sh\" (UID: \"1c51ee65-7247-4790-885c-304ae178695c\") " pod="openshift-console/console-7944586585-9b8sh" Jul 14 17:33:20.995151 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:20.995115 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-djx6r\" (UniqueName: \"kubernetes.io/projected/1c51ee65-7247-4790-885c-304ae178695c-kube-api-access-djx6r\") pod \"console-7944586585-9b8sh\" (UID: \"1c51ee65-7247-4790-885c-304ae178695c\") " pod="openshift-console/console-7944586585-9b8sh" Jul 14 17:33:20.995384 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:20.995154 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1c51ee65-7247-4790-885c-304ae178695c-oauth-serving-cert\") pod \"console-7944586585-9b8sh\" (UID: \"1c51ee65-7247-4790-885c-304ae178695c\") " pod="openshift-console/console-7944586585-9b8sh" Jul 14 17:33:20.995384 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:20.995231 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1c51ee65-7247-4790-885c-304ae178695c-trusted-ca-bundle\") pod \"console-7944586585-9b8sh\" (UID: \"1c51ee65-7247-4790-885c-304ae178695c\") " pod="openshift-console/console-7944586585-9b8sh" Jul 14 17:33:20.995384 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:20.995253 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1c51ee65-7247-4790-885c-304ae178695c-console-serving-cert\") pod \"console-7944586585-9b8sh\" (UID: \"1c51ee65-7247-4790-885c-304ae178695c\") " pod="openshift-console/console-7944586585-9b8sh" Jul 14 17:33:20.995384 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:20.995274 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1c51ee65-7247-4790-885c-304ae178695c-console-oauth-config\") pod \"console-7944586585-9b8sh\" (UID: \"1c51ee65-7247-4790-885c-304ae178695c\") " pod="openshift-console/console-7944586585-9b8sh" Jul 14 17:33:20.995384 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:20.995302 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1c51ee65-7247-4790-885c-304ae178695c-console-config\") pod \"console-7944586585-9b8sh\" (UID: \"1c51ee65-7247-4790-885c-304ae178695c\") " pod="openshift-console/console-7944586585-9b8sh" Jul 14 17:33:20.995871 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:20.995847 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1c51ee65-7247-4790-885c-304ae178695c-console-config\") pod \"console-7944586585-9b8sh\" (UID: \"1c51ee65-7247-4790-885c-304ae178695c\") " pod="openshift-console/console-7944586585-9b8sh" Jul 14 17:33:20.995959 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:20.995913 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1c51ee65-7247-4790-885c-304ae178695c-service-ca\") pod \"console-7944586585-9b8sh\" (UID: \"1c51ee65-7247-4790-885c-304ae178695c\") " pod="openshift-console/console-7944586585-9b8sh" Jul 14 17:33:20.996095 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:20.996074 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1c51ee65-7247-4790-885c-304ae178695c-oauth-serving-cert\") pod \"console-7944586585-9b8sh\" (UID: \"1c51ee65-7247-4790-885c-304ae178695c\") " pod="openshift-console/console-7944586585-9b8sh" Jul 14 17:33:20.996337 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:20.996302 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1c51ee65-7247-4790-885c-304ae178695c-trusted-ca-bundle\") pod \"console-7944586585-9b8sh\" (UID: \"1c51ee65-7247-4790-885c-304ae178695c\") " pod="openshift-console/console-7944586585-9b8sh" Jul 14 17:33:20.998943 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:20.998919 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1c51ee65-7247-4790-885c-304ae178695c-console-serving-cert\") pod \"console-7944586585-9b8sh\" (UID: \"1c51ee65-7247-4790-885c-304ae178695c\") " pod="openshift-console/console-7944586585-9b8sh" Jul 14 17:33:20.999018 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:20.998964 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1c51ee65-7247-4790-885c-304ae178695c-console-oauth-config\") pod \"console-7944586585-9b8sh\" (UID: \"1c51ee65-7247-4790-885c-304ae178695c\") " pod="openshift-console/console-7944586585-9b8sh" Jul 14 17:33:21.009564 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:21.009540 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-djx6r\" (UniqueName: \"kubernetes.io/projected/1c51ee65-7247-4790-885c-304ae178695c-kube-api-access-djx6r\") pod \"console-7944586585-9b8sh\" (UID: \"1c51ee65-7247-4790-885c-304ae178695c\") " pod="openshift-console/console-7944586585-9b8sh" Jul 14 17:33:21.083517 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:21.083420 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7944586585-9b8sh" Jul 14 17:33:25.243112 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:25.242888 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7944586585-9b8sh"] Jul 14 17:33:25.248831 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:33:25.248722 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c51ee65_7247_4790_885c_304ae178695c.slice/crio-3e11060635018417ba818105204e2e7ef8583606cdb954358ca53c822f456107 WatchSource:0}: Error finding container 3e11060635018417ba818105204e2e7ef8583606cdb954358ca53c822f456107: Status 404 returned error can't find the container with id 3e11060635018417ba818105204e2e7ef8583606cdb954358ca53c822f456107 Jul 14 17:33:25.747458 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:25.747420 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7944586585-9b8sh" event={"ID":"1c51ee65-7247-4790-885c-304ae178695c","Type":"ContainerStarted","Data":"f7bb6ab7de848d6a44dc4d2b81309303532a72d9101b17cc39fbee177f80ecee"} Jul 14 17:33:25.747645 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:25.747501 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7944586585-9b8sh" event={"ID":"1c51ee65-7247-4790-885c-304ae178695c","Type":"ContainerStarted","Data":"3e11060635018417ba818105204e2e7ef8583606cdb954358ca53c822f456107"} Jul 14 17:33:25.750085 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:25.750057 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-wjkk8" event={"ID":"95b1cf03-081d-49cb-a74d-9c6e5e165e7d","Type":"ContainerStarted","Data":"b00fc42bcaf6e0156e2ce8dfa868d39b030e08d5e3c583aca06eb2a41ceaa125"} Jul 14 17:33:25.750179 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:25.750101 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-wjkk8" event={"ID":"95b1cf03-081d-49cb-a74d-9c6e5e165e7d","Type":"ContainerStarted","Data":"13e6a653be034eadfaf43e01a9d00141b3fcde95a75257c8bd5c14bb9e213278"} Jul 14 17:33:25.750593 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:25.750576 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-wjkk8" Jul 14 17:33:25.752215 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:25.752172 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-z4vr6" event={"ID":"ebe882b9-ebc1-4be5-acba-06f13a445bd2","Type":"ContainerStarted","Data":"4bb483b9c49014b48c74a8dc8bebbecbbea86937c096c5a763fbf140adc04db2"} Jul 14 17:33:25.753725 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:25.753707 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8r6jg" event={"ID":"7635e71d-e195-4ef9-8eff-980969d9f360","Type":"ContainerStarted","Data":"cdf36a61759dddb3372c5ff8570ff3040e922005e8cb62585cb8321008e3c108"} Jul 14 17:33:25.755081 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:25.755062 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8669f6cd78-rjntn" event={"ID":"b56ef57c-4634-40df-a68d-e4ae43d07c3d","Type":"ContainerStarted","Data":"e4ed7b64c09216a66ce320bd69b5427bf77fb7bf1f11d83d617bf3b41105aa56"} Jul 14 17:33:25.766522 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:25.766481 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7944586585-9b8sh" podStartSLOduration=5.7664698130000005 podStartE2EDuration="5.766469813s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:33:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:33:25.766134726 +0000 UTC m=+54.842592265" watchObservedRunningTime="2026-07-14 17:33:25.766469813 +0000 UTC m=+54.842927329" Jul 14 17:33:25.785645 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:25.785598 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-z4vr6" podStartSLOduration=1.493365398 podStartE2EDuration="8.785580274s" totalImagesPullingTime="7.292214876s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:33:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:33:25.784867762 +0000 UTC m=+54.861325302" watchObservedRunningTime="2026-07-14 17:33:25.785580274 +0000 UTC m=+54.862037813" Jul 14 17:33:25.807668 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:25.807620 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-wjkk8" podStartSLOduration=1.5987503429999999 podStartE2EDuration="8.807603131s" totalImagesPullingTime="7.208852788s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:33:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:33:25.80663641 +0000 UTC m=+54.883093962" watchObservedRunningTime="2026-07-14 17:33:25.807603131 +0000 UTC m=+54.884060674" Jul 14 17:33:25.828524 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:25.828479 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-8669f6cd78-rjntn" podStartSLOduration=1.72108199 podStartE2EDuration="7.828462555s" totalImagesPullingTime="6.107380565s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:33:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:33:25.82729818 +0000 UTC m=+54.903755719" watchObservedRunningTime="2026-07-14 17:33:25.828462555 +0000 UTC m=+54.904920094" Jul 14 17:33:26.760559 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:26.760509 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8r6jg" event={"ID":"7635e71d-e195-4ef9-8eff-980969d9f360","Type":"ContainerStarted","Data":"8b4bdd7658a751f88cd6ee900f6c762567029e7ea9225cb1185b86ab95e42f68"} Jul 14 17:33:28.821209 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:28.821148 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-8669f6cd78-rjntn" Jul 14 17:33:28.821715 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:28.821350 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-8669f6cd78-rjntn" Jul 14 17:33:28.826495 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:28.826420 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-8669f6cd78-rjntn" Jul 14 17:33:29.190067 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:29.190042 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-bwhfp"] Jul 14 17:33:29.212598 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:29.212571 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-bwhfp" Jul 14 17:33:29.214889 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:29.214866 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 14 17:33:29.220977 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:29.220948 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 14 17:33:29.221090 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:29.220954 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 14 17:33:29.221731 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:29.221716 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 14 17:33:29.222363 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:29.222348 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 14 17:33:29.222465 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:29.222450 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 14 17:33:29.228812 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:29.228793 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-bq24s\"" Jul 14 17:33:29.362051 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:29.361958 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ba05a4f0-94e4-4462-9bc4-7d0d7cc8e230-node-exporter-textfile\") pod \"node-exporter-bwhfp\" (UID: \"ba05a4f0-94e4-4462-9bc4-7d0d7cc8e230\") " pod="openshift-monitoring/node-exporter-bwhfp" Jul 14 17:33:29.362051 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:29.361992 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sptzc\" (UniqueName: \"kubernetes.io/projected/ba05a4f0-94e4-4462-9bc4-7d0d7cc8e230-kube-api-access-sptzc\") pod \"node-exporter-bwhfp\" (UID: \"ba05a4f0-94e4-4462-9bc4-7d0d7cc8e230\") " pod="openshift-monitoring/node-exporter-bwhfp" Jul 14 17:33:29.362051 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:29.362033 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ba05a4f0-94e4-4462-9bc4-7d0d7cc8e230-node-exporter-tls\") pod \"node-exporter-bwhfp\" (UID: \"ba05a4f0-94e4-4462-9bc4-7d0d7cc8e230\") " pod="openshift-monitoring/node-exporter-bwhfp" Jul 14 17:33:29.362051 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:29.362050 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ba05a4f0-94e4-4462-9bc4-7d0d7cc8e230-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-bwhfp\" (UID: \"ba05a4f0-94e4-4462-9bc4-7d0d7cc8e230\") " pod="openshift-monitoring/node-exporter-bwhfp" Jul 14 17:33:29.362341 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:29.362120 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ba05a4f0-94e4-4462-9bc4-7d0d7cc8e230-sys\") pod \"node-exporter-bwhfp\" (UID: \"ba05a4f0-94e4-4462-9bc4-7d0d7cc8e230\") " pod="openshift-monitoring/node-exporter-bwhfp" Jul 14 17:33:29.362341 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:29.362158 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ba05a4f0-94e4-4462-9bc4-7d0d7cc8e230-node-exporter-accelerators-collector-config\") pod \"node-exporter-bwhfp\" (UID: \"ba05a4f0-94e4-4462-9bc4-7d0d7cc8e230\") " pod="openshift-monitoring/node-exporter-bwhfp" Jul 14 17:33:29.362341 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:29.362203 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ba05a4f0-94e4-4462-9bc4-7d0d7cc8e230-root\") pod \"node-exporter-bwhfp\" (UID: \"ba05a4f0-94e4-4462-9bc4-7d0d7cc8e230\") " pod="openshift-monitoring/node-exporter-bwhfp" Jul 14 17:33:29.362341 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:29.362253 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ba05a4f0-94e4-4462-9bc4-7d0d7cc8e230-node-exporter-wtmp\") pod \"node-exporter-bwhfp\" (UID: \"ba05a4f0-94e4-4462-9bc4-7d0d7cc8e230\") " pod="openshift-monitoring/node-exporter-bwhfp" Jul 14 17:33:29.362341 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:29.362323 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ba05a4f0-94e4-4462-9bc4-7d0d7cc8e230-metrics-client-ca\") pod \"node-exporter-bwhfp\" (UID: \"ba05a4f0-94e4-4462-9bc4-7d0d7cc8e230\") " pod="openshift-monitoring/node-exporter-bwhfp" Jul 14 17:33:29.463540 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:29.463490 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ba05a4f0-94e4-4462-9bc4-7d0d7cc8e230-node-exporter-textfile\") pod \"node-exporter-bwhfp\" (UID: \"ba05a4f0-94e4-4462-9bc4-7d0d7cc8e230\") " pod="openshift-monitoring/node-exporter-bwhfp" Jul 14 17:33:29.463540 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:29.463548 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sptzc\" (UniqueName: \"kubernetes.io/projected/ba05a4f0-94e4-4462-9bc4-7d0d7cc8e230-kube-api-access-sptzc\") pod \"node-exporter-bwhfp\" (UID: \"ba05a4f0-94e4-4462-9bc4-7d0d7cc8e230\") " pod="openshift-monitoring/node-exporter-bwhfp" Jul 14 17:33:29.463769 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:29.463593 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ba05a4f0-94e4-4462-9bc4-7d0d7cc8e230-node-exporter-tls\") pod \"node-exporter-bwhfp\" (UID: \"ba05a4f0-94e4-4462-9bc4-7d0d7cc8e230\") " pod="openshift-monitoring/node-exporter-bwhfp" Jul 14 17:33:29.463769 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:29.463617 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ba05a4f0-94e4-4462-9bc4-7d0d7cc8e230-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-bwhfp\" (UID: \"ba05a4f0-94e4-4462-9bc4-7d0d7cc8e230\") " pod="openshift-monitoring/node-exporter-bwhfp" Jul 14 17:33:29.463769 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:29.463688 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ba05a4f0-94e4-4462-9bc4-7d0d7cc8e230-sys\") pod \"node-exporter-bwhfp\" (UID: \"ba05a4f0-94e4-4462-9bc4-7d0d7cc8e230\") " pod="openshift-monitoring/node-exporter-bwhfp" Jul 14 17:33:29.463769 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:29.463738 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ba05a4f0-94e4-4462-9bc4-7d0d7cc8e230-node-exporter-accelerators-collector-config\") pod \"node-exporter-bwhfp\" (UID: \"ba05a4f0-94e4-4462-9bc4-7d0d7cc8e230\") " pod="openshift-monitoring/node-exporter-bwhfp" Jul 14 17:33:29.463769 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:29.463760 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ba05a4f0-94e4-4462-9bc4-7d0d7cc8e230-root\") pod \"node-exporter-bwhfp\" (UID: \"ba05a4f0-94e4-4462-9bc4-7d0d7cc8e230\") " pod="openshift-monitoring/node-exporter-bwhfp" Jul 14 17:33:29.464018 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:29.463788 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ba05a4f0-94e4-4462-9bc4-7d0d7cc8e230-node-exporter-wtmp\") pod \"node-exporter-bwhfp\" (UID: \"ba05a4f0-94e4-4462-9bc4-7d0d7cc8e230\") " pod="openshift-monitoring/node-exporter-bwhfp" Jul 14 17:33:29.464018 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:29.463867 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ba05a4f0-94e4-4462-9bc4-7d0d7cc8e230-metrics-client-ca\") pod \"node-exporter-bwhfp\" (UID: \"ba05a4f0-94e4-4462-9bc4-7d0d7cc8e230\") " pod="openshift-monitoring/node-exporter-bwhfp" Jul 14 17:33:29.464018 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:29.463872 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ba05a4f0-94e4-4462-9bc4-7d0d7cc8e230-node-exporter-textfile\") pod \"node-exporter-bwhfp\" (UID: \"ba05a4f0-94e4-4462-9bc4-7d0d7cc8e230\") " pod="openshift-monitoring/node-exporter-bwhfp" Jul 14 17:33:29.464018 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:29.463872 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ba05a4f0-94e4-4462-9bc4-7d0d7cc8e230-root\") pod \"node-exporter-bwhfp\" (UID: \"ba05a4f0-94e4-4462-9bc4-7d0d7cc8e230\") " pod="openshift-monitoring/node-exporter-bwhfp" Jul 14 17:33:29.464018 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:29.463910 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ba05a4f0-94e4-4462-9bc4-7d0d7cc8e230-sys\") pod \"node-exporter-bwhfp\" (UID: \"ba05a4f0-94e4-4462-9bc4-7d0d7cc8e230\") " pod="openshift-monitoring/node-exporter-bwhfp" Jul 14 17:33:29.464225 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:29.464022 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ba05a4f0-94e4-4462-9bc4-7d0d7cc8e230-node-exporter-wtmp\") pod \"node-exporter-bwhfp\" (UID: \"ba05a4f0-94e4-4462-9bc4-7d0d7cc8e230\") " pod="openshift-monitoring/node-exporter-bwhfp" Jul 14 17:33:29.464327 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:29.464309 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ba05a4f0-94e4-4462-9bc4-7d0d7cc8e230-node-exporter-accelerators-collector-config\") pod \"node-exporter-bwhfp\" (UID: \"ba05a4f0-94e4-4462-9bc4-7d0d7cc8e230\") " pod="openshift-monitoring/node-exporter-bwhfp" Jul 14 17:33:29.464402 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:29.464327 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ba05a4f0-94e4-4462-9bc4-7d0d7cc8e230-metrics-client-ca\") pod \"node-exporter-bwhfp\" (UID: \"ba05a4f0-94e4-4462-9bc4-7d0d7cc8e230\") " pod="openshift-monitoring/node-exporter-bwhfp" Jul 14 17:33:29.467227 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:29.467208 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ba05a4f0-94e4-4462-9bc4-7d0d7cc8e230-node-exporter-tls\") pod \"node-exporter-bwhfp\" (UID: \"ba05a4f0-94e4-4462-9bc4-7d0d7cc8e230\") " pod="openshift-monitoring/node-exporter-bwhfp" Jul 14 17:33:29.467297 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:29.467208 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ba05a4f0-94e4-4462-9bc4-7d0d7cc8e230-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-bwhfp\" (UID: \"ba05a4f0-94e4-4462-9bc4-7d0d7cc8e230\") " pod="openshift-monitoring/node-exporter-bwhfp" Jul 14 17:33:29.472109 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:29.472085 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sptzc\" (UniqueName: \"kubernetes.io/projected/ba05a4f0-94e4-4462-9bc4-7d0d7cc8e230-kube-api-access-sptzc\") pod \"node-exporter-bwhfp\" (UID: \"ba05a4f0-94e4-4462-9bc4-7d0d7cc8e230\") " pod="openshift-monitoring/node-exporter-bwhfp" Jul 14 17:33:29.522309 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:29.522285 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-bwhfp" Jul 14 17:33:29.531849 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:33:29.531816 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba05a4f0_94e4_4462_9bc4_7d0d7cc8e230.slice/crio-9d02b8a249d061553e4735d299fc0e22a3c7c0617e5165a0b29b8fad6eac0857 WatchSource:0}: Error finding container 9d02b8a249d061553e4735d299fc0e22a3c7c0617e5165a0b29b8fad6eac0857: Status 404 returned error can't find the container with id 9d02b8a249d061553e4735d299fc0e22a3c7c0617e5165a0b29b8fad6eac0857 Jul 14 17:33:29.772641 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:29.772608 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-bwhfp" event={"ID":"ba05a4f0-94e4-4462-9bc4-7d0d7cc8e230","Type":"ContainerStarted","Data":"9d02b8a249d061553e4735d299fc0e22a3c7c0617e5165a0b29b8fad6eac0857"} Jul 14 17:33:29.774671 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:29.774639 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8r6jg" event={"ID":"7635e71d-e195-4ef9-8eff-980969d9f360","Type":"ContainerStarted","Data":"1db4b87bae48bfb703ac068d7abd962d7d796788ec32ff4c370bd6bb169a0ebc"} Jul 14 17:33:29.778644 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:29.778622 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-8669f6cd78-rjntn" Jul 14 17:33:29.797077 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:29.797028 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-8r6jg" podStartSLOduration=1.7766822530000002 podStartE2EDuration="12.797011748s" totalImagesPullingTime="11.020329495s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 17:33:17 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:33:29.796797521 +0000 UTC m=+58.873255061" watchObservedRunningTime="2026-07-14 17:33:29.797011748 +0000 UTC m=+58.873469312" Jul 14 17:33:30.152607 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:30.152572 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 14 17:33:30.155007 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:30.154981 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:33:30.157563 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:30.157391 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-bg757\"" Jul 14 17:33:30.157563 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:30.157440 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 14 17:33:30.157933 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:30.157904 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 14 17:33:30.158053 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:30.157934 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 14 17:33:30.158204 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:30.158171 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 14 17:33:30.158325 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:30.158211 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 14 17:33:30.158460 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:30.158438 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 14 17:33:30.158460 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:30.158456 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 14 17:33:30.158600 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:30.158456 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 14 17:33:30.158811 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:30.158785 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 14 17:33:30.178622 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:30.178594 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 14 17:33:30.271013 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:30.270971 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9496f0bd-bcba-453e-b249-868c02fac411-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"9496f0bd-bcba-453e-b249-868c02fac411\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:33:30.271242 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:30.271035 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9496f0bd-bcba-453e-b249-868c02fac411-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"9496f0bd-bcba-453e-b249-868c02fac411\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:33:30.271242 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:30.271089 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9496f0bd-bcba-453e-b249-868c02fac411-tls-assets\") pod \"alertmanager-main-0\" (UID: \"9496f0bd-bcba-453e-b249-868c02fac411\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:33:30.271242 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:30.271127 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pkfjp\" (UniqueName: \"kubernetes.io/projected/9496f0bd-bcba-453e-b249-868c02fac411-kube-api-access-pkfjp\") pod \"alertmanager-main-0\" (UID: \"9496f0bd-bcba-453e-b249-868c02fac411\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:33:30.271418 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:30.271178 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/9496f0bd-bcba-453e-b249-868c02fac411-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"9496f0bd-bcba-453e-b249-868c02fac411\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:33:30.271418 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:30.271286 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9496f0bd-bcba-453e-b249-868c02fac411-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"9496f0bd-bcba-453e-b249-868c02fac411\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:33:30.271418 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:30.271314 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/9496f0bd-bcba-453e-b249-868c02fac411-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"9496f0bd-bcba-453e-b249-868c02fac411\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:33:30.271418 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:30.271358 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/9496f0bd-bcba-453e-b249-868c02fac411-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"9496f0bd-bcba-453e-b249-868c02fac411\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:33:30.271418 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:30.271402 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9496f0bd-bcba-453e-b249-868c02fac411-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"9496f0bd-bcba-453e-b249-868c02fac411\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:33:30.271662 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:30.271449 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/9496f0bd-bcba-453e-b249-868c02fac411-config-volume\") pod \"alertmanager-main-0\" (UID: \"9496f0bd-bcba-453e-b249-868c02fac411\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:33:30.271662 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:30.271488 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9496f0bd-bcba-453e-b249-868c02fac411-web-config\") pod \"alertmanager-main-0\" (UID: \"9496f0bd-bcba-453e-b249-868c02fac411\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:33:30.271662 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:30.271541 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9496f0bd-bcba-453e-b249-868c02fac411-config-out\") pod \"alertmanager-main-0\" (UID: \"9496f0bd-bcba-453e-b249-868c02fac411\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:33:30.271662 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:30.271575 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/9496f0bd-bcba-453e-b249-868c02fac411-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"9496f0bd-bcba-453e-b249-868c02fac411\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:33:30.372919 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:30.372886 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/9496f0bd-bcba-453e-b249-868c02fac411-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"9496f0bd-bcba-453e-b249-868c02fac411\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:33:30.372919 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:30.372925 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9496f0bd-bcba-453e-b249-868c02fac411-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"9496f0bd-bcba-453e-b249-868c02fac411\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:33:30.373150 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:30.372948 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/9496f0bd-bcba-453e-b249-868c02fac411-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"9496f0bd-bcba-453e-b249-868c02fac411\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:33:30.373150 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:30.372972 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/9496f0bd-bcba-453e-b249-868c02fac411-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"9496f0bd-bcba-453e-b249-868c02fac411\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:33:30.373150 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:30.373009 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9496f0bd-bcba-453e-b249-868c02fac411-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"9496f0bd-bcba-453e-b249-868c02fac411\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:33:30.373150 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:30.373033 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/9496f0bd-bcba-453e-b249-868c02fac411-config-volume\") pod \"alertmanager-main-0\" (UID: \"9496f0bd-bcba-453e-b249-868c02fac411\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:33:30.373150 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:30.373057 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9496f0bd-bcba-453e-b249-868c02fac411-web-config\") pod \"alertmanager-main-0\" (UID: \"9496f0bd-bcba-453e-b249-868c02fac411\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:33:30.373150 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:33:30.373088 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9496f0bd-bcba-453e-b249-868c02fac411-alertmanager-trusted-ca-bundle podName:9496f0bd-bcba-453e-b249-868c02fac411 nodeName:}" failed. No retries permitted until 2026-07-14 17:33:30.873063417 +0000 UTC m=+59.949520954 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/9496f0bd-bcba-453e-b249-868c02fac411-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "9496f0bd-bcba-453e-b249-868c02fac411") : configmap references non-existent config key: ca-bundle.crt Jul 14 17:33:30.373493 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:30.373284 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9496f0bd-bcba-453e-b249-868c02fac411-config-out\") pod \"alertmanager-main-0\" (UID: \"9496f0bd-bcba-453e-b249-868c02fac411\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:33:30.373493 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:30.373323 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/9496f0bd-bcba-453e-b249-868c02fac411-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"9496f0bd-bcba-453e-b249-868c02fac411\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:33:30.373493 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:30.373369 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9496f0bd-bcba-453e-b249-868c02fac411-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"9496f0bd-bcba-453e-b249-868c02fac411\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:33:30.373493 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:30.373404 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9496f0bd-bcba-453e-b249-868c02fac411-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"9496f0bd-bcba-453e-b249-868c02fac411\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:33:30.373493 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:30.373467 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9496f0bd-bcba-453e-b249-868c02fac411-tls-assets\") pod \"alertmanager-main-0\" (UID: \"9496f0bd-bcba-453e-b249-868c02fac411\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:33:30.373898 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:30.373498 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pkfjp\" (UniqueName: \"kubernetes.io/projected/9496f0bd-bcba-453e-b249-868c02fac411-kube-api-access-pkfjp\") pod \"alertmanager-main-0\" (UID: \"9496f0bd-bcba-453e-b249-868c02fac411\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:33:30.373898 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:30.373560 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/9496f0bd-bcba-453e-b249-868c02fac411-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"9496f0bd-bcba-453e-b249-868c02fac411\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:33:30.374358 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:30.374280 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9496f0bd-bcba-453e-b249-868c02fac411-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"9496f0bd-bcba-453e-b249-868c02fac411\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:33:30.376669 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:30.376249 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/9496f0bd-bcba-453e-b249-868c02fac411-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"9496f0bd-bcba-453e-b249-868c02fac411\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:33:30.376669 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:30.376383 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9496f0bd-bcba-453e-b249-868c02fac411-web-config\") pod \"alertmanager-main-0\" (UID: \"9496f0bd-bcba-453e-b249-868c02fac411\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:33:30.377145 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:30.377119 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/9496f0bd-bcba-453e-b249-868c02fac411-config-volume\") pod \"alertmanager-main-0\" (UID: \"9496f0bd-bcba-453e-b249-868c02fac411\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:33:30.377922 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:30.377819 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9496f0bd-bcba-453e-b249-868c02fac411-config-out\") pod \"alertmanager-main-0\" (UID: \"9496f0bd-bcba-453e-b249-868c02fac411\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:33:30.378039 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:30.377994 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9496f0bd-bcba-453e-b249-868c02fac411-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"9496f0bd-bcba-453e-b249-868c02fac411\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:33:30.378131 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:30.378098 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9496f0bd-bcba-453e-b249-868c02fac411-tls-assets\") pod \"alertmanager-main-0\" (UID: \"9496f0bd-bcba-453e-b249-868c02fac411\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:33:30.378274 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:30.378204 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/9496f0bd-bcba-453e-b249-868c02fac411-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"9496f0bd-bcba-453e-b249-868c02fac411\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:33:30.378624 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:30.378602 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9496f0bd-bcba-453e-b249-868c02fac411-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"9496f0bd-bcba-453e-b249-868c02fac411\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:33:30.379231 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:30.379179 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/9496f0bd-bcba-453e-b249-868c02fac411-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"9496f0bd-bcba-453e-b249-868c02fac411\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:33:30.383485 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:30.383466 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pkfjp\" (UniqueName: \"kubernetes.io/projected/9496f0bd-bcba-453e-b249-868c02fac411-kube-api-access-pkfjp\") pod \"alertmanager-main-0\" (UID: \"9496f0bd-bcba-453e-b249-868c02fac411\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:33:30.674978 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:30.674943 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-2sbww" Jul 14 17:33:30.779904 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:30.779874 2562 generic.go:356] "Generic (PLEG): container finished" podID="ba05a4f0-94e4-4462-9bc4-7d0d7cc8e230" containerID="56d9541d18a4de8824e900bcaf321f2e44010554ecc7ba6ee9f230a0aaa81a74" exitCode=0 Jul 14 17:33:30.780061 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:30.779952 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-bwhfp" event={"ID":"ba05a4f0-94e4-4462-9bc4-7d0d7cc8e230","Type":"ContainerDied","Data":"56d9541d18a4de8824e900bcaf321f2e44010554ecc7ba6ee9f230a0aaa81a74"} Jul 14 17:33:30.877261 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:30.877232 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9496f0bd-bcba-453e-b249-868c02fac411-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"9496f0bd-bcba-453e-b249-868c02fac411\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:33:30.878031 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:30.878002 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9496f0bd-bcba-453e-b249-868c02fac411-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"9496f0bd-bcba-453e-b249-868c02fac411\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:33:31.068208 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:31.068154 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:33:31.084077 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:31.084052 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7944586585-9b8sh" Jul 14 17:33:31.084217 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:31.084102 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7944586585-9b8sh" Jul 14 17:33:31.089110 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:31.089087 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7944586585-9b8sh" Jul 14 17:33:31.180874 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:31.180841 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-57d5c4cc6b-4j54g"] Jul 14 17:33:31.184739 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:31.184716 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-57d5c4cc6b-4j54g" Jul 14 17:33:31.190687 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:31.190386 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 14 17:33:31.190931 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:31.190908 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 14 17:33:31.191063 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:31.191006 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 14 17:33:31.191326 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:31.191311 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 14 17:33:31.191427 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:31.191348 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-ppttw\"" Jul 14 17:33:31.191495 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:31.191480 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-c6dc7devqlurg\"" Jul 14 17:33:31.191543 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:31.191502 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 14 17:33:31.224001 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:31.223971 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-57d5c4cc6b-4j54g"] Jul 14 17:33:31.244064 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:33:31.244039 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9496f0bd_bcba_453e_b249_868c02fac411.slice/crio-6278848b1756ff75d7ddd35cb9d883a85d638e9785d01c859f24fbcff741d1c4 WatchSource:0}: Error finding container 6278848b1756ff75d7ddd35cb9d883a85d638e9785d01c859f24fbcff741d1c4: Status 404 returned error can't find the container with id 6278848b1756ff75d7ddd35cb9d883a85d638e9785d01c859f24fbcff741d1c4 Jul 14 17:33:31.249163 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:31.248068 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 14 17:33:31.281786 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:31.281754 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/c20e734f-caf7-4ac3-a4f2-4297dfe34ec1-secret-thanos-querier-tls\") pod \"thanos-querier-57d5c4cc6b-4j54g\" (UID: \"c20e734f-caf7-4ac3-a4f2-4297dfe34ec1\") " pod="openshift-monitoring/thanos-querier-57d5c4cc6b-4j54g" Jul 14 17:33:31.281786 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:31.281786 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c20e734f-caf7-4ac3-a4f2-4297dfe34ec1-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-57d5c4cc6b-4j54g\" (UID: \"c20e734f-caf7-4ac3-a4f2-4297dfe34ec1\") " pod="openshift-monitoring/thanos-querier-57d5c4cc6b-4j54g" Jul 14 17:33:31.281941 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:31.281810 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c20e734f-caf7-4ac3-a4f2-4297dfe34ec1-metrics-client-ca\") pod \"thanos-querier-57d5c4cc6b-4j54g\" (UID: \"c20e734f-caf7-4ac3-a4f2-4297dfe34ec1\") " pod="openshift-monitoring/thanos-querier-57d5c4cc6b-4j54g" Jul 14 17:33:31.281941 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:31.281913 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zqh4l\" (UniqueName: \"kubernetes.io/projected/c20e734f-caf7-4ac3-a4f2-4297dfe34ec1-kube-api-access-zqh4l\") pod \"thanos-querier-57d5c4cc6b-4j54g\" (UID: \"c20e734f-caf7-4ac3-a4f2-4297dfe34ec1\") " pod="openshift-monitoring/thanos-querier-57d5c4cc6b-4j54g" Jul 14 17:33:31.282000 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:31.281949 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/c20e734f-caf7-4ac3-a4f2-4297dfe34ec1-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-57d5c4cc6b-4j54g\" (UID: \"c20e734f-caf7-4ac3-a4f2-4297dfe34ec1\") " pod="openshift-monitoring/thanos-querier-57d5c4cc6b-4j54g" Jul 14 17:33:31.282000 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:31.281976 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c20e734f-caf7-4ac3-a4f2-4297dfe34ec1-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-57d5c4cc6b-4j54g\" (UID: \"c20e734f-caf7-4ac3-a4f2-4297dfe34ec1\") " pod="openshift-monitoring/thanos-querier-57d5c4cc6b-4j54g" Jul 14 17:33:31.282081 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:31.282004 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/c20e734f-caf7-4ac3-a4f2-4297dfe34ec1-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-57d5c4cc6b-4j54g\" (UID: \"c20e734f-caf7-4ac3-a4f2-4297dfe34ec1\") " pod="openshift-monitoring/thanos-querier-57d5c4cc6b-4j54g" Jul 14 17:33:31.282081 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:31.282033 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c20e734f-caf7-4ac3-a4f2-4297dfe34ec1-secret-grpc-tls\") pod \"thanos-querier-57d5c4cc6b-4j54g\" (UID: \"c20e734f-caf7-4ac3-a4f2-4297dfe34ec1\") " pod="openshift-monitoring/thanos-querier-57d5c4cc6b-4j54g" Jul 14 17:33:31.383099 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:31.383026 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/c20e734f-caf7-4ac3-a4f2-4297dfe34ec1-secret-thanos-querier-tls\") pod \"thanos-querier-57d5c4cc6b-4j54g\" (UID: \"c20e734f-caf7-4ac3-a4f2-4297dfe34ec1\") " pod="openshift-monitoring/thanos-querier-57d5c4cc6b-4j54g" Jul 14 17:33:31.383099 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:31.383060 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c20e734f-caf7-4ac3-a4f2-4297dfe34ec1-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-57d5c4cc6b-4j54g\" (UID: \"c20e734f-caf7-4ac3-a4f2-4297dfe34ec1\") " pod="openshift-monitoring/thanos-querier-57d5c4cc6b-4j54g" Jul 14 17:33:31.383099 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:31.383082 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c20e734f-caf7-4ac3-a4f2-4297dfe34ec1-metrics-client-ca\") pod \"thanos-querier-57d5c4cc6b-4j54g\" (UID: \"c20e734f-caf7-4ac3-a4f2-4297dfe34ec1\") " pod="openshift-monitoring/thanos-querier-57d5c4cc6b-4j54g" Jul 14 17:33:31.383384 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:31.383122 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zqh4l\" (UniqueName: \"kubernetes.io/projected/c20e734f-caf7-4ac3-a4f2-4297dfe34ec1-kube-api-access-zqh4l\") pod \"thanos-querier-57d5c4cc6b-4j54g\" (UID: \"c20e734f-caf7-4ac3-a4f2-4297dfe34ec1\") " pod="openshift-monitoring/thanos-querier-57d5c4cc6b-4j54g" Jul 14 17:33:31.383384 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:31.383156 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/c20e734f-caf7-4ac3-a4f2-4297dfe34ec1-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-57d5c4cc6b-4j54g\" (UID: \"c20e734f-caf7-4ac3-a4f2-4297dfe34ec1\") " pod="openshift-monitoring/thanos-querier-57d5c4cc6b-4j54g" Jul 14 17:33:31.383384 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:31.383206 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c20e734f-caf7-4ac3-a4f2-4297dfe34ec1-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-57d5c4cc6b-4j54g\" (UID: \"c20e734f-caf7-4ac3-a4f2-4297dfe34ec1\") " pod="openshift-monitoring/thanos-querier-57d5c4cc6b-4j54g" Jul 14 17:33:31.383529 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:31.383391 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/c20e734f-caf7-4ac3-a4f2-4297dfe34ec1-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-57d5c4cc6b-4j54g\" (UID: \"c20e734f-caf7-4ac3-a4f2-4297dfe34ec1\") " pod="openshift-monitoring/thanos-querier-57d5c4cc6b-4j54g" Jul 14 17:33:31.383529 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:31.383450 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c20e734f-caf7-4ac3-a4f2-4297dfe34ec1-secret-grpc-tls\") pod \"thanos-querier-57d5c4cc6b-4j54g\" (UID: \"c20e734f-caf7-4ac3-a4f2-4297dfe34ec1\") " pod="openshift-monitoring/thanos-querier-57d5c4cc6b-4j54g" Jul 14 17:33:31.383984 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:31.383960 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c20e734f-caf7-4ac3-a4f2-4297dfe34ec1-metrics-client-ca\") pod \"thanos-querier-57d5c4cc6b-4j54g\" (UID: \"c20e734f-caf7-4ac3-a4f2-4297dfe34ec1\") " pod="openshift-monitoring/thanos-querier-57d5c4cc6b-4j54g" Jul 14 17:33:31.385902 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:31.385885 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 14 17:33:31.386308 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:31.386291 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 14 17:33:31.386432 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:31.386372 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 14 17:33:31.386675 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:31.386658 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 14 17:33:31.386741 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:31.386680 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 14 17:33:31.386741 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:31.386659 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-c6dc7devqlurg\"" Jul 14 17:33:31.394535 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:31.394514 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zqh4l\" (UniqueName: \"kubernetes.io/projected/c20e734f-caf7-4ac3-a4f2-4297dfe34ec1-kube-api-access-zqh4l\") pod \"thanos-querier-57d5c4cc6b-4j54g\" (UID: \"c20e734f-caf7-4ac3-a4f2-4297dfe34ec1\") " pod="openshift-monitoring/thanos-querier-57d5c4cc6b-4j54g" Jul 14 17:33:31.396149 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:31.396120 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/c20e734f-caf7-4ac3-a4f2-4297dfe34ec1-secret-thanos-querier-tls\") pod \"thanos-querier-57d5c4cc6b-4j54g\" (UID: \"c20e734f-caf7-4ac3-a4f2-4297dfe34ec1\") " pod="openshift-monitoring/thanos-querier-57d5c4cc6b-4j54g" Jul 14 17:33:31.396285 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:31.396219 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c20e734f-caf7-4ac3-a4f2-4297dfe34ec1-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-57d5c4cc6b-4j54g\" (UID: \"c20e734f-caf7-4ac3-a4f2-4297dfe34ec1\") " pod="openshift-monitoring/thanos-querier-57d5c4cc6b-4j54g" Jul 14 17:33:31.396755 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:31.396730 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c20e734f-caf7-4ac3-a4f2-4297dfe34ec1-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-57d5c4cc6b-4j54g\" (UID: \"c20e734f-caf7-4ac3-a4f2-4297dfe34ec1\") " pod="openshift-monitoring/thanos-querier-57d5c4cc6b-4j54g" Jul 14 17:33:31.396847 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:31.396738 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/c20e734f-caf7-4ac3-a4f2-4297dfe34ec1-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-57d5c4cc6b-4j54g\" (UID: \"c20e734f-caf7-4ac3-a4f2-4297dfe34ec1\") " pod="openshift-monitoring/thanos-querier-57d5c4cc6b-4j54g" Jul 14 17:33:31.397062 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:31.397044 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c20e734f-caf7-4ac3-a4f2-4297dfe34ec1-secret-grpc-tls\") pod \"thanos-querier-57d5c4cc6b-4j54g\" (UID: \"c20e734f-caf7-4ac3-a4f2-4297dfe34ec1\") " pod="openshift-monitoring/thanos-querier-57d5c4cc6b-4j54g" Jul 14 17:33:31.397661 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:31.397640 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/c20e734f-caf7-4ac3-a4f2-4297dfe34ec1-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-57d5c4cc6b-4j54g\" (UID: \"c20e734f-caf7-4ac3-a4f2-4297dfe34ec1\") " pod="openshift-monitoring/thanos-querier-57d5c4cc6b-4j54g" Jul 14 17:33:31.496096 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:31.496064 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-ppttw\"" Jul 14 17:33:31.504091 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:31.504072 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-57d5c4cc6b-4j54g" Jul 14 17:33:31.637706 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:31.637619 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-57d5c4cc6b-4j54g"] Jul 14 17:33:31.641149 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:33:31.641119 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc20e734f_caf7_4ac3_a4f2_4297dfe34ec1.slice/crio-28b8aa6b23a9b6143d9f44adce15cac172f32d82eef966fc81a638734d2c1ef2 WatchSource:0}: Error finding container 28b8aa6b23a9b6143d9f44adce15cac172f32d82eef966fc81a638734d2c1ef2: Status 404 returned error can't find the container with id 28b8aa6b23a9b6143d9f44adce15cac172f32d82eef966fc81a638734d2c1ef2 Jul 14 17:33:31.783806 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:31.783770 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9496f0bd-bcba-453e-b249-868c02fac411","Type":"ContainerStarted","Data":"6278848b1756ff75d7ddd35cb9d883a85d638e9785d01c859f24fbcff741d1c4"} Jul 14 17:33:31.785815 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:31.785791 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-bwhfp" event={"ID":"ba05a4f0-94e4-4462-9bc4-7d0d7cc8e230","Type":"ContainerStarted","Data":"2014930252c7dbc9ba47a213495ab65e9c96ce540915c763d3fec55359b97d55"} Jul 14 17:33:31.785938 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:31.785843 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-bwhfp" event={"ID":"ba05a4f0-94e4-4462-9bc4-7d0d7cc8e230","Type":"ContainerStarted","Data":"2e9692374d7c3c036398b76be9d37aff47493de7c4413e6d368c5e2a6a486631"} Jul 14 17:33:31.786944 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:31.786920 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-57d5c4cc6b-4j54g" event={"ID":"c20e734f-caf7-4ac3-a4f2-4297dfe34ec1","Type":"ContainerStarted","Data":"28b8aa6b23a9b6143d9f44adce15cac172f32d82eef966fc81a638734d2c1ef2"} Jul 14 17:33:31.791012 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:31.790991 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7944586585-9b8sh" Jul 14 17:33:31.810851 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:31.810792 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-bwhfp" podStartSLOduration=1.680417711 podStartE2EDuration="2.810768906s" totalImagesPullingTime="1.058174567s" totalInitContainerRuntime="72.176628ms" isStatefulPod=true podCreationTimestamp="2026-07-14 17:33:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:33:31.809068636 +0000 UTC m=+60.885526175" watchObservedRunningTime="2026-07-14 17:33:31.810768906 +0000 UTC m=+60.887226446" Jul 14 17:33:31.878441 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:31.878414 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8669f6cd78-rjntn"] Jul 14 17:33:33.673267 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:33.673236 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-69dd5d8d98-25vlb"] Jul 14 17:33:33.675483 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:33.675462 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-69dd5d8d98-25vlb" Jul 14 17:33:33.680344 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:33.680314 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 14 17:33:33.680566 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:33.680420 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-98rzl\"" Jul 14 17:33:33.681204 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:33.680921 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-3co6bbcq5uuo8\"" Jul 14 17:33:33.681204 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:33.681018 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 14 17:33:33.681481 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:33.681371 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 14 17:33:33.681779 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:33.681762 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 14 17:33:33.693023 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:33.693001 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-69dd5d8d98-25vlb"] Jul 14 17:33:33.705489 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:33.705468 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/15eb5778-e2da-43d6-b724-dde07774086a-metrics-server-audit-profiles\") pod \"metrics-server-69dd5d8d98-25vlb\" (UID: \"15eb5778-e2da-43d6-b724-dde07774086a\") " pod="openshift-monitoring/metrics-server-69dd5d8d98-25vlb" Jul 14 17:33:33.705571 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:33.705509 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/15eb5778-e2da-43d6-b724-dde07774086a-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-69dd5d8d98-25vlb\" (UID: \"15eb5778-e2da-43d6-b724-dde07774086a\") " pod="openshift-monitoring/metrics-server-69dd5d8d98-25vlb" Jul 14 17:33:33.705571 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:33.705531 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/15eb5778-e2da-43d6-b724-dde07774086a-client-ca-bundle\") pod \"metrics-server-69dd5d8d98-25vlb\" (UID: \"15eb5778-e2da-43d6-b724-dde07774086a\") " pod="openshift-monitoring/metrics-server-69dd5d8d98-25vlb" Jul 14 17:33:33.705640 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:33.705609 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/15eb5778-e2da-43d6-b724-dde07774086a-secret-metrics-server-tls\") pod \"metrics-server-69dd5d8d98-25vlb\" (UID: \"15eb5778-e2da-43d6-b724-dde07774086a\") " pod="openshift-monitoring/metrics-server-69dd5d8d98-25vlb" Jul 14 17:33:33.705640 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:33.705633 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/15eb5778-e2da-43d6-b724-dde07774086a-audit-log\") pod \"metrics-server-69dd5d8d98-25vlb\" (UID: \"15eb5778-e2da-43d6-b724-dde07774086a\") " pod="openshift-monitoring/metrics-server-69dd5d8d98-25vlb" Jul 14 17:33:33.705703 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:33.705650 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hr8n9\" (UniqueName: \"kubernetes.io/projected/15eb5778-e2da-43d6-b724-dde07774086a-kube-api-access-hr8n9\") pod \"metrics-server-69dd5d8d98-25vlb\" (UID: \"15eb5778-e2da-43d6-b724-dde07774086a\") " pod="openshift-monitoring/metrics-server-69dd5d8d98-25vlb" Jul 14 17:33:33.705731 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:33.705710 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/15eb5778-e2da-43d6-b724-dde07774086a-secret-metrics-server-client-certs\") pod \"metrics-server-69dd5d8d98-25vlb\" (UID: \"15eb5778-e2da-43d6-b724-dde07774086a\") " pod="openshift-monitoring/metrics-server-69dd5d8d98-25vlb" Jul 14 17:33:33.799026 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:33.799002 2562 generic.go:356] "Generic (PLEG): container finished" podID="9496f0bd-bcba-453e-b249-868c02fac411" containerID="1bd6e783c122326db01313d12d0bc98128bfdc4f28a87f0727315472c77140b1" exitCode=0 Jul 14 17:33:33.799150 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:33.799082 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9496f0bd-bcba-453e-b249-868c02fac411","Type":"ContainerDied","Data":"1bd6e783c122326db01313d12d0bc98128bfdc4f28a87f0727315472c77140b1"} Jul 14 17:33:33.806232 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:33.806177 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/15eb5778-e2da-43d6-b724-dde07774086a-secret-metrics-server-tls\") pod \"metrics-server-69dd5d8d98-25vlb\" (UID: \"15eb5778-e2da-43d6-b724-dde07774086a\") " pod="openshift-monitoring/metrics-server-69dd5d8d98-25vlb" Jul 14 17:33:33.806332 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:33.806254 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/15eb5778-e2da-43d6-b724-dde07774086a-audit-log\") pod \"metrics-server-69dd5d8d98-25vlb\" (UID: \"15eb5778-e2da-43d6-b724-dde07774086a\") " pod="openshift-monitoring/metrics-server-69dd5d8d98-25vlb" Jul 14 17:33:33.806332 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:33.806283 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hr8n9\" (UniqueName: \"kubernetes.io/projected/15eb5778-e2da-43d6-b724-dde07774086a-kube-api-access-hr8n9\") pod \"metrics-server-69dd5d8d98-25vlb\" (UID: \"15eb5778-e2da-43d6-b724-dde07774086a\") " pod="openshift-monitoring/metrics-server-69dd5d8d98-25vlb" Jul 14 17:33:33.806412 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:33.806347 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/15eb5778-e2da-43d6-b724-dde07774086a-secret-metrics-server-client-certs\") pod \"metrics-server-69dd5d8d98-25vlb\" (UID: \"15eb5778-e2da-43d6-b724-dde07774086a\") " pod="openshift-monitoring/metrics-server-69dd5d8d98-25vlb" Jul 14 17:33:33.806412 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:33.806395 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/15eb5778-e2da-43d6-b724-dde07774086a-metrics-server-audit-profiles\") pod \"metrics-server-69dd5d8d98-25vlb\" (UID: \"15eb5778-e2da-43d6-b724-dde07774086a\") " pod="openshift-monitoring/metrics-server-69dd5d8d98-25vlb" Jul 14 17:33:33.806479 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:33.806431 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/15eb5778-e2da-43d6-b724-dde07774086a-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-69dd5d8d98-25vlb\" (UID: \"15eb5778-e2da-43d6-b724-dde07774086a\") " pod="openshift-monitoring/metrics-server-69dd5d8d98-25vlb" Jul 14 17:33:33.806479 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:33.806462 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/15eb5778-e2da-43d6-b724-dde07774086a-client-ca-bundle\") pod \"metrics-server-69dd5d8d98-25vlb\" (UID: \"15eb5778-e2da-43d6-b724-dde07774086a\") " pod="openshift-monitoring/metrics-server-69dd5d8d98-25vlb" Jul 14 17:33:33.806687 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:33.806639 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/15eb5778-e2da-43d6-b724-dde07774086a-audit-log\") pod \"metrics-server-69dd5d8d98-25vlb\" (UID: \"15eb5778-e2da-43d6-b724-dde07774086a\") " pod="openshift-monitoring/metrics-server-69dd5d8d98-25vlb" Jul 14 17:33:33.807200 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:33.807167 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/15eb5778-e2da-43d6-b724-dde07774086a-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-69dd5d8d98-25vlb\" (UID: \"15eb5778-e2da-43d6-b724-dde07774086a\") " pod="openshift-monitoring/metrics-server-69dd5d8d98-25vlb" Jul 14 17:33:33.807720 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:33.807697 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/15eb5778-e2da-43d6-b724-dde07774086a-metrics-server-audit-profiles\") pod \"metrics-server-69dd5d8d98-25vlb\" (UID: \"15eb5778-e2da-43d6-b724-dde07774086a\") " pod="openshift-monitoring/metrics-server-69dd5d8d98-25vlb" Jul 14 17:33:33.808753 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:33.808727 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/15eb5778-e2da-43d6-b724-dde07774086a-secret-metrics-server-tls\") pod \"metrics-server-69dd5d8d98-25vlb\" (UID: \"15eb5778-e2da-43d6-b724-dde07774086a\") " pod="openshift-monitoring/metrics-server-69dd5d8d98-25vlb" Jul 14 17:33:33.808832 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:33.808769 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/15eb5778-e2da-43d6-b724-dde07774086a-client-ca-bundle\") pod \"metrics-server-69dd5d8d98-25vlb\" (UID: \"15eb5778-e2da-43d6-b724-dde07774086a\") " pod="openshift-monitoring/metrics-server-69dd5d8d98-25vlb" Jul 14 17:33:33.808929 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:33.808912 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/15eb5778-e2da-43d6-b724-dde07774086a-secret-metrics-server-client-certs\") pod \"metrics-server-69dd5d8d98-25vlb\" (UID: \"15eb5778-e2da-43d6-b724-dde07774086a\") " pod="openshift-monitoring/metrics-server-69dd5d8d98-25vlb" Jul 14 17:33:33.814713 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:33.814689 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hr8n9\" (UniqueName: \"kubernetes.io/projected/15eb5778-e2da-43d6-b724-dde07774086a-kube-api-access-hr8n9\") pod \"metrics-server-69dd5d8d98-25vlb\" (UID: \"15eb5778-e2da-43d6-b724-dde07774086a\") " pod="openshift-monitoring/metrics-server-69dd5d8d98-25vlb" Jul 14 17:33:33.986547 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:33.986456 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-69dd5d8d98-25vlb" Jul 14 17:33:34.121239 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:34.121207 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-69dd5d8d98-25vlb"] Jul 14 17:33:34.521390 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:34.521351 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-6fd6d747b5-qcc65"] Jul 14 17:33:34.527046 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:34.527013 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6fd6d747b5-qcc65" Jul 14 17:33:34.530514 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:34.530483 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 14 17:33:34.530772 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:34.530759 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 14 17:33:34.530855 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:34.530835 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-5bbps\"" Jul 14 17:33:34.531236 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:34.531216 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 14 17:33:34.531333 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:34.531251 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 14 17:33:34.533601 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:34.533583 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 14 17:33:34.537548 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:34.537521 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 14 17:33:34.546286 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:33:34.546232 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15eb5778_e2da_43d6_b724_dde07774086a.slice/crio-87f6699d4f11360fd766e18e11f17c2712f17810279ba65715a39fb828afc645 WatchSource:0}: Error finding container 87f6699d4f11360fd766e18e11f17c2712f17810279ba65715a39fb828afc645: Status 404 returned error can't find the container with id 87f6699d4f11360fd766e18e11f17c2712f17810279ba65715a39fb828afc645 Jul 14 17:33:34.548576 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:34.548554 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6fd6d747b5-qcc65"] Jul 14 17:33:34.613378 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:34.613353 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2bcb63de-5908-4bdf-9c18-86937b58b55d-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6fd6d747b5-qcc65\" (UID: \"2bcb63de-5908-4bdf-9c18-86937b58b55d\") " pod="openshift-monitoring/telemeter-client-6fd6d747b5-qcc65" Jul 14 17:33:34.613469 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:34.613392 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2bcb63de-5908-4bdf-9c18-86937b58b55d-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6fd6d747b5-qcc65\" (UID: \"2bcb63de-5908-4bdf-9c18-86937b58b55d\") " pod="openshift-monitoring/telemeter-client-6fd6d747b5-qcc65" Jul 14 17:33:34.613469 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:34.613431 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2bcb63de-5908-4bdf-9c18-86937b58b55d-serving-certs-ca-bundle\") pod \"telemeter-client-6fd6d747b5-qcc65\" (UID: \"2bcb63de-5908-4bdf-9c18-86937b58b55d\") " pod="openshift-monitoring/telemeter-client-6fd6d747b5-qcc65" Jul 14 17:33:34.613549 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:34.613492 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/2bcb63de-5908-4bdf-9c18-86937b58b55d-telemeter-client-tls\") pod \"telemeter-client-6fd6d747b5-qcc65\" (UID: \"2bcb63de-5908-4bdf-9c18-86937b58b55d\") " pod="openshift-monitoring/telemeter-client-6fd6d747b5-qcc65" Jul 14 17:33:34.613586 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:34.613541 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2bcb63de-5908-4bdf-9c18-86937b58b55d-metrics-client-ca\") pod \"telemeter-client-6fd6d747b5-qcc65\" (UID: \"2bcb63de-5908-4bdf-9c18-86937b58b55d\") " pod="openshift-monitoring/telemeter-client-6fd6d747b5-qcc65" Jul 14 17:33:34.613618 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:34.613588 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/2bcb63de-5908-4bdf-9c18-86937b58b55d-secret-telemeter-client\") pod \"telemeter-client-6fd6d747b5-qcc65\" (UID: \"2bcb63de-5908-4bdf-9c18-86937b58b55d\") " pod="openshift-monitoring/telemeter-client-6fd6d747b5-qcc65" Jul 14 17:33:34.613668 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:34.613652 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/2bcb63de-5908-4bdf-9c18-86937b58b55d-federate-client-tls\") pod \"telemeter-client-6fd6d747b5-qcc65\" (UID: \"2bcb63de-5908-4bdf-9c18-86937b58b55d\") " pod="openshift-monitoring/telemeter-client-6fd6d747b5-qcc65" Jul 14 17:33:34.613703 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:34.613692 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rjj4z\" (UniqueName: \"kubernetes.io/projected/2bcb63de-5908-4bdf-9c18-86937b58b55d-kube-api-access-rjj4z\") pod \"telemeter-client-6fd6d747b5-qcc65\" (UID: \"2bcb63de-5908-4bdf-9c18-86937b58b55d\") " pod="openshift-monitoring/telemeter-client-6fd6d747b5-qcc65" Jul 14 17:33:34.714469 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:34.714446 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2bcb63de-5908-4bdf-9c18-86937b58b55d-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6fd6d747b5-qcc65\" (UID: \"2bcb63de-5908-4bdf-9c18-86937b58b55d\") " pod="openshift-monitoring/telemeter-client-6fd6d747b5-qcc65" Jul 14 17:33:34.714826 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:34.714493 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2bcb63de-5908-4bdf-9c18-86937b58b55d-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6fd6d747b5-qcc65\" (UID: \"2bcb63de-5908-4bdf-9c18-86937b58b55d\") " pod="openshift-monitoring/telemeter-client-6fd6d747b5-qcc65" Jul 14 17:33:34.714826 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:34.714558 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2bcb63de-5908-4bdf-9c18-86937b58b55d-serving-certs-ca-bundle\") pod \"telemeter-client-6fd6d747b5-qcc65\" (UID: \"2bcb63de-5908-4bdf-9c18-86937b58b55d\") " pod="openshift-monitoring/telemeter-client-6fd6d747b5-qcc65" Jul 14 17:33:34.714826 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:34.714597 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/2bcb63de-5908-4bdf-9c18-86937b58b55d-telemeter-client-tls\") pod \"telemeter-client-6fd6d747b5-qcc65\" (UID: \"2bcb63de-5908-4bdf-9c18-86937b58b55d\") " pod="openshift-monitoring/telemeter-client-6fd6d747b5-qcc65" Jul 14 17:33:34.714826 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:34.714633 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2bcb63de-5908-4bdf-9c18-86937b58b55d-metrics-client-ca\") pod \"telemeter-client-6fd6d747b5-qcc65\" (UID: \"2bcb63de-5908-4bdf-9c18-86937b58b55d\") " pod="openshift-monitoring/telemeter-client-6fd6d747b5-qcc65" Jul 14 17:33:34.714826 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:34.714674 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/2bcb63de-5908-4bdf-9c18-86937b58b55d-secret-telemeter-client\") pod \"telemeter-client-6fd6d747b5-qcc65\" (UID: \"2bcb63de-5908-4bdf-9c18-86937b58b55d\") " pod="openshift-monitoring/telemeter-client-6fd6d747b5-qcc65" Jul 14 17:33:34.714826 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:34.714789 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/2bcb63de-5908-4bdf-9c18-86937b58b55d-federate-client-tls\") pod \"telemeter-client-6fd6d747b5-qcc65\" (UID: \"2bcb63de-5908-4bdf-9c18-86937b58b55d\") " pod="openshift-monitoring/telemeter-client-6fd6d747b5-qcc65" Jul 14 17:33:34.715094 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:34.714842 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rjj4z\" (UniqueName: \"kubernetes.io/projected/2bcb63de-5908-4bdf-9c18-86937b58b55d-kube-api-access-rjj4z\") pod \"telemeter-client-6fd6d747b5-qcc65\" (UID: \"2bcb63de-5908-4bdf-9c18-86937b58b55d\") " pod="openshift-monitoring/telemeter-client-6fd6d747b5-qcc65" Jul 14 17:33:34.715328 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:34.715306 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2bcb63de-5908-4bdf-9c18-86937b58b55d-serving-certs-ca-bundle\") pod \"telemeter-client-6fd6d747b5-qcc65\" (UID: \"2bcb63de-5908-4bdf-9c18-86937b58b55d\") " pod="openshift-monitoring/telemeter-client-6fd6d747b5-qcc65" Jul 14 17:33:34.715396 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:34.715329 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2bcb63de-5908-4bdf-9c18-86937b58b55d-metrics-client-ca\") pod \"telemeter-client-6fd6d747b5-qcc65\" (UID: \"2bcb63de-5908-4bdf-9c18-86937b58b55d\") " pod="openshift-monitoring/telemeter-client-6fd6d747b5-qcc65" Jul 14 17:33:34.715533 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:34.715470 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2bcb63de-5908-4bdf-9c18-86937b58b55d-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6fd6d747b5-qcc65\" (UID: \"2bcb63de-5908-4bdf-9c18-86937b58b55d\") " pod="openshift-monitoring/telemeter-client-6fd6d747b5-qcc65" Jul 14 17:33:34.717089 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:34.717064 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2bcb63de-5908-4bdf-9c18-86937b58b55d-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6fd6d747b5-qcc65\" (UID: \"2bcb63de-5908-4bdf-9c18-86937b58b55d\") " pod="openshift-monitoring/telemeter-client-6fd6d747b5-qcc65" Jul 14 17:33:34.717649 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:34.717629 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/2bcb63de-5908-4bdf-9c18-86937b58b55d-federate-client-tls\") pod \"telemeter-client-6fd6d747b5-qcc65\" (UID: \"2bcb63de-5908-4bdf-9c18-86937b58b55d\") " pod="openshift-monitoring/telemeter-client-6fd6d747b5-qcc65" Jul 14 17:33:34.717814 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:34.717796 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/2bcb63de-5908-4bdf-9c18-86937b58b55d-telemeter-client-tls\") pod \"telemeter-client-6fd6d747b5-qcc65\" (UID: \"2bcb63de-5908-4bdf-9c18-86937b58b55d\") " pod="openshift-monitoring/telemeter-client-6fd6d747b5-qcc65" Jul 14 17:33:34.717957 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:34.717938 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/2bcb63de-5908-4bdf-9c18-86937b58b55d-secret-telemeter-client\") pod \"telemeter-client-6fd6d747b5-qcc65\" (UID: \"2bcb63de-5908-4bdf-9c18-86937b58b55d\") " pod="openshift-monitoring/telemeter-client-6fd6d747b5-qcc65" Jul 14 17:33:34.724834 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:34.724811 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rjj4z\" (UniqueName: \"kubernetes.io/projected/2bcb63de-5908-4bdf-9c18-86937b58b55d-kube-api-access-rjj4z\") pod \"telemeter-client-6fd6d747b5-qcc65\" (UID: \"2bcb63de-5908-4bdf-9c18-86937b58b55d\") " pod="openshift-monitoring/telemeter-client-6fd6d747b5-qcc65" Jul 14 17:33:34.805858 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:34.805781 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-57d5c4cc6b-4j54g" event={"ID":"c20e734f-caf7-4ac3-a4f2-4297dfe34ec1","Type":"ContainerStarted","Data":"f07413525e375f3f1f4cd2ecb65317893f51e08397a03535089f55739ab9f740"} Jul 14 17:33:34.805999 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:34.805865 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-57d5c4cc6b-4j54g" event={"ID":"c20e734f-caf7-4ac3-a4f2-4297dfe34ec1","Type":"ContainerStarted","Data":"d6dbeba336cd96ef08c45e2b0edaf4ccc572a6e0671f6f0217890013a9b118fe"} Jul 14 17:33:34.805999 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:34.805903 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-57d5c4cc6b-4j54g" event={"ID":"c20e734f-caf7-4ac3-a4f2-4297dfe34ec1","Type":"ContainerStarted","Data":"4061fa62adabb4642a99af17a0c7bb515c4e8f8da26b3f478d01488b7214553f"} Jul 14 17:33:34.807465 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:34.807436 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-69dd5d8d98-25vlb" event={"ID":"15eb5778-e2da-43d6-b724-dde07774086a","Type":"ContainerStarted","Data":"87f6699d4f11360fd766e18e11f17c2712f17810279ba65715a39fb828afc645"} Jul 14 17:33:34.838874 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:34.838847 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6fd6d747b5-qcc65" Jul 14 17:33:34.971330 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:34.971301 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6fd6d747b5-qcc65"] Jul 14 17:33:34.974236 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:33:34.974209 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2bcb63de_5908_4bdf_9c18_86937b58b55d.slice/crio-0b89b76f9d48ed8397e65572059b967bbef18e28f54388a0e3ed5584296fa160 WatchSource:0}: Error finding container 0b89b76f9d48ed8397e65572059b967bbef18e28f54388a0e3ed5584296fa160: Status 404 returned error can't find the container with id 0b89b76f9d48ed8397e65572059b967bbef18e28f54388a0e3ed5584296fa160 Jul 14 17:33:35.484482 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:35.484451 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-cd46f694b-z4vjj"] Jul 14 17:33:35.486477 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:35.486452 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-cd46f694b-z4vjj" Jul 14 17:33:35.499877 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:35.499862 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-cd46f694b-z4vjj"] Jul 14 17:33:35.521278 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:35.521256 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/67e1d577-9cfd-475f-b758-4ef21fac1a15-service-ca\") pod \"console-cd46f694b-z4vjj\" (UID: \"67e1d577-9cfd-475f-b758-4ef21fac1a15\") " pod="openshift-console/console-cd46f694b-z4vjj" Jul 14 17:33:35.521400 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:35.521297 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/67e1d577-9cfd-475f-b758-4ef21fac1a15-console-serving-cert\") pod \"console-cd46f694b-z4vjj\" (UID: \"67e1d577-9cfd-475f-b758-4ef21fac1a15\") " pod="openshift-console/console-cd46f694b-z4vjj" Jul 14 17:33:35.521443 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:35.521408 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/67e1d577-9cfd-475f-b758-4ef21fac1a15-console-oauth-config\") pod \"console-cd46f694b-z4vjj\" (UID: \"67e1d577-9cfd-475f-b758-4ef21fac1a15\") " pod="openshift-console/console-cd46f694b-z4vjj" Jul 14 17:33:35.521487 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:35.521447 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/67e1d577-9cfd-475f-b758-4ef21fac1a15-console-config\") pod \"console-cd46f694b-z4vjj\" (UID: \"67e1d577-9cfd-475f-b758-4ef21fac1a15\") " pod="openshift-console/console-cd46f694b-z4vjj" Jul 14 17:33:35.521487 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:35.521472 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pgrz9\" (UniqueName: \"kubernetes.io/projected/67e1d577-9cfd-475f-b758-4ef21fac1a15-kube-api-access-pgrz9\") pod \"console-cd46f694b-z4vjj\" (UID: \"67e1d577-9cfd-475f-b758-4ef21fac1a15\") " pod="openshift-console/console-cd46f694b-z4vjj" Jul 14 17:33:35.521585 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:35.521558 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/67e1d577-9cfd-475f-b758-4ef21fac1a15-trusted-ca-bundle\") pod \"console-cd46f694b-z4vjj\" (UID: \"67e1d577-9cfd-475f-b758-4ef21fac1a15\") " pod="openshift-console/console-cd46f694b-z4vjj" Jul 14 17:33:35.521636 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:35.521595 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/67e1d577-9cfd-475f-b758-4ef21fac1a15-oauth-serving-cert\") pod \"console-cd46f694b-z4vjj\" (UID: \"67e1d577-9cfd-475f-b758-4ef21fac1a15\") " pod="openshift-console/console-cd46f694b-z4vjj" Jul 14 17:33:35.555996 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:35.554140 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 14 17:33:35.557168 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:35.557144 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:33:35.560654 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:35.560632 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 14 17:33:35.561172 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:35.561151 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 14 17:33:35.561657 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:35.561638 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 14 17:33:35.562576 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:35.561945 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-54mnitaep2gna\"" Jul 14 17:33:35.562576 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:35.562003 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 14 17:33:35.562576 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:35.562214 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 14 17:33:35.562576 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:35.562287 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-jrt8q\"" Jul 14 17:33:35.562576 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:35.562426 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 14 17:33:35.562864 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:35.562684 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 14 17:33:35.562920 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:35.562888 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 14 17:33:35.563414 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:35.563088 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 14 17:33:35.563504 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:35.563441 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 14 17:33:35.563648 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:35.563629 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 14 17:33:35.565387 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:35.565341 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 14 17:33:35.574613 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:35.574596 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 14 17:33:35.584472 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:35.584450 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 14 17:33:35.592934 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:35.592915 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 14 17:33:35.622115 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:35.622082 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/162964a8-ba20-4887-b8b9-e98369835a0d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"162964a8-ba20-4887-b8b9-e98369835a0d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:33:35.622244 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:35.622126 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/162964a8-ba20-4887-b8b9-e98369835a0d-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"162964a8-ba20-4887-b8b9-e98369835a0d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:33:35.622244 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:35.622161 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/67e1d577-9cfd-475f-b758-4ef21fac1a15-trusted-ca-bundle\") pod \"console-cd46f694b-z4vjj\" (UID: \"67e1d577-9cfd-475f-b758-4ef21fac1a15\") " pod="openshift-console/console-cd46f694b-z4vjj" Jul 14 17:33:35.622244 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:35.622222 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/162964a8-ba20-4887-b8b9-e98369835a0d-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"162964a8-ba20-4887-b8b9-e98369835a0d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:33:35.622349 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:35.622301 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/67e1d577-9cfd-475f-b758-4ef21fac1a15-oauth-serving-cert\") pod \"console-cd46f694b-z4vjj\" (UID: \"67e1d577-9cfd-475f-b758-4ef21fac1a15\") " pod="openshift-console/console-cd46f694b-z4vjj" Jul 14 17:33:35.622382 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:35.622371 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/162964a8-ba20-4887-b8b9-e98369835a0d-config\") pod \"prometheus-k8s-0\" (UID: \"162964a8-ba20-4887-b8b9-e98369835a0d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:33:35.622433 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:35.622395 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/162964a8-ba20-4887-b8b9-e98369835a0d-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"162964a8-ba20-4887-b8b9-e98369835a0d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:33:35.622480 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:35.622453 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/162964a8-ba20-4887-b8b9-e98369835a0d-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"162964a8-ba20-4887-b8b9-e98369835a0d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:33:35.622528 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:35.622491 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/162964a8-ba20-4887-b8b9-e98369835a0d-config-out\") pod \"prometheus-k8s-0\" (UID: \"162964a8-ba20-4887-b8b9-e98369835a0d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:33:35.622528 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:35.622518 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/162964a8-ba20-4887-b8b9-e98369835a0d-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"162964a8-ba20-4887-b8b9-e98369835a0d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:33:35.622635 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:35.622552 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/162964a8-ba20-4887-b8b9-e98369835a0d-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"162964a8-ba20-4887-b8b9-e98369835a0d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:33:35.622635 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:35.622582 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gwqft\" (UniqueName: \"kubernetes.io/projected/162964a8-ba20-4887-b8b9-e98369835a0d-kube-api-access-gwqft\") pod \"prometheus-k8s-0\" (UID: \"162964a8-ba20-4887-b8b9-e98369835a0d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:33:35.622719 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:35.622643 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/162964a8-ba20-4887-b8b9-e98369835a0d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"162964a8-ba20-4887-b8b9-e98369835a0d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:33:35.622719 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:35.622685 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/162964a8-ba20-4887-b8b9-e98369835a0d-web-config\") pod \"prometheus-k8s-0\" (UID: \"162964a8-ba20-4887-b8b9-e98369835a0d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:33:35.622807 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:35.622717 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/67e1d577-9cfd-475f-b758-4ef21fac1a15-service-ca\") pod \"console-cd46f694b-z4vjj\" (UID: \"67e1d577-9cfd-475f-b758-4ef21fac1a15\") " pod="openshift-console/console-cd46f694b-z4vjj" Jul 14 17:33:35.622807 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:35.622741 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/162964a8-ba20-4887-b8b9-e98369835a0d-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"162964a8-ba20-4887-b8b9-e98369835a0d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:33:35.622914 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:35.622813 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/67e1d577-9cfd-475f-b758-4ef21fac1a15-console-serving-cert\") pod \"console-cd46f694b-z4vjj\" (UID: \"67e1d577-9cfd-475f-b758-4ef21fac1a15\") " pod="openshift-console/console-cd46f694b-z4vjj" Jul 14 17:33:35.622914 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:35.622853 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/162964a8-ba20-4887-b8b9-e98369835a0d-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"162964a8-ba20-4887-b8b9-e98369835a0d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:33:35.622914 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:35.622882 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/162964a8-ba20-4887-b8b9-e98369835a0d-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"162964a8-ba20-4887-b8b9-e98369835a0d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:33:35.622914 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:35.622907 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/162964a8-ba20-4887-b8b9-e98369835a0d-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"162964a8-ba20-4887-b8b9-e98369835a0d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:33:35.623104 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:35.622932 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/162964a8-ba20-4887-b8b9-e98369835a0d-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"162964a8-ba20-4887-b8b9-e98369835a0d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:33:35.623104 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:35.622968 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/162964a8-ba20-4887-b8b9-e98369835a0d-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"162964a8-ba20-4887-b8b9-e98369835a0d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:33:35.623104 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:35.623005 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/67e1d577-9cfd-475f-b758-4ef21fac1a15-console-oauth-config\") pod \"console-cd46f694b-z4vjj\" (UID: \"67e1d577-9cfd-475f-b758-4ef21fac1a15\") " pod="openshift-console/console-cd46f694b-z4vjj" Jul 14 17:33:35.623104 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:35.623020 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/67e1d577-9cfd-475f-b758-4ef21fac1a15-oauth-serving-cert\") pod \"console-cd46f694b-z4vjj\" (UID: \"67e1d577-9cfd-475f-b758-4ef21fac1a15\") " pod="openshift-console/console-cd46f694b-z4vjj" Jul 14 17:33:35.623104 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:35.623028 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/162964a8-ba20-4887-b8b9-e98369835a0d-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"162964a8-ba20-4887-b8b9-e98369835a0d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:33:35.623104 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:35.623084 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/67e1d577-9cfd-475f-b758-4ef21fac1a15-console-config\") pod \"console-cd46f694b-z4vjj\" (UID: \"67e1d577-9cfd-475f-b758-4ef21fac1a15\") " pod="openshift-console/console-cd46f694b-z4vjj" Jul 14 17:33:35.623412 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:35.623111 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pgrz9\" (UniqueName: \"kubernetes.io/projected/67e1d577-9cfd-475f-b758-4ef21fac1a15-kube-api-access-pgrz9\") pod \"console-cd46f694b-z4vjj\" (UID: \"67e1d577-9cfd-475f-b758-4ef21fac1a15\") " pod="openshift-console/console-cd46f694b-z4vjj" Jul 14 17:33:35.623412 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:35.623137 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/67e1d577-9cfd-475f-b758-4ef21fac1a15-trusted-ca-bundle\") pod \"console-cd46f694b-z4vjj\" (UID: \"67e1d577-9cfd-475f-b758-4ef21fac1a15\") " pod="openshift-console/console-cd46f694b-z4vjj" Jul 14 17:33:35.623412 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:35.623380 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/67e1d577-9cfd-475f-b758-4ef21fac1a15-service-ca\") pod \"console-cd46f694b-z4vjj\" (UID: \"67e1d577-9cfd-475f-b758-4ef21fac1a15\") " pod="openshift-console/console-cd46f694b-z4vjj" Jul 14 17:33:35.623548 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:35.623468 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/162964a8-ba20-4887-b8b9-e98369835a0d-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"162964a8-ba20-4887-b8b9-e98369835a0d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:33:35.624064 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:35.624039 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/67e1d577-9cfd-475f-b758-4ef21fac1a15-console-config\") pod \"console-cd46f694b-z4vjj\" (UID: \"67e1d577-9cfd-475f-b758-4ef21fac1a15\") " pod="openshift-console/console-cd46f694b-z4vjj" Jul 14 17:33:35.625942 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:35.625924 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/67e1d577-9cfd-475f-b758-4ef21fac1a15-console-oauth-config\") pod \"console-cd46f694b-z4vjj\" (UID: \"67e1d577-9cfd-475f-b758-4ef21fac1a15\") " pod="openshift-console/console-cd46f694b-z4vjj" Jul 14 17:33:35.626121 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:35.626098 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/67e1d577-9cfd-475f-b758-4ef21fac1a15-console-serving-cert\") pod \"console-cd46f694b-z4vjj\" (UID: \"67e1d577-9cfd-475f-b758-4ef21fac1a15\") " pod="openshift-console/console-cd46f694b-z4vjj" Jul 14 17:33:35.634963 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:35.634942 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pgrz9\" (UniqueName: \"kubernetes.io/projected/67e1d577-9cfd-475f-b758-4ef21fac1a15-kube-api-access-pgrz9\") pod \"console-cd46f694b-z4vjj\" (UID: \"67e1d577-9cfd-475f-b758-4ef21fac1a15\") " pod="openshift-console/console-cd46f694b-z4vjj" Jul 14 17:33:35.724100 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:35.724058 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/162964a8-ba20-4887-b8b9-e98369835a0d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"162964a8-ba20-4887-b8b9-e98369835a0d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:33:35.724526 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:35.724107 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/162964a8-ba20-4887-b8b9-e98369835a0d-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"162964a8-ba20-4887-b8b9-e98369835a0d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:33:35.724526 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:35.724139 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/162964a8-ba20-4887-b8b9-e98369835a0d-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"162964a8-ba20-4887-b8b9-e98369835a0d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:33:35.724526 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:35.724238 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/162964a8-ba20-4887-b8b9-e98369835a0d-config\") pod \"prometheus-k8s-0\" (UID: \"162964a8-ba20-4887-b8b9-e98369835a0d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:33:35.724526 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:35.724264 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/162964a8-ba20-4887-b8b9-e98369835a0d-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"162964a8-ba20-4887-b8b9-e98369835a0d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:33:35.724526 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:35.724289 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/162964a8-ba20-4887-b8b9-e98369835a0d-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"162964a8-ba20-4887-b8b9-e98369835a0d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:33:35.724526 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:35.724315 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/162964a8-ba20-4887-b8b9-e98369835a0d-config-out\") pod \"prometheus-k8s-0\" (UID: \"162964a8-ba20-4887-b8b9-e98369835a0d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:33:35.724526 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:35.724340 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/162964a8-ba20-4887-b8b9-e98369835a0d-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"162964a8-ba20-4887-b8b9-e98369835a0d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:33:35.724526 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:35.724366 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/162964a8-ba20-4887-b8b9-e98369835a0d-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"162964a8-ba20-4887-b8b9-e98369835a0d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:33:35.724526 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:35.724392 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gwqft\" (UniqueName: \"kubernetes.io/projected/162964a8-ba20-4887-b8b9-e98369835a0d-kube-api-access-gwqft\") pod \"prometheus-k8s-0\" (UID: \"162964a8-ba20-4887-b8b9-e98369835a0d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:33:35.724526 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:35.724421 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/162964a8-ba20-4887-b8b9-e98369835a0d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"162964a8-ba20-4887-b8b9-e98369835a0d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:33:35.724526 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:35.724451 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/162964a8-ba20-4887-b8b9-e98369835a0d-web-config\") pod \"prometheus-k8s-0\" (UID: \"162964a8-ba20-4887-b8b9-e98369835a0d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:33:35.724526 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:35.724478 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/162964a8-ba20-4887-b8b9-e98369835a0d-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"162964a8-ba20-4887-b8b9-e98369835a0d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:33:35.724918 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:35.724545 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/162964a8-ba20-4887-b8b9-e98369835a0d-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"162964a8-ba20-4887-b8b9-e98369835a0d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:33:35.724918 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:35.724575 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/162964a8-ba20-4887-b8b9-e98369835a0d-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"162964a8-ba20-4887-b8b9-e98369835a0d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:33:35.724918 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:35.724598 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/162964a8-ba20-4887-b8b9-e98369835a0d-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"162964a8-ba20-4887-b8b9-e98369835a0d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:33:35.724918 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:35.724624 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/162964a8-ba20-4887-b8b9-e98369835a0d-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"162964a8-ba20-4887-b8b9-e98369835a0d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:33:35.724918 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:35.724665 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/162964a8-ba20-4887-b8b9-e98369835a0d-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"162964a8-ba20-4887-b8b9-e98369835a0d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:33:35.724918 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:35.724706 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/162964a8-ba20-4887-b8b9-e98369835a0d-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"162964a8-ba20-4887-b8b9-e98369835a0d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:33:35.724918 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:35.724741 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/162964a8-ba20-4887-b8b9-e98369835a0d-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"162964a8-ba20-4887-b8b9-e98369835a0d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:33:35.726965 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:35.726853 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/162964a8-ba20-4887-b8b9-e98369835a0d-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"162964a8-ba20-4887-b8b9-e98369835a0d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:33:35.729539 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:35.727753 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/162964a8-ba20-4887-b8b9-e98369835a0d-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"162964a8-ba20-4887-b8b9-e98369835a0d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:33:35.729539 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:35.728370 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/162964a8-ba20-4887-b8b9-e98369835a0d-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"162964a8-ba20-4887-b8b9-e98369835a0d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:33:35.729539 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:35.729117 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/162964a8-ba20-4887-b8b9-e98369835a0d-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"162964a8-ba20-4887-b8b9-e98369835a0d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:33:35.729539 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:35.729222 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/162964a8-ba20-4887-b8b9-e98369835a0d-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"162964a8-ba20-4887-b8b9-e98369835a0d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:33:35.731055 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:35.731030 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/162964a8-ba20-4887-b8b9-e98369835a0d-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"162964a8-ba20-4887-b8b9-e98369835a0d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:33:35.731357 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:35.731334 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/162964a8-ba20-4887-b8b9-e98369835a0d-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"162964a8-ba20-4887-b8b9-e98369835a0d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:33:35.731638 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:35.731612 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/162964a8-ba20-4887-b8b9-e98369835a0d-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"162964a8-ba20-4887-b8b9-e98369835a0d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:33:35.735175 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:35.735100 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/162964a8-ba20-4887-b8b9-e98369835a0d-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"162964a8-ba20-4887-b8b9-e98369835a0d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:33:35.735449 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:35.735421 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/162964a8-ba20-4887-b8b9-e98369835a0d-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"162964a8-ba20-4887-b8b9-e98369835a0d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:33:35.736103 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:35.736074 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/162964a8-ba20-4887-b8b9-e98369835a0d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"162964a8-ba20-4887-b8b9-e98369835a0d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:33:35.736721 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:35.736643 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/162964a8-ba20-4887-b8b9-e98369835a0d-web-config\") pod \"prometheus-k8s-0\" (UID: \"162964a8-ba20-4887-b8b9-e98369835a0d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:33:35.736721 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:35.736681 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/162964a8-ba20-4887-b8b9-e98369835a0d-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"162964a8-ba20-4887-b8b9-e98369835a0d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:33:35.736891 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:35.736867 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/162964a8-ba20-4887-b8b9-e98369835a0d-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"162964a8-ba20-4887-b8b9-e98369835a0d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:33:35.737559 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:35.737517 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/162964a8-ba20-4887-b8b9-e98369835a0d-config-out\") pod \"prometheus-k8s-0\" (UID: \"162964a8-ba20-4887-b8b9-e98369835a0d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:33:35.738160 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:35.738077 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/162964a8-ba20-4887-b8b9-e98369835a0d-config\") pod \"prometheus-k8s-0\" (UID: \"162964a8-ba20-4887-b8b9-e98369835a0d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:33:35.739208 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:35.739167 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gwqft\" (UniqueName: \"kubernetes.io/projected/162964a8-ba20-4887-b8b9-e98369835a0d-kube-api-access-gwqft\") pod \"prometheus-k8s-0\" (UID: \"162964a8-ba20-4887-b8b9-e98369835a0d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:33:35.739719 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:35.739679 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/162964a8-ba20-4887-b8b9-e98369835a0d-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"162964a8-ba20-4887-b8b9-e98369835a0d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:33:35.740315 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:35.740271 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/162964a8-ba20-4887-b8b9-e98369835a0d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"162964a8-ba20-4887-b8b9-e98369835a0d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:33:35.741813 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:35.741783 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/162964a8-ba20-4887-b8b9-e98369835a0d-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"162964a8-ba20-4887-b8b9-e98369835a0d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:33:35.796929 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:35.796899 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-cd46f694b-z4vjj" Jul 14 17:33:35.813091 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:35.813027 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6fd6d747b5-qcc65" event={"ID":"2bcb63de-5908-4bdf-9c18-86937b58b55d","Type":"ContainerStarted","Data":"0b89b76f9d48ed8397e65572059b967bbef18e28f54388a0e3ed5584296fa160"} Jul 14 17:33:35.870779 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:35.870727 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:33:36.109723 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:36.109232 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 14 17:33:36.115893 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:36.115400 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-cd46f694b-z4vjj"] Jul 14 17:33:36.767104 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:36.767083 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-wjkk8" Jul 14 17:33:36.818953 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:36.818914 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-cd46f694b-z4vjj" event={"ID":"67e1d577-9cfd-475f-b758-4ef21fac1a15","Type":"ContainerStarted","Data":"d85889fc836afed25fb9960c1f37e57ece461118cf4dcfce472ab9a9bdb73aeb"} Jul 14 17:33:36.819249 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:36.818961 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-cd46f694b-z4vjj" event={"ID":"67e1d577-9cfd-475f-b758-4ef21fac1a15","Type":"ContainerStarted","Data":"ad986580cdfdd641ed63da25ea5d656cab0979b54df8f8412c895ee9e1c108de"} Jul 14 17:33:36.820604 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:36.820581 2562 generic.go:356] "Generic (PLEG): container finished" podID="162964a8-ba20-4887-b8b9-e98369835a0d" containerID="774ee46e9534c817adfd22129cf38b9a5069b761a444f0b1a6dc9eb5d1419954" exitCode=0 Jul 14 17:33:36.820703 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:36.820657 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"162964a8-ba20-4887-b8b9-e98369835a0d","Type":"ContainerDied","Data":"774ee46e9534c817adfd22129cf38b9a5069b761a444f0b1a6dc9eb5d1419954"} Jul 14 17:33:36.820759 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:36.820731 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"162964a8-ba20-4887-b8b9-e98369835a0d","Type":"ContainerStarted","Data":"37490a16d2fd0352738080f29c2f3d8adf08c49dc5bbf83ac070ae6dd0939f37"} Jul 14 17:33:36.829029 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:36.829004 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9496f0bd-bcba-453e-b249-868c02fac411","Type":"ContainerStarted","Data":"8987cb03f837b6062e4d3dcc65fe66a5031d9d4f5e3ec0bf8f599f27dc13152d"} Jul 14 17:33:36.829131 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:36.829074 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9496f0bd-bcba-453e-b249-868c02fac411","Type":"ContainerStarted","Data":"62b409308e2d72caabdd09589bd74653e7cb03756e5c043c14c88e5a327da2b1"} Jul 14 17:33:36.829131 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:36.829099 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9496f0bd-bcba-453e-b249-868c02fac411","Type":"ContainerStarted","Data":"ec6b33b846b1bc764e90c66f3e78dae7544b643009009d6a357ac49f378d58d8"} Jul 14 17:33:36.829131 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:36.829121 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9496f0bd-bcba-453e-b249-868c02fac411","Type":"ContainerStarted","Data":"7394bf0460524ff21abefd0ec0d487e94747d5c33857dd64a8cd161e5562d7b4"} Jul 14 17:33:36.829303 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:36.829142 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9496f0bd-bcba-453e-b249-868c02fac411","Type":"ContainerStarted","Data":"8f2a41cd79ab0381c72598412420403d55dee3c1308db49e44d26d37df3e5ba5"} Jul 14 17:33:36.829303 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:36.829170 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9496f0bd-bcba-453e-b249-868c02fac411","Type":"ContainerStarted","Data":"fdf56b3ca4c387e31fda8fe854447d4662fe6bd511a2627b68347b213f9d6f74"} Jul 14 17:33:36.831334 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:36.831286 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-69dd5d8d98-25vlb" event={"ID":"15eb5778-e2da-43d6-b724-dde07774086a","Type":"ContainerStarted","Data":"b419bc658516f68e25f25b522cfded8c36d93312ca0844396716bf29fbc2eff4"} Jul 14 17:33:36.835294 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:36.835270 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-57d5c4cc6b-4j54g" event={"ID":"c20e734f-caf7-4ac3-a4f2-4297dfe34ec1","Type":"ContainerStarted","Data":"0cb255dd763ef4de89d831371e19bfb82783287969fb20b358f1bc0f1f95ac11"} Jul 14 17:33:36.835381 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:36.835349 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-57d5c4cc6b-4j54g" event={"ID":"c20e734f-caf7-4ac3-a4f2-4297dfe34ec1","Type":"ContainerStarted","Data":"b04f9242bf50e485ad868e1fb8ba8a0ac950cdd40dd044c94f948dea452c7ee2"} Jul 14 17:33:36.835464 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:36.835392 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-57d5c4cc6b-4j54g" event={"ID":"c20e734f-caf7-4ac3-a4f2-4297dfe34ec1","Type":"ContainerStarted","Data":"f82c953053b10b06100378de2fcdf4a4919f022738205b09eae114a011a1f679"} Jul 14 17:33:36.836008 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:36.835987 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-57d5c4cc6b-4j54g" Jul 14 17:33:36.855659 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:36.855611 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-cd46f694b-z4vjj" podStartSLOduration=1.855592128 podStartE2EDuration="1.855592128s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:33:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:33:36.853810424 +0000 UTC m=+65.930267963" watchObservedRunningTime="2026-07-14 17:33:36.855592128 +0000 UTC m=+65.932049668" Jul 14 17:33:36.884796 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:36.884741 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.976158108 podStartE2EDuration="6.884721071s" totalImagesPullingTime="3.891085177s" totalInitContainerRuntime="17.477786ms" isStatefulPod=false podCreationTimestamp="2026-07-14 17:33:30 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:33:36.883050551 +0000 UTC m=+65.959508091" watchObservedRunningTime="2026-07-14 17:33:36.884721071 +0000 UTC m=+65.961178612" Jul 14 17:33:36.910909 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:36.910857 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-57d5c4cc6b-4j54g" podStartSLOduration=1.7722220480000002 podStartE2EDuration="5.91084211s" totalImagesPullingTime="4.138620062s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:33:31 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:33:36.908881315 +0000 UTC m=+65.985338879" watchObservedRunningTime="2026-07-14 17:33:36.91084211 +0000 UTC m=+65.987299648" Jul 14 17:33:36.930301 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:36.930248 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-69dd5d8d98-25vlb" podStartSLOduration=1.7229503149999998 podStartE2EDuration="3.930227575s" totalImagesPullingTime="2.20727726s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:33:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:33:36.929557574 +0000 UTC m=+66.006015117" watchObservedRunningTime="2026-07-14 17:33:36.930227575 +0000 UTC m=+66.006685114" Jul 14 17:33:37.159812 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:37.159775 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ac89d28e-0c57-4be5-8c3a-f0482ce0ba6f-original-pull-secret\") pod \"global-pull-secret-syncer-kv2bk\" (UID: \"ac89d28e-0c57-4be5-8c3a-f0482ce0ba6f\") " pod="kube-system/global-pull-secret-syncer-kv2bk" Jul 14 17:33:37.159977 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:37.159824 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8321d9c3-c963-45a9-8a94-ef9911a8d33f-metrics-certs\") pod \"network-metrics-daemon-h8t96\" (UID: \"8321d9c3-c963-45a9-8a94-ef9911a8d33f\") " pod="openshift-multus/network-metrics-daemon-h8t96" Jul 14 17:33:37.162347 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:37.162320 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 14 17:33:37.162347 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:37.162338 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 14 17:33:37.173231 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:37.173182 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ac89d28e-0c57-4be5-8c3a-f0482ce0ba6f-original-pull-secret\") pod \"global-pull-secret-syncer-kv2bk\" (UID: \"ac89d28e-0c57-4be5-8c3a-f0482ce0ba6f\") " pod="kube-system/global-pull-secret-syncer-kv2bk" Jul 14 17:33:37.173497 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:37.173478 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8321d9c3-c963-45a9-8a94-ef9911a8d33f-metrics-certs\") pod \"network-metrics-daemon-h8t96\" (UID: \"8321d9c3-c963-45a9-8a94-ef9911a8d33f\") " pod="openshift-multus/network-metrics-daemon-h8t96" Jul 14 17:33:37.261129 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:37.261098 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qqfb\" (UniqueName: \"kubernetes.io/projected/5b2d2388-62da-4462-af63-9230c6c43715-kube-api-access-5qqfb\") pod \"network-check-target-9m2f6\" (UID: \"5b2d2388-62da-4462-af63-9230c6c43715\") " pod="openshift-network-diagnostics/network-check-target-9m2f6" Jul 14 17:33:37.263743 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:37.263721 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 14 17:33:37.274222 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:37.274182 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 14 17:33:37.291786 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:37.291764 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5qqfb\" (UniqueName: \"kubernetes.io/projected/5b2d2388-62da-4462-af63-9230c6c43715-kube-api-access-5qqfb\") pod \"network-check-target-9m2f6\" (UID: \"5b2d2388-62da-4462-af63-9230c6c43715\") " pod="openshift-network-diagnostics/network-check-target-9m2f6" Jul 14 17:33:37.327793 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:37.327768 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-79b6j\"" Jul 14 17:33:37.335782 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:37.335762 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h8t96" Jul 14 17:33:37.338039 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:37.338022 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-q7745\"" Jul 14 17:33:37.340107 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:37.340091 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kv2bk" Jul 14 17:33:37.346228 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:37.346107 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-9m2f6" Jul 14 17:33:37.513112 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:37.512760 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-h8t96"] Jul 14 17:33:37.517053 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:33:37.517010 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8321d9c3_c963_45a9_8a94_ef9911a8d33f.slice/crio-160832de411d17c56cac12aef3164641a772032dfda3b99d93b4a4684aa88533 WatchSource:0}: Error finding container 160832de411d17c56cac12aef3164641a772032dfda3b99d93b4a4684aa88533: Status 404 returned error can't find the container with id 160832de411d17c56cac12aef3164641a772032dfda3b99d93b4a4684aa88533 Jul 14 17:33:37.741428 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:37.741406 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-kv2bk"] Jul 14 17:33:37.744159 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:33:37.744121 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac89d28e_0c57_4be5_8c3a_f0482ce0ba6f.slice/crio-f5c2d4064640350d72e7ce8a52921724e803da74922d8ebc3ab9321e43a6f09b WatchSource:0}: Error finding container f5c2d4064640350d72e7ce8a52921724e803da74922d8ebc3ab9321e43a6f09b: Status 404 returned error can't find the container with id f5c2d4064640350d72e7ce8a52921724e803da74922d8ebc3ab9321e43a6f09b Jul 14 17:33:37.745891 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:37.745862 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-9m2f6"] Jul 14 17:33:37.750292 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:33:37.750264 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b2d2388_62da_4462_af63_9230c6c43715.slice/crio-b2ac397a26c58337aea20f0eb2c5a43d2e33c1c211ac0c45417363107f6f13e9 WatchSource:0}: Error finding container b2ac397a26c58337aea20f0eb2c5a43d2e33c1c211ac0c45417363107f6f13e9: Status 404 returned error can't find the container with id b2ac397a26c58337aea20f0eb2c5a43d2e33c1c211ac0c45417363107f6f13e9 Jul 14 17:33:37.841327 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:37.841295 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-h8t96" event={"ID":"8321d9c3-c963-45a9-8a94-ef9911a8d33f","Type":"ContainerStarted","Data":"160832de411d17c56cac12aef3164641a772032dfda3b99d93b4a4684aa88533"} Jul 14 17:33:37.842724 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:37.842693 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-kv2bk" event={"ID":"ac89d28e-0c57-4be5-8c3a-f0482ce0ba6f","Type":"ContainerStarted","Data":"f5c2d4064640350d72e7ce8a52921724e803da74922d8ebc3ab9321e43a6f09b"} Jul 14 17:33:37.845270 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:37.845243 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6fd6d747b5-qcc65" event={"ID":"2bcb63de-5908-4bdf-9c18-86937b58b55d","Type":"ContainerStarted","Data":"ace34e2c62cec9085e4d92adeeb8572e82b5c12bd690162bae88fe281e6eb9c5"} Jul 14 17:33:37.845380 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:37.845306 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6fd6d747b5-qcc65" event={"ID":"2bcb63de-5908-4bdf-9c18-86937b58b55d","Type":"ContainerStarted","Data":"29988d86084cef9ad0bfe43e31d4f20d22ef3374944e5ab303dbd9e56f4cc01f"} Jul 14 17:33:37.845380 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:37.845340 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6fd6d747b5-qcc65" event={"ID":"2bcb63de-5908-4bdf-9c18-86937b58b55d","Type":"ContainerStarted","Data":"ac249b5bbe86d051b0af85b2829d0f20f149dd337587bc4dd72d53327ee7ce9f"} Jul 14 17:33:37.846467 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:37.846408 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-9m2f6" event={"ID":"5b2d2388-62da-4462-af63-9230c6c43715","Type":"ContainerStarted","Data":"b2ac397a26c58337aea20f0eb2c5a43d2e33c1c211ac0c45417363107f6f13e9"} Jul 14 17:33:37.869677 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:37.869622 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-6fd6d747b5-qcc65" podStartSLOduration=1.505458881 podStartE2EDuration="3.869603583s" totalImagesPullingTime="2.364144702s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:33:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:33:37.867546758 +0000 UTC m=+66.944004333" watchObservedRunningTime="2026-07-14 17:33:37.869603583 +0000 UTC m=+66.946061124" Jul 14 17:33:38.854351 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:38.854249 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-h8t96" event={"ID":"8321d9c3-c963-45a9-8a94-ef9911a8d33f","Type":"ContainerStarted","Data":"cb13ee2c4f8b8b7e275de80495363a2390592dee1d65adbc5f680e78723b401f"} Jul 14 17:33:38.854351 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:38.854330 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-h8t96" event={"ID":"8321d9c3-c963-45a9-8a94-ef9911a8d33f","Type":"ContainerStarted","Data":"28a70eecedf95c0d053d72cb32af280c671dbac1fb434c901561d1961057d4f4"} Jul 14 17:33:40.050831 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:40.048891 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-h8t96" podStartSLOduration=68.060791468 podStartE2EDuration="1m9.048869404s" totalImagesPullingTime="988.077936ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:32:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:33:38.871948744 +0000 UTC m=+67.948406283" watchObservedRunningTime="2026-07-14 17:33:40.048869404 +0000 UTC m=+69.125326944" Jul 14 17:33:40.053405 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:40.053370 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-cd46f694b-z4vjj"] Jul 14 17:33:40.086947 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:40.086917 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-67dfb76bf5-hvq26"] Jul 14 17:33:40.090597 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:40.090571 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-67dfb76bf5-hvq26" Jul 14 17:33:40.111967 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:40.111944 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-67dfb76bf5-hvq26"] Jul 14 17:33:40.191870 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:40.191840 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e190d466-c58f-4604-bf7f-77d668a5cba4-console-oauth-config\") pod \"console-67dfb76bf5-hvq26\" (UID: \"e190d466-c58f-4604-bf7f-77d668a5cba4\") " pod="openshift-console/console-67dfb76bf5-hvq26" Jul 14 17:33:40.191870 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:40.191883 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hgqvd\" (UniqueName: \"kubernetes.io/projected/e190d466-c58f-4604-bf7f-77d668a5cba4-kube-api-access-hgqvd\") pod \"console-67dfb76bf5-hvq26\" (UID: \"e190d466-c58f-4604-bf7f-77d668a5cba4\") " pod="openshift-console/console-67dfb76bf5-hvq26" Jul 14 17:33:40.192101 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:40.191931 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e190d466-c58f-4604-bf7f-77d668a5cba4-console-config\") pod \"console-67dfb76bf5-hvq26\" (UID: \"e190d466-c58f-4604-bf7f-77d668a5cba4\") " pod="openshift-console/console-67dfb76bf5-hvq26" Jul 14 17:33:40.192101 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:40.191975 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e190d466-c58f-4604-bf7f-77d668a5cba4-trusted-ca-bundle\") pod \"console-67dfb76bf5-hvq26\" (UID: \"e190d466-c58f-4604-bf7f-77d668a5cba4\") " pod="openshift-console/console-67dfb76bf5-hvq26" Jul 14 17:33:40.192101 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:40.191995 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e190d466-c58f-4604-bf7f-77d668a5cba4-oauth-serving-cert\") pod \"console-67dfb76bf5-hvq26\" (UID: \"e190d466-c58f-4604-bf7f-77d668a5cba4\") " pod="openshift-console/console-67dfb76bf5-hvq26" Jul 14 17:33:40.192101 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:40.192019 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e190d466-c58f-4604-bf7f-77d668a5cba4-service-ca\") pod \"console-67dfb76bf5-hvq26\" (UID: \"e190d466-c58f-4604-bf7f-77d668a5cba4\") " pod="openshift-console/console-67dfb76bf5-hvq26" Jul 14 17:33:40.192101 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:40.192044 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e190d466-c58f-4604-bf7f-77d668a5cba4-console-serving-cert\") pod \"console-67dfb76bf5-hvq26\" (UID: \"e190d466-c58f-4604-bf7f-77d668a5cba4\") " pod="openshift-console/console-67dfb76bf5-hvq26" Jul 14 17:33:40.294651 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:40.292751 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hgqvd\" (UniqueName: \"kubernetes.io/projected/e190d466-c58f-4604-bf7f-77d668a5cba4-kube-api-access-hgqvd\") pod \"console-67dfb76bf5-hvq26\" (UID: \"e190d466-c58f-4604-bf7f-77d668a5cba4\") " pod="openshift-console/console-67dfb76bf5-hvq26" Jul 14 17:33:40.294651 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:40.292807 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e190d466-c58f-4604-bf7f-77d668a5cba4-console-config\") pod \"console-67dfb76bf5-hvq26\" (UID: \"e190d466-c58f-4604-bf7f-77d668a5cba4\") " pod="openshift-console/console-67dfb76bf5-hvq26" Jul 14 17:33:40.294651 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:40.292838 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e190d466-c58f-4604-bf7f-77d668a5cba4-trusted-ca-bundle\") pod \"console-67dfb76bf5-hvq26\" (UID: \"e190d466-c58f-4604-bf7f-77d668a5cba4\") " pod="openshift-console/console-67dfb76bf5-hvq26" Jul 14 17:33:40.294651 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:40.292861 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e190d466-c58f-4604-bf7f-77d668a5cba4-oauth-serving-cert\") pod \"console-67dfb76bf5-hvq26\" (UID: \"e190d466-c58f-4604-bf7f-77d668a5cba4\") " pod="openshift-console/console-67dfb76bf5-hvq26" Jul 14 17:33:40.294651 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:40.292881 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e190d466-c58f-4604-bf7f-77d668a5cba4-service-ca\") pod \"console-67dfb76bf5-hvq26\" (UID: \"e190d466-c58f-4604-bf7f-77d668a5cba4\") " pod="openshift-console/console-67dfb76bf5-hvq26" Jul 14 17:33:40.294651 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:40.292897 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e190d466-c58f-4604-bf7f-77d668a5cba4-console-serving-cert\") pod \"console-67dfb76bf5-hvq26\" (UID: \"e190d466-c58f-4604-bf7f-77d668a5cba4\") " pod="openshift-console/console-67dfb76bf5-hvq26" Jul 14 17:33:40.294651 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:40.292947 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e190d466-c58f-4604-bf7f-77d668a5cba4-console-oauth-config\") pod \"console-67dfb76bf5-hvq26\" (UID: \"e190d466-c58f-4604-bf7f-77d668a5cba4\") " pod="openshift-console/console-67dfb76bf5-hvq26" Jul 14 17:33:40.294651 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:40.294040 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e190d466-c58f-4604-bf7f-77d668a5cba4-service-ca\") pod \"console-67dfb76bf5-hvq26\" (UID: \"e190d466-c58f-4604-bf7f-77d668a5cba4\") " pod="openshift-console/console-67dfb76bf5-hvq26" Jul 14 17:33:40.294651 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:40.294109 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e190d466-c58f-4604-bf7f-77d668a5cba4-trusted-ca-bundle\") pod \"console-67dfb76bf5-hvq26\" (UID: \"e190d466-c58f-4604-bf7f-77d668a5cba4\") " pod="openshift-console/console-67dfb76bf5-hvq26" Jul 14 17:33:40.294651 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:40.294608 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e190d466-c58f-4604-bf7f-77d668a5cba4-oauth-serving-cert\") pod \"console-67dfb76bf5-hvq26\" (UID: \"e190d466-c58f-4604-bf7f-77d668a5cba4\") " pod="openshift-console/console-67dfb76bf5-hvq26" Jul 14 17:33:40.294651 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:40.294644 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e190d466-c58f-4604-bf7f-77d668a5cba4-console-config\") pod \"console-67dfb76bf5-hvq26\" (UID: \"e190d466-c58f-4604-bf7f-77d668a5cba4\") " pod="openshift-console/console-67dfb76bf5-hvq26" Jul 14 17:33:40.295829 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:40.295804 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e190d466-c58f-4604-bf7f-77d668a5cba4-console-oauth-config\") pod \"console-67dfb76bf5-hvq26\" (UID: \"e190d466-c58f-4604-bf7f-77d668a5cba4\") " pod="openshift-console/console-67dfb76bf5-hvq26" Jul 14 17:33:40.298082 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:40.298062 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e190d466-c58f-4604-bf7f-77d668a5cba4-console-serving-cert\") pod \"console-67dfb76bf5-hvq26\" (UID: \"e190d466-c58f-4604-bf7f-77d668a5cba4\") " pod="openshift-console/console-67dfb76bf5-hvq26" Jul 14 17:33:40.303556 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:40.303501 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hgqvd\" (UniqueName: \"kubernetes.io/projected/e190d466-c58f-4604-bf7f-77d668a5cba4-kube-api-access-hgqvd\") pod \"console-67dfb76bf5-hvq26\" (UID: \"e190d466-c58f-4604-bf7f-77d668a5cba4\") " pod="openshift-console/console-67dfb76bf5-hvq26" Jul 14 17:33:40.403072 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:40.403043 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-67dfb76bf5-hvq26" Jul 14 17:33:42.854170 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:42.854142 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-57d5c4cc6b-4j54g" Jul 14 17:33:43.324445 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:43.324414 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-67dfb76bf5-hvq26"] Jul 14 17:33:43.326302 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:33:43.326275 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode190d466_c58f_4604_bf7f_77d668a5cba4.slice/crio-65107d5dc1432806e0fd19cdb76dacb0d2ef2227dea691b15a54e8d363994ab1 WatchSource:0}: Error finding container 65107d5dc1432806e0fd19cdb76dacb0d2ef2227dea691b15a54e8d363994ab1: Status 404 returned error can't find the container with id 65107d5dc1432806e0fd19cdb76dacb0d2ef2227dea691b15a54e8d363994ab1 Jul 14 17:33:43.876218 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:43.876162 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-kv2bk" event={"ID":"ac89d28e-0c57-4be5-8c3a-f0482ce0ba6f","Type":"ContainerStarted","Data":"62e5a8970f26d5fb607eb158f1500cb6a131529f88ac8f14bc30fb9964be7f1b"} Jul 14 17:33:43.878050 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:43.878014 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-67dfb76bf5-hvq26" event={"ID":"e190d466-c58f-4604-bf7f-77d668a5cba4","Type":"ContainerStarted","Data":"c6b9ce4131cc409da2587e5370555046abc8501d0b8c73efe7aa8c76335a0558"} Jul 14 17:33:43.878230 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:43.878067 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-67dfb76bf5-hvq26" event={"ID":"e190d466-c58f-4604-bf7f-77d668a5cba4","Type":"ContainerStarted","Data":"65107d5dc1432806e0fd19cdb76dacb0d2ef2227dea691b15a54e8d363994ab1"} Jul 14 17:33:43.881633 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:43.881607 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"162964a8-ba20-4887-b8b9-e98369835a0d","Type":"ContainerStarted","Data":"210019baefe3a41037b37b54a22651e239fb4c085cb7af1e3d44667fa347af67"} Jul 14 17:33:43.881747 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:43.881669 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"162964a8-ba20-4887-b8b9-e98369835a0d","Type":"ContainerStarted","Data":"808f856eed06c1568c0bd1a7973a20d1113f8d7dee82f513131fff832f1ba39a"} Jul 14 17:33:43.881747 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:43.881700 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"162964a8-ba20-4887-b8b9-e98369835a0d","Type":"ContainerStarted","Data":"82df51680a9ebde69f9e3d98674a97daa5d013af7d99f3880fa05209ce71ed39"} Jul 14 17:33:43.881747 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:43.881730 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"162964a8-ba20-4887-b8b9-e98369835a0d","Type":"ContainerStarted","Data":"303a1cededc32c93c1f7a013e72ede71495ef80842bd6ee6ddada0bc1f2c5b37"} Jul 14 17:33:43.881875 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:43.881777 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"162964a8-ba20-4887-b8b9-e98369835a0d","Type":"ContainerStarted","Data":"f3f5cacddd0c2a27cdfc51d6e2022c88ec5b13d7d475353933eac7ffdee5a9ae"} Jul 14 17:33:43.881875 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:43.881816 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"162964a8-ba20-4887-b8b9-e98369835a0d","Type":"ContainerStarted","Data":"14eddb98ff4482142de2825a71c0306903354b4c989dba6b63e46f9981a1eb93"} Jul 14 17:33:43.883216 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:43.883170 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-9m2f6" event={"ID":"5b2d2388-62da-4462-af63-9230c6c43715","Type":"ContainerStarted","Data":"dc94b551a6dd853af6724fface3ac4e3d67c84ad948f3dab6c9331fb782565ec"} Jul 14 17:33:43.883320 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:43.883307 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-9m2f6" Jul 14 17:33:43.899546 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:43.899485 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-kv2bk" podStartSLOduration=67.462862802 podStartE2EDuration="1m12.899471133s" totalImagesPullingTime="5.436608331s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 17:32:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:33:43.898842278 +0000 UTC m=+72.975299820" watchObservedRunningTime="2026-07-14 17:33:43.899471133 +0000 UTC m=+72.975928671" Jul 14 17:33:43.925845 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:43.925808 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-67dfb76bf5-hvq26" podStartSLOduration=3.925794296 podStartE2EDuration="3.925794296s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:33:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:33:43.925310462 +0000 UTC m=+73.001768003" watchObservedRunningTime="2026-07-14 17:33:43.925794296 +0000 UTC m=+73.002251835" Jul 14 17:33:43.983853 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:43.983807 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.614682575 podStartE2EDuration="8.983793679s" totalImagesPullingTime="6.347520132s" totalInitContainerRuntime="21.590972ms" isStatefulPod=false podCreationTimestamp="2026-07-14 17:33:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:33:43.982151409 +0000 UTC m=+73.058608948" watchObservedRunningTime="2026-07-14 17:33:43.983793679 +0000 UTC m=+73.060251218" Jul 14 17:33:44.010714 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:44.010678 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-9m2f6" podStartSLOduration=67.587969041 podStartE2EDuration="1m13.010669281s" totalImagesPullingTime="5.42270024s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:32:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:33:44.00976594 +0000 UTC m=+73.086223498" watchObservedRunningTime="2026-07-14 17:33:44.010669281 +0000 UTC m=+73.087126820" Jul 14 17:33:45.797631 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:45.797600 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-cd46f694b-z4vjj" Jul 14 17:33:45.871058 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:45.871023 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:33:50.403669 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:50.403629 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-67dfb76bf5-hvq26" Jul 14 17:33:50.404075 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:50.403694 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-67dfb76bf5-hvq26" Jul 14 17:33:50.408490 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:50.408470 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-67dfb76bf5-hvq26" Jul 14 17:33:50.916155 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:50.916124 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-67dfb76bf5-hvq26" Jul 14 17:33:50.967785 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:50.967758 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7944586585-9b8sh"] Jul 14 17:33:53.986903 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:53.986790 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-69dd5d8d98-25vlb" Jul 14 17:33:53.986903 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:53.986854 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-69dd5d8d98-25vlb" Jul 14 17:33:56.899812 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:56.899767 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-8669f6cd78-rjntn" podUID="b56ef57c-4634-40df-a68d-e4ae43d07c3d" containerName="console" containerID="cri-o://e4ed7b64c09216a66ce320bd69b5427bf77fb7bf1f11d83d617bf3b41105aa56" gracePeriod=15 Jul 14 17:33:57.676634 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:57.676605 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8669f6cd78-rjntn" Jul 14 17:33:57.859215 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:57.859104 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b56ef57c-4634-40df-a68d-e4ae43d07c3d-console-config\") pod \"b56ef57c-4634-40df-a68d-e4ae43d07c3d\" (UID: \"b56ef57c-4634-40df-a68d-e4ae43d07c3d\") " Jul 14 17:33:57.859215 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:57.859142 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b56ef57c-4634-40df-a68d-e4ae43d07c3d-console-oauth-config\") pod \"b56ef57c-4634-40df-a68d-e4ae43d07c3d\" (UID: \"b56ef57c-4634-40df-a68d-e4ae43d07c3d\") " Jul 14 17:33:57.859215 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:57.859182 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b56ef57c-4634-40df-a68d-e4ae43d07c3d-service-ca\") pod \"b56ef57c-4634-40df-a68d-e4ae43d07c3d\" (UID: \"b56ef57c-4634-40df-a68d-e4ae43d07c3d\") " Jul 14 17:33:57.859499 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:57.859222 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b56ef57c-4634-40df-a68d-e4ae43d07c3d-console-serving-cert\") pod \"b56ef57c-4634-40df-a68d-e4ae43d07c3d\" (UID: \"b56ef57c-4634-40df-a68d-e4ae43d07c3d\") " Jul 14 17:33:57.859499 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:57.859280 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7dlbm\" (UniqueName: \"kubernetes.io/projected/b56ef57c-4634-40df-a68d-e4ae43d07c3d-kube-api-access-7dlbm\") pod \"b56ef57c-4634-40df-a68d-e4ae43d07c3d\" (UID: \"b56ef57c-4634-40df-a68d-e4ae43d07c3d\") " Jul 14 17:33:57.859499 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:57.859344 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b56ef57c-4634-40df-a68d-e4ae43d07c3d-oauth-serving-cert\") pod \"b56ef57c-4634-40df-a68d-e4ae43d07c3d\" (UID: \"b56ef57c-4634-40df-a68d-e4ae43d07c3d\") " Jul 14 17:33:57.859707 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:57.859678 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b56ef57c-4634-40df-a68d-e4ae43d07c3d-console-config" (OuterVolumeSpecName: "console-config") pod "b56ef57c-4634-40df-a68d-e4ae43d07c3d" (UID: "b56ef57c-4634-40df-a68d-e4ae43d07c3d"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 17:33:57.859800 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:57.859691 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b56ef57c-4634-40df-a68d-e4ae43d07c3d-service-ca" (OuterVolumeSpecName: "service-ca") pod "b56ef57c-4634-40df-a68d-e4ae43d07c3d" (UID: "b56ef57c-4634-40df-a68d-e4ae43d07c3d"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 17:33:57.859800 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:57.859782 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b56ef57c-4634-40df-a68d-e4ae43d07c3d-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "b56ef57c-4634-40df-a68d-e4ae43d07c3d" (UID: "b56ef57c-4634-40df-a68d-e4ae43d07c3d"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 17:33:57.861595 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:57.861564 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b56ef57c-4634-40df-a68d-e4ae43d07c3d-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "b56ef57c-4634-40df-a68d-e4ae43d07c3d" (UID: "b56ef57c-4634-40df-a68d-e4ae43d07c3d"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 17:33:57.861744 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:57.861719 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b56ef57c-4634-40df-a68d-e4ae43d07c3d-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "b56ef57c-4634-40df-a68d-e4ae43d07c3d" (UID: "b56ef57c-4634-40df-a68d-e4ae43d07c3d"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 17:33:57.861807 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:57.861761 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b56ef57c-4634-40df-a68d-e4ae43d07c3d-kube-api-access-7dlbm" (OuterVolumeSpecName: "kube-api-access-7dlbm") pod "b56ef57c-4634-40df-a68d-e4ae43d07c3d" (UID: "b56ef57c-4634-40df-a68d-e4ae43d07c3d"). InnerVolumeSpecName "kube-api-access-7dlbm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 17:33:57.942136 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:57.942095 2562 generic.go:356] "Generic (PLEG): container finished" podID="b56ef57c-4634-40df-a68d-e4ae43d07c3d" containerID="e4ed7b64c09216a66ce320bd69b5427bf77fb7bf1f11d83d617bf3b41105aa56" exitCode=0 Jul 14 17:33:57.942533 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:57.942144 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8669f6cd78-rjntn" Jul 14 17:33:57.942533 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:57.942169 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8669f6cd78-rjntn" event={"ID":"b56ef57c-4634-40df-a68d-e4ae43d07c3d","Type":"ContainerDied","Data":"e4ed7b64c09216a66ce320bd69b5427bf77fb7bf1f11d83d617bf3b41105aa56"} Jul 14 17:33:57.942533 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:57.942237 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8669f6cd78-rjntn" event={"ID":"b56ef57c-4634-40df-a68d-e4ae43d07c3d","Type":"ContainerDied","Data":"a2f12cc32c4904ada497dae43f4f2b4a40a0273f5a5b566d47b033a9ea04c155"} Jul 14 17:33:57.942533 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:57.942260 2562 scope.go:117] "RemoveContainer" containerID="e4ed7b64c09216a66ce320bd69b5427bf77fb7bf1f11d83d617bf3b41105aa56" Jul 14 17:33:57.951499 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:57.951484 2562 scope.go:117] "RemoveContainer" containerID="e4ed7b64c09216a66ce320bd69b5427bf77fb7bf1f11d83d617bf3b41105aa56" Jul 14 17:33:57.951757 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:33:57.951738 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e4ed7b64c09216a66ce320bd69b5427bf77fb7bf1f11d83d617bf3b41105aa56\": container with ID starting with e4ed7b64c09216a66ce320bd69b5427bf77fb7bf1f11d83d617bf3b41105aa56 not found: ID does not exist" containerID="e4ed7b64c09216a66ce320bd69b5427bf77fb7bf1f11d83d617bf3b41105aa56" Jul 14 17:33:57.951823 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:57.951769 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e4ed7b64c09216a66ce320bd69b5427bf77fb7bf1f11d83d617bf3b41105aa56"} err="failed to get container status \"e4ed7b64c09216a66ce320bd69b5427bf77fb7bf1f11d83d617bf3b41105aa56\": rpc error: code = NotFound desc = could not find container \"e4ed7b64c09216a66ce320bd69b5427bf77fb7bf1f11d83d617bf3b41105aa56\": container with ID starting with e4ed7b64c09216a66ce320bd69b5427bf77fb7bf1f11d83d617bf3b41105aa56 not found: ID does not exist" Jul 14 17:33:57.960252 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:57.960226 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7dlbm\" (UniqueName: \"kubernetes.io/projected/b56ef57c-4634-40df-a68d-e4ae43d07c3d-kube-api-access-7dlbm\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 17:33:57.960355 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:57.960251 2562 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b56ef57c-4634-40df-a68d-e4ae43d07c3d-oauth-serving-cert\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 17:33:57.960355 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:57.960266 2562 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b56ef57c-4634-40df-a68d-e4ae43d07c3d-console-config\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 17:33:57.960355 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:57.960275 2562 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b56ef57c-4634-40df-a68d-e4ae43d07c3d-console-oauth-config\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 17:33:57.960355 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:57.960283 2562 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b56ef57c-4634-40df-a68d-e4ae43d07c3d-service-ca\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 17:33:57.960355 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:57.960292 2562 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b56ef57c-4634-40df-a68d-e4ae43d07c3d-console-serving-cert\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 17:33:57.962787 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:57.962767 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8669f6cd78-rjntn"] Jul 14 17:33:57.966014 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:57.965994 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-8669f6cd78-rjntn"] Jul 14 17:33:59.519244 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:33:59.519209 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b56ef57c-4634-40df-a68d-e4ae43d07c3d" path="/var/lib/kubelet/pods/b56ef57c-4634-40df-a68d-e4ae43d07c3d/volumes" Jul 14 17:34:05.081765 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:05.081722 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-cd46f694b-z4vjj" podUID="67e1d577-9cfd-475f-b758-4ef21fac1a15" containerName="console" containerID="cri-o://d85889fc836afed25fb9960c1f37e57ece461118cf4dcfce472ab9a9bdb73aeb" gracePeriod=15 Jul 14 17:34:05.329403 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:05.329372 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-cd46f694b-z4vjj" Jul 14 17:34:05.430095 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:05.430070 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/67e1d577-9cfd-475f-b758-4ef21fac1a15-service-ca\") pod \"67e1d577-9cfd-475f-b758-4ef21fac1a15\" (UID: \"67e1d577-9cfd-475f-b758-4ef21fac1a15\") " Jul 14 17:34:05.430284 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:05.430103 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/67e1d577-9cfd-475f-b758-4ef21fac1a15-trusted-ca-bundle\") pod \"67e1d577-9cfd-475f-b758-4ef21fac1a15\" (UID: \"67e1d577-9cfd-475f-b758-4ef21fac1a15\") " Jul 14 17:34:05.430284 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:05.430236 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/67e1d577-9cfd-475f-b758-4ef21fac1a15-console-oauth-config\") pod \"67e1d577-9cfd-475f-b758-4ef21fac1a15\" (UID: \"67e1d577-9cfd-475f-b758-4ef21fac1a15\") " Jul 14 17:34:05.430421 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:05.430287 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/67e1d577-9cfd-475f-b758-4ef21fac1a15-console-config\") pod \"67e1d577-9cfd-475f-b758-4ef21fac1a15\" (UID: \"67e1d577-9cfd-475f-b758-4ef21fac1a15\") " Jul 14 17:34:05.430421 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:05.430325 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/67e1d577-9cfd-475f-b758-4ef21fac1a15-oauth-serving-cert\") pod \"67e1d577-9cfd-475f-b758-4ef21fac1a15\" (UID: \"67e1d577-9cfd-475f-b758-4ef21fac1a15\") " Jul 14 17:34:05.430421 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:05.430402 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/67e1d577-9cfd-475f-b758-4ef21fac1a15-console-serving-cert\") pod \"67e1d577-9cfd-475f-b758-4ef21fac1a15\" (UID: \"67e1d577-9cfd-475f-b758-4ef21fac1a15\") " Jul 14 17:34:05.430567 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:05.430435 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pgrz9\" (UniqueName: \"kubernetes.io/projected/67e1d577-9cfd-475f-b758-4ef21fac1a15-kube-api-access-pgrz9\") pod \"67e1d577-9cfd-475f-b758-4ef21fac1a15\" (UID: \"67e1d577-9cfd-475f-b758-4ef21fac1a15\") " Jul 14 17:34:05.430567 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:05.430437 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/67e1d577-9cfd-475f-b758-4ef21fac1a15-service-ca" (OuterVolumeSpecName: "service-ca") pod "67e1d577-9cfd-475f-b758-4ef21fac1a15" (UID: "67e1d577-9cfd-475f-b758-4ef21fac1a15"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 17:34:05.430695 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:05.430671 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/67e1d577-9cfd-475f-b758-4ef21fac1a15-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "67e1d577-9cfd-475f-b758-4ef21fac1a15" (UID: "67e1d577-9cfd-475f-b758-4ef21fac1a15"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 17:34:05.430746 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:05.430698 2562 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/67e1d577-9cfd-475f-b758-4ef21fac1a15-service-ca\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 17:34:05.430746 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:05.430691 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/67e1d577-9cfd-475f-b758-4ef21fac1a15-console-config" (OuterVolumeSpecName: "console-config") pod "67e1d577-9cfd-475f-b758-4ef21fac1a15" (UID: "67e1d577-9cfd-475f-b758-4ef21fac1a15"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 17:34:05.430746 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:05.430697 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/67e1d577-9cfd-475f-b758-4ef21fac1a15-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "67e1d577-9cfd-475f-b758-4ef21fac1a15" (UID: "67e1d577-9cfd-475f-b758-4ef21fac1a15"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 17:34:05.432691 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:05.432665 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/67e1d577-9cfd-475f-b758-4ef21fac1a15-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "67e1d577-9cfd-475f-b758-4ef21fac1a15" (UID: "67e1d577-9cfd-475f-b758-4ef21fac1a15"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 17:34:05.432787 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:05.432692 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/67e1d577-9cfd-475f-b758-4ef21fac1a15-kube-api-access-pgrz9" (OuterVolumeSpecName: "kube-api-access-pgrz9") pod "67e1d577-9cfd-475f-b758-4ef21fac1a15" (UID: "67e1d577-9cfd-475f-b758-4ef21fac1a15"). InnerVolumeSpecName "kube-api-access-pgrz9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 17:34:05.432787 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:05.432710 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/67e1d577-9cfd-475f-b758-4ef21fac1a15-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "67e1d577-9cfd-475f-b758-4ef21fac1a15" (UID: "67e1d577-9cfd-475f-b758-4ef21fac1a15"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 17:34:05.531028 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:05.531003 2562 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/67e1d577-9cfd-475f-b758-4ef21fac1a15-console-oauth-config\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 17:34:05.531028 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:05.531027 2562 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/67e1d577-9cfd-475f-b758-4ef21fac1a15-console-config\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 17:34:05.531166 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:05.531037 2562 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/67e1d577-9cfd-475f-b758-4ef21fac1a15-oauth-serving-cert\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 17:34:05.531166 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:05.531045 2562 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/67e1d577-9cfd-475f-b758-4ef21fac1a15-console-serving-cert\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 17:34:05.531166 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:05.531054 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pgrz9\" (UniqueName: \"kubernetes.io/projected/67e1d577-9cfd-475f-b758-4ef21fac1a15-kube-api-access-pgrz9\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 17:34:05.531166 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:05.531063 2562 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/67e1d577-9cfd-475f-b758-4ef21fac1a15-trusted-ca-bundle\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 17:34:05.975790 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:05.975756 2562 generic.go:356] "Generic (PLEG): container finished" podID="67e1d577-9cfd-475f-b758-4ef21fac1a15" containerID="d85889fc836afed25fb9960c1f37e57ece461118cf4dcfce472ab9a9bdb73aeb" exitCode=0 Jul 14 17:34:05.975968 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:05.975816 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-cd46f694b-z4vjj" Jul 14 17:34:05.975968 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:05.975852 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-cd46f694b-z4vjj" event={"ID":"67e1d577-9cfd-475f-b758-4ef21fac1a15","Type":"ContainerDied","Data":"d85889fc836afed25fb9960c1f37e57ece461118cf4dcfce472ab9a9bdb73aeb"} Jul 14 17:34:05.975968 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:05.975942 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-cd46f694b-z4vjj" event={"ID":"67e1d577-9cfd-475f-b758-4ef21fac1a15","Type":"ContainerDied","Data":"ad986580cdfdd641ed63da25ea5d656cab0979b54df8f8412c895ee9e1c108de"} Jul 14 17:34:05.976131 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:05.975971 2562 scope.go:117] "RemoveContainer" containerID="d85889fc836afed25fb9960c1f37e57ece461118cf4dcfce472ab9a9bdb73aeb" Jul 14 17:34:05.984003 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:05.983982 2562 scope.go:117] "RemoveContainer" containerID="d85889fc836afed25fb9960c1f37e57ece461118cf4dcfce472ab9a9bdb73aeb" Jul 14 17:34:05.984256 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:34:05.984238 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d85889fc836afed25fb9960c1f37e57ece461118cf4dcfce472ab9a9bdb73aeb\": container with ID starting with d85889fc836afed25fb9960c1f37e57ece461118cf4dcfce472ab9a9bdb73aeb not found: ID does not exist" containerID="d85889fc836afed25fb9960c1f37e57ece461118cf4dcfce472ab9a9bdb73aeb" Jul 14 17:34:05.984304 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:05.984263 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d85889fc836afed25fb9960c1f37e57ece461118cf4dcfce472ab9a9bdb73aeb"} err="failed to get container status \"d85889fc836afed25fb9960c1f37e57ece461118cf4dcfce472ab9a9bdb73aeb\": rpc error: code = NotFound desc = could not find container \"d85889fc836afed25fb9960c1f37e57ece461118cf4dcfce472ab9a9bdb73aeb\": container with ID starting with d85889fc836afed25fb9960c1f37e57ece461118cf4dcfce472ab9a9bdb73aeb not found: ID does not exist" Jul 14 17:34:06.003235 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:06.003215 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-cd46f694b-z4vjj"] Jul 14 17:34:06.007584 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:06.007563 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-cd46f694b-z4vjj"] Jul 14 17:34:07.518866 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:07.518836 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="67e1d577-9cfd-475f-b758-4ef21fac1a15" path="/var/lib/kubelet/pods/67e1d577-9cfd-475f-b758-4ef21fac1a15/volumes" Jul 14 17:34:13.991868 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:13.991837 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-69dd5d8d98-25vlb" Jul 14 17:34:13.995716 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:13.995698 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-69dd5d8d98-25vlb" Jul 14 17:34:14.890332 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:14.890300 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-9m2f6" Jul 14 17:34:15.993424 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:15.993384 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7944586585-9b8sh" podUID="1c51ee65-7247-4790-885c-304ae178695c" containerName="console" containerID="cri-o://f7bb6ab7de848d6a44dc4d2b81309303532a72d9101b17cc39fbee177f80ecee" gracePeriod=15 Jul 14 17:34:16.261154 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:16.261095 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7944586585-9b8sh" Jul 14 17:34:16.322804 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:16.322769 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1c51ee65-7247-4790-885c-304ae178695c-console-oauth-config\") pod \"1c51ee65-7247-4790-885c-304ae178695c\" (UID: \"1c51ee65-7247-4790-885c-304ae178695c\") " Jul 14 17:34:16.322965 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:16.322819 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1c51ee65-7247-4790-885c-304ae178695c-console-serving-cert\") pod \"1c51ee65-7247-4790-885c-304ae178695c\" (UID: \"1c51ee65-7247-4790-885c-304ae178695c\") " Jul 14 17:34:16.322965 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:16.322855 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1c51ee65-7247-4790-885c-304ae178695c-trusted-ca-bundle\") pod \"1c51ee65-7247-4790-885c-304ae178695c\" (UID: \"1c51ee65-7247-4790-885c-304ae178695c\") " Jul 14 17:34:16.322965 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:16.322878 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1c51ee65-7247-4790-885c-304ae178695c-console-config\") pod \"1c51ee65-7247-4790-885c-304ae178695c\" (UID: \"1c51ee65-7247-4790-885c-304ae178695c\") " Jul 14 17:34:16.322965 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:16.322896 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1c51ee65-7247-4790-885c-304ae178695c-service-ca\") pod \"1c51ee65-7247-4790-885c-304ae178695c\" (UID: \"1c51ee65-7247-4790-885c-304ae178695c\") " Jul 14 17:34:16.322965 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:16.322918 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-djx6r\" (UniqueName: \"kubernetes.io/projected/1c51ee65-7247-4790-885c-304ae178695c-kube-api-access-djx6r\") pod \"1c51ee65-7247-4790-885c-304ae178695c\" (UID: \"1c51ee65-7247-4790-885c-304ae178695c\") " Jul 14 17:34:16.322965 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:16.322953 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1c51ee65-7247-4790-885c-304ae178695c-oauth-serving-cert\") pod \"1c51ee65-7247-4790-885c-304ae178695c\" (UID: \"1c51ee65-7247-4790-885c-304ae178695c\") " Jul 14 17:34:16.323354 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:16.323322 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1c51ee65-7247-4790-885c-304ae178695c-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "1c51ee65-7247-4790-885c-304ae178695c" (UID: "1c51ee65-7247-4790-885c-304ae178695c"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 17:34:16.323605 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:16.323576 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1c51ee65-7247-4790-885c-304ae178695c-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "1c51ee65-7247-4790-885c-304ae178695c" (UID: "1c51ee65-7247-4790-885c-304ae178695c"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 17:34:16.323715 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:16.323441 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1c51ee65-7247-4790-885c-304ae178695c-console-config" (OuterVolumeSpecName: "console-config") pod "1c51ee65-7247-4790-885c-304ae178695c" (UID: "1c51ee65-7247-4790-885c-304ae178695c"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 17:34:16.323775 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:16.323468 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1c51ee65-7247-4790-885c-304ae178695c-service-ca" (OuterVolumeSpecName: "service-ca") pod "1c51ee65-7247-4790-885c-304ae178695c" (UID: "1c51ee65-7247-4790-885c-304ae178695c"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 17:34:16.325381 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:16.325353 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1c51ee65-7247-4790-885c-304ae178695c-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "1c51ee65-7247-4790-885c-304ae178695c" (UID: "1c51ee65-7247-4790-885c-304ae178695c"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 17:34:16.325466 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:16.325395 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1c51ee65-7247-4790-885c-304ae178695c-kube-api-access-djx6r" (OuterVolumeSpecName: "kube-api-access-djx6r") pod "1c51ee65-7247-4790-885c-304ae178695c" (UID: "1c51ee65-7247-4790-885c-304ae178695c"). InnerVolumeSpecName "kube-api-access-djx6r". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 17:34:16.325466 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:16.325406 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1c51ee65-7247-4790-885c-304ae178695c-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "1c51ee65-7247-4790-885c-304ae178695c" (UID: "1c51ee65-7247-4790-885c-304ae178695c"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 17:34:16.423672 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:16.423633 2562 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1c51ee65-7247-4790-885c-304ae178695c-console-oauth-config\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 17:34:16.423672 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:16.423663 2562 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1c51ee65-7247-4790-885c-304ae178695c-console-serving-cert\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 17:34:16.423672 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:16.423673 2562 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1c51ee65-7247-4790-885c-304ae178695c-trusted-ca-bundle\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 17:34:16.423672 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:16.423683 2562 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1c51ee65-7247-4790-885c-304ae178695c-console-config\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 17:34:16.423942 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:16.423691 2562 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1c51ee65-7247-4790-885c-304ae178695c-service-ca\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 17:34:16.423942 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:16.423700 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-djx6r\" (UniqueName: \"kubernetes.io/projected/1c51ee65-7247-4790-885c-304ae178695c-kube-api-access-djx6r\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 17:34:16.423942 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:16.423709 2562 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1c51ee65-7247-4790-885c-304ae178695c-oauth-serving-cert\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 17:34:17.019949 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:17.019908 2562 generic.go:356] "Generic (PLEG): container finished" podID="1c51ee65-7247-4790-885c-304ae178695c" containerID="f7bb6ab7de848d6a44dc4d2b81309303532a72d9101b17cc39fbee177f80ecee" exitCode=0 Jul 14 17:34:17.020417 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:17.020000 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7944586585-9b8sh" Jul 14 17:34:17.020417 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:17.020005 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7944586585-9b8sh" event={"ID":"1c51ee65-7247-4790-885c-304ae178695c","Type":"ContainerDied","Data":"f7bb6ab7de848d6a44dc4d2b81309303532a72d9101b17cc39fbee177f80ecee"} Jul 14 17:34:17.020417 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:17.020107 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7944586585-9b8sh" event={"ID":"1c51ee65-7247-4790-885c-304ae178695c","Type":"ContainerDied","Data":"3e11060635018417ba818105204e2e7ef8583606cdb954358ca53c822f456107"} Jul 14 17:34:17.020417 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:17.020139 2562 scope.go:117] "RemoveContainer" containerID="f7bb6ab7de848d6a44dc4d2b81309303532a72d9101b17cc39fbee177f80ecee" Jul 14 17:34:17.028600 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:17.028580 2562 scope.go:117] "RemoveContainer" containerID="f7bb6ab7de848d6a44dc4d2b81309303532a72d9101b17cc39fbee177f80ecee" Jul 14 17:34:17.028913 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:34:17.028889 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f7bb6ab7de848d6a44dc4d2b81309303532a72d9101b17cc39fbee177f80ecee\": container with ID starting with f7bb6ab7de848d6a44dc4d2b81309303532a72d9101b17cc39fbee177f80ecee not found: ID does not exist" containerID="f7bb6ab7de848d6a44dc4d2b81309303532a72d9101b17cc39fbee177f80ecee" Jul 14 17:34:17.029008 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:17.028917 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7bb6ab7de848d6a44dc4d2b81309303532a72d9101b17cc39fbee177f80ecee"} err="failed to get container status \"f7bb6ab7de848d6a44dc4d2b81309303532a72d9101b17cc39fbee177f80ecee\": rpc error: code = NotFound desc = could not find container \"f7bb6ab7de848d6a44dc4d2b81309303532a72d9101b17cc39fbee177f80ecee\": container with ID starting with f7bb6ab7de848d6a44dc4d2b81309303532a72d9101b17cc39fbee177f80ecee not found: ID does not exist" Jul 14 17:34:17.057954 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:17.057927 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7944586585-9b8sh"] Jul 14 17:34:17.063424 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:17.063402 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7944586585-9b8sh"] Jul 14 17:34:17.519381 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:17.519347 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1c51ee65-7247-4790-885c-304ae178695c" path="/var/lib/kubelet/pods/1c51ee65-7247-4790-885c-304ae178695c/volumes" Jul 14 17:34:35.871746 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:35.871710 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:34:35.891744 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:35.891722 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:34:36.118067 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:36.118040 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:34:49.722877 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:49.722841 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 14 17:34:49.723512 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:49.723164 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="9496f0bd-bcba-453e-b249-868c02fac411" containerName="alertmanager" containerID="cri-o://fdf56b3ca4c387e31fda8fe854447d4662fe6bd511a2627b68347b213f9d6f74" gracePeriod=120 Jul 14 17:34:49.723512 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:49.723230 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="9496f0bd-bcba-453e-b249-868c02fac411" containerName="kube-rbac-proxy-metric" containerID="cri-o://62b409308e2d72caabdd09589bd74653e7cb03756e5c043c14c88e5a327da2b1" gracePeriod=120 Jul 14 17:34:49.723512 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:49.723272 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="9496f0bd-bcba-453e-b249-868c02fac411" containerName="prom-label-proxy" containerID="cri-o://8987cb03f837b6062e4d3dcc65fe66a5031d9d4f5e3ec0bf8f599f27dc13152d" gracePeriod=120 Jul 14 17:34:49.723512 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:49.723400 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="9496f0bd-bcba-453e-b249-868c02fac411" containerName="kube-rbac-proxy-web" containerID="cri-o://7394bf0460524ff21abefd0ec0d487e94747d5c33857dd64a8cd161e5562d7b4" gracePeriod=120 Jul 14 17:34:49.723512 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:49.723425 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="9496f0bd-bcba-453e-b249-868c02fac411" containerName="config-reloader" containerID="cri-o://8f2a41cd79ab0381c72598412420403d55dee3c1308db49e44d26d37df3e5ba5" gracePeriod=120 Jul 14 17:34:49.723512 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:49.723450 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="9496f0bd-bcba-453e-b249-868c02fac411" containerName="kube-rbac-proxy" containerID="cri-o://ec6b33b846b1bc764e90c66f3e78dae7544b643009009d6a357ac49f378d58d8" gracePeriod=120 Jul 14 17:34:50.159296 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:50.159265 2562 generic.go:356] "Generic (PLEG): container finished" podID="9496f0bd-bcba-453e-b249-868c02fac411" containerID="8987cb03f837b6062e4d3dcc65fe66a5031d9d4f5e3ec0bf8f599f27dc13152d" exitCode=0 Jul 14 17:34:50.159296 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:50.159290 2562 generic.go:356] "Generic (PLEG): container finished" podID="9496f0bd-bcba-453e-b249-868c02fac411" containerID="ec6b33b846b1bc764e90c66f3e78dae7544b643009009d6a357ac49f378d58d8" exitCode=0 Jul 14 17:34:50.159296 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:50.159298 2562 generic.go:356] "Generic (PLEG): container finished" podID="9496f0bd-bcba-453e-b249-868c02fac411" containerID="8f2a41cd79ab0381c72598412420403d55dee3c1308db49e44d26d37df3e5ba5" exitCode=0 Jul 14 17:34:50.159296 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:50.159304 2562 generic.go:356] "Generic (PLEG): container finished" podID="9496f0bd-bcba-453e-b249-868c02fac411" containerID="fdf56b3ca4c387e31fda8fe854447d4662fe6bd511a2627b68347b213f9d6f74" exitCode=0 Jul 14 17:34:50.159531 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:50.159334 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9496f0bd-bcba-453e-b249-868c02fac411","Type":"ContainerDied","Data":"8987cb03f837b6062e4d3dcc65fe66a5031d9d4f5e3ec0bf8f599f27dc13152d"} Jul 14 17:34:50.159531 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:50.159406 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9496f0bd-bcba-453e-b249-868c02fac411","Type":"ContainerDied","Data":"ec6b33b846b1bc764e90c66f3e78dae7544b643009009d6a357ac49f378d58d8"} Jul 14 17:34:50.159531 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:50.159433 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9496f0bd-bcba-453e-b249-868c02fac411","Type":"ContainerDied","Data":"8f2a41cd79ab0381c72598412420403d55dee3c1308db49e44d26d37df3e5ba5"} Jul 14 17:34:50.159531 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:50.159457 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9496f0bd-bcba-453e-b249-868c02fac411","Type":"ContainerDied","Data":"fdf56b3ca4c387e31fda8fe854447d4662fe6bd511a2627b68347b213f9d6f74"} Jul 14 17:34:50.967464 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:50.967442 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:34:51.024816 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.024786 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/9496f0bd-bcba-453e-b249-868c02fac411-cluster-tls-config\") pod \"9496f0bd-bcba-453e-b249-868c02fac411\" (UID: \"9496f0bd-bcba-453e-b249-868c02fac411\") " Jul 14 17:34:51.024959 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.024855 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9496f0bd-bcba-453e-b249-868c02fac411-secret-alertmanager-kube-rbac-proxy-web\") pod \"9496f0bd-bcba-453e-b249-868c02fac411\" (UID: \"9496f0bd-bcba-453e-b249-868c02fac411\") " Jul 14 17:34:51.024959 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.024903 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9496f0bd-bcba-453e-b249-868c02fac411-config-out\") pod \"9496f0bd-bcba-453e-b249-868c02fac411\" (UID: \"9496f0bd-bcba-453e-b249-868c02fac411\") " Jul 14 17:34:51.024959 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.024934 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9496f0bd-bcba-453e-b249-868c02fac411-metrics-client-ca\") pod \"9496f0bd-bcba-453e-b249-868c02fac411\" (UID: \"9496f0bd-bcba-453e-b249-868c02fac411\") " Jul 14 17:34:51.025108 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.024960 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/9496f0bd-bcba-453e-b249-868c02fac411-secret-alertmanager-main-tls\") pod \"9496f0bd-bcba-453e-b249-868c02fac411\" (UID: \"9496f0bd-bcba-453e-b249-868c02fac411\") " Jul 14 17:34:51.025108 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.024990 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/9496f0bd-bcba-453e-b249-868c02fac411-config-volume\") pod \"9496f0bd-bcba-453e-b249-868c02fac411\" (UID: \"9496f0bd-bcba-453e-b249-868c02fac411\") " Jul 14 17:34:51.025108 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.025020 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9496f0bd-bcba-453e-b249-868c02fac411-secret-alertmanager-kube-rbac-proxy\") pod \"9496f0bd-bcba-453e-b249-868c02fac411\" (UID: \"9496f0bd-bcba-453e-b249-868c02fac411\") " Jul 14 17:34:51.025108 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.025048 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pkfjp\" (UniqueName: \"kubernetes.io/projected/9496f0bd-bcba-453e-b249-868c02fac411-kube-api-access-pkfjp\") pod \"9496f0bd-bcba-453e-b249-868c02fac411\" (UID: \"9496f0bd-bcba-453e-b249-868c02fac411\") " Jul 14 17:34:51.025108 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.025082 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/9496f0bd-bcba-453e-b249-868c02fac411-secret-alertmanager-kube-rbac-proxy-metric\") pod \"9496f0bd-bcba-453e-b249-868c02fac411\" (UID: \"9496f0bd-bcba-453e-b249-868c02fac411\") " Jul 14 17:34:51.025392 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.025112 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9496f0bd-bcba-453e-b249-868c02fac411-tls-assets\") pod \"9496f0bd-bcba-453e-b249-868c02fac411\" (UID: \"9496f0bd-bcba-453e-b249-868c02fac411\") " Jul 14 17:34:51.026504 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.026467 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9496f0bd-bcba-453e-b249-868c02fac411-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "9496f0bd-bcba-453e-b249-868c02fac411" (UID: "9496f0bd-bcba-453e-b249-868c02fac411"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 17:34:51.026650 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.026537 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9496f0bd-bcba-453e-b249-868c02fac411-alertmanager-trusted-ca-bundle\") pod \"9496f0bd-bcba-453e-b249-868c02fac411\" (UID: \"9496f0bd-bcba-453e-b249-868c02fac411\") " Jul 14 17:34:51.026650 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.026580 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/9496f0bd-bcba-453e-b249-868c02fac411-alertmanager-main-db\") pod \"9496f0bd-bcba-453e-b249-868c02fac411\" (UID: \"9496f0bd-bcba-453e-b249-868c02fac411\") " Jul 14 17:34:51.026650 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.026648 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9496f0bd-bcba-453e-b249-868c02fac411-web-config\") pod \"9496f0bd-bcba-453e-b249-868c02fac411\" (UID: \"9496f0bd-bcba-453e-b249-868c02fac411\") " Jul 14 17:34:51.027010 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.026979 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9496f0bd-bcba-453e-b249-868c02fac411-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "9496f0bd-bcba-453e-b249-868c02fac411" (UID: "9496f0bd-bcba-453e-b249-868c02fac411"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 17:34:51.027154 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.027127 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9496f0bd-bcba-453e-b249-868c02fac411-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "9496f0bd-bcba-453e-b249-868c02fac411" (UID: "9496f0bd-bcba-453e-b249-868c02fac411"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 17:34:51.027461 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.027399 2562 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9496f0bd-bcba-453e-b249-868c02fac411-metrics-client-ca\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 17:34:51.027461 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.027425 2562 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9496f0bd-bcba-453e-b249-868c02fac411-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 17:34:51.027461 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.027439 2562 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/9496f0bd-bcba-453e-b249-868c02fac411-alertmanager-main-db\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 17:34:51.028005 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.027966 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9496f0bd-bcba-453e-b249-868c02fac411-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "9496f0bd-bcba-453e-b249-868c02fac411" (UID: "9496f0bd-bcba-453e-b249-868c02fac411"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 17:34:51.029028 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.028998 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9496f0bd-bcba-453e-b249-868c02fac411-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "9496f0bd-bcba-453e-b249-868c02fac411" (UID: "9496f0bd-bcba-453e-b249-868c02fac411"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 17:34:51.029456 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.029367 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9496f0bd-bcba-453e-b249-868c02fac411-config-volume" (OuterVolumeSpecName: "config-volume") pod "9496f0bd-bcba-453e-b249-868c02fac411" (UID: "9496f0bd-bcba-453e-b249-868c02fac411"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 17:34:51.029588 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.029555 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9496f0bd-bcba-453e-b249-868c02fac411-config-out" (OuterVolumeSpecName: "config-out") pod "9496f0bd-bcba-453e-b249-868c02fac411" (UID: "9496f0bd-bcba-453e-b249-868c02fac411"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 17:34:51.029688 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.029662 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9496f0bd-bcba-453e-b249-868c02fac411-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "9496f0bd-bcba-453e-b249-868c02fac411" (UID: "9496f0bd-bcba-453e-b249-868c02fac411"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 17:34:51.029988 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.029950 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9496f0bd-bcba-453e-b249-868c02fac411-kube-api-access-pkfjp" (OuterVolumeSpecName: "kube-api-access-pkfjp") pod "9496f0bd-bcba-453e-b249-868c02fac411" (UID: "9496f0bd-bcba-453e-b249-868c02fac411"). InnerVolumeSpecName "kube-api-access-pkfjp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 17:34:51.030730 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.030704 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9496f0bd-bcba-453e-b249-868c02fac411-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "9496f0bd-bcba-453e-b249-868c02fac411" (UID: "9496f0bd-bcba-453e-b249-868c02fac411"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 17:34:51.031005 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.030985 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9496f0bd-bcba-453e-b249-868c02fac411-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "9496f0bd-bcba-453e-b249-868c02fac411" (UID: "9496f0bd-bcba-453e-b249-868c02fac411"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 17:34:51.034036 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.034008 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9496f0bd-bcba-453e-b249-868c02fac411-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "9496f0bd-bcba-453e-b249-868c02fac411" (UID: "9496f0bd-bcba-453e-b249-868c02fac411"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 17:34:51.041546 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.041524 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9496f0bd-bcba-453e-b249-868c02fac411-web-config" (OuterVolumeSpecName: "web-config") pod "9496f0bd-bcba-453e-b249-868c02fac411" (UID: "9496f0bd-bcba-453e-b249-868c02fac411"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 17:34:51.128458 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.128437 2562 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9496f0bd-bcba-453e-b249-868c02fac411-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 17:34:51.128557 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.128461 2562 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9496f0bd-bcba-453e-b249-868c02fac411-config-out\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 17:34:51.128557 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.128472 2562 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/9496f0bd-bcba-453e-b249-868c02fac411-secret-alertmanager-main-tls\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 17:34:51.128557 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.128482 2562 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/9496f0bd-bcba-453e-b249-868c02fac411-config-volume\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 17:34:51.128557 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.128491 2562 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9496f0bd-bcba-453e-b249-868c02fac411-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 17:34:51.128557 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.128499 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pkfjp\" (UniqueName: \"kubernetes.io/projected/9496f0bd-bcba-453e-b249-868c02fac411-kube-api-access-pkfjp\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 17:34:51.128557 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.128508 2562 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/9496f0bd-bcba-453e-b249-868c02fac411-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 17:34:51.128557 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.128517 2562 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9496f0bd-bcba-453e-b249-868c02fac411-tls-assets\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 17:34:51.128557 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.128524 2562 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9496f0bd-bcba-453e-b249-868c02fac411-web-config\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 17:34:51.128557 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.128532 2562 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/9496f0bd-bcba-453e-b249-868c02fac411-cluster-tls-config\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 17:34:51.166418 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.166397 2562 generic.go:356] "Generic (PLEG): container finished" podID="9496f0bd-bcba-453e-b249-868c02fac411" containerID="62b409308e2d72caabdd09589bd74653e7cb03756e5c043c14c88e5a327da2b1" exitCode=0 Jul 14 17:34:51.166418 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.166417 2562 generic.go:356] "Generic (PLEG): container finished" podID="9496f0bd-bcba-453e-b249-868c02fac411" containerID="7394bf0460524ff21abefd0ec0d487e94747d5c33857dd64a8cd161e5562d7b4" exitCode=0 Jul 14 17:34:51.166547 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.166440 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9496f0bd-bcba-453e-b249-868c02fac411","Type":"ContainerDied","Data":"62b409308e2d72caabdd09589bd74653e7cb03756e5c043c14c88e5a327da2b1"} Jul 14 17:34:51.166547 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.166476 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:34:51.166547 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.166498 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9496f0bd-bcba-453e-b249-868c02fac411","Type":"ContainerDied","Data":"7394bf0460524ff21abefd0ec0d487e94747d5c33857dd64a8cd161e5562d7b4"} Jul 14 17:34:51.166547 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.166525 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9496f0bd-bcba-453e-b249-868c02fac411","Type":"ContainerDied","Data":"6278848b1756ff75d7ddd35cb9d883a85d638e9785d01c859f24fbcff741d1c4"} Jul 14 17:34:51.166713 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.166557 2562 scope.go:117] "RemoveContainer" containerID="8987cb03f837b6062e4d3dcc65fe66a5031d9d4f5e3ec0bf8f599f27dc13152d" Jul 14 17:34:51.174520 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.174504 2562 scope.go:117] "RemoveContainer" containerID="62b409308e2d72caabdd09589bd74653e7cb03756e5c043c14c88e5a327da2b1" Jul 14 17:34:51.184004 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.183987 2562 scope.go:117] "RemoveContainer" containerID="ec6b33b846b1bc764e90c66f3e78dae7544b643009009d6a357ac49f378d58d8" Jul 14 17:34:51.190989 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.190964 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 14 17:34:51.191796 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.191779 2562 scope.go:117] "RemoveContainer" containerID="7394bf0460524ff21abefd0ec0d487e94747d5c33857dd64a8cd161e5562d7b4" Jul 14 17:34:51.193913 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.193889 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 14 17:34:51.198649 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.198635 2562 scope.go:117] "RemoveContainer" containerID="8f2a41cd79ab0381c72598412420403d55dee3c1308db49e44d26d37df3e5ba5" Jul 14 17:34:51.205546 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.205519 2562 scope.go:117] "RemoveContainer" containerID="fdf56b3ca4c387e31fda8fe854447d4662fe6bd511a2627b68347b213f9d6f74" Jul 14 17:34:51.212584 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.212569 2562 scope.go:117] "RemoveContainer" containerID="1bd6e783c122326db01313d12d0bc98128bfdc4f28a87f0727315472c77140b1" Jul 14 17:34:51.218744 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.218729 2562 scope.go:117] "RemoveContainer" containerID="8987cb03f837b6062e4d3dcc65fe66a5031d9d4f5e3ec0bf8f599f27dc13152d" Jul 14 17:34:51.218985 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:34:51.218966 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8987cb03f837b6062e4d3dcc65fe66a5031d9d4f5e3ec0bf8f599f27dc13152d\": container with ID starting with 8987cb03f837b6062e4d3dcc65fe66a5031d9d4f5e3ec0bf8f599f27dc13152d not found: ID does not exist" containerID="8987cb03f837b6062e4d3dcc65fe66a5031d9d4f5e3ec0bf8f599f27dc13152d" Jul 14 17:34:51.219027 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.218993 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8987cb03f837b6062e4d3dcc65fe66a5031d9d4f5e3ec0bf8f599f27dc13152d"} err="failed to get container status \"8987cb03f837b6062e4d3dcc65fe66a5031d9d4f5e3ec0bf8f599f27dc13152d\": rpc error: code = NotFound desc = could not find container \"8987cb03f837b6062e4d3dcc65fe66a5031d9d4f5e3ec0bf8f599f27dc13152d\": container with ID starting with 8987cb03f837b6062e4d3dcc65fe66a5031d9d4f5e3ec0bf8f599f27dc13152d not found: ID does not exist" Jul 14 17:34:51.219027 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.219011 2562 scope.go:117] "RemoveContainer" containerID="62b409308e2d72caabdd09589bd74653e7cb03756e5c043c14c88e5a327da2b1" Jul 14 17:34:51.219330 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:34:51.219311 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"62b409308e2d72caabdd09589bd74653e7cb03756e5c043c14c88e5a327da2b1\": container with ID starting with 62b409308e2d72caabdd09589bd74653e7cb03756e5c043c14c88e5a327da2b1 not found: ID does not exist" containerID="62b409308e2d72caabdd09589bd74653e7cb03756e5c043c14c88e5a327da2b1" Jul 14 17:34:51.219404 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.219336 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"62b409308e2d72caabdd09589bd74653e7cb03756e5c043c14c88e5a327da2b1"} err="failed to get container status \"62b409308e2d72caabdd09589bd74653e7cb03756e5c043c14c88e5a327da2b1\": rpc error: code = NotFound desc = could not find container \"62b409308e2d72caabdd09589bd74653e7cb03756e5c043c14c88e5a327da2b1\": container with ID starting with 62b409308e2d72caabdd09589bd74653e7cb03756e5c043c14c88e5a327da2b1 not found: ID does not exist" Jul 14 17:34:51.219404 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.219352 2562 scope.go:117] "RemoveContainer" containerID="ec6b33b846b1bc764e90c66f3e78dae7544b643009009d6a357ac49f378d58d8" Jul 14 17:34:51.219613 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:34:51.219582 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ec6b33b846b1bc764e90c66f3e78dae7544b643009009d6a357ac49f378d58d8\": container with ID starting with ec6b33b846b1bc764e90c66f3e78dae7544b643009009d6a357ac49f378d58d8 not found: ID does not exist" containerID="ec6b33b846b1bc764e90c66f3e78dae7544b643009009d6a357ac49f378d58d8" Jul 14 17:34:51.219687 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.219608 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec6b33b846b1bc764e90c66f3e78dae7544b643009009d6a357ac49f378d58d8"} err="failed to get container status \"ec6b33b846b1bc764e90c66f3e78dae7544b643009009d6a357ac49f378d58d8\": rpc error: code = NotFound desc = could not find container \"ec6b33b846b1bc764e90c66f3e78dae7544b643009009d6a357ac49f378d58d8\": container with ID starting with ec6b33b846b1bc764e90c66f3e78dae7544b643009009d6a357ac49f378d58d8 not found: ID does not exist" Jul 14 17:34:51.219687 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.219632 2562 scope.go:117] "RemoveContainer" containerID="7394bf0460524ff21abefd0ec0d487e94747d5c33857dd64a8cd161e5562d7b4" Jul 14 17:34:51.220123 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:34:51.219981 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7394bf0460524ff21abefd0ec0d487e94747d5c33857dd64a8cd161e5562d7b4\": container with ID starting with 7394bf0460524ff21abefd0ec0d487e94747d5c33857dd64a8cd161e5562d7b4 not found: ID does not exist" containerID="7394bf0460524ff21abefd0ec0d487e94747d5c33857dd64a8cd161e5562d7b4" Jul 14 17:34:51.220123 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.220013 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7394bf0460524ff21abefd0ec0d487e94747d5c33857dd64a8cd161e5562d7b4"} err="failed to get container status \"7394bf0460524ff21abefd0ec0d487e94747d5c33857dd64a8cd161e5562d7b4\": rpc error: code = NotFound desc = could not find container \"7394bf0460524ff21abefd0ec0d487e94747d5c33857dd64a8cd161e5562d7b4\": container with ID starting with 7394bf0460524ff21abefd0ec0d487e94747d5c33857dd64a8cd161e5562d7b4 not found: ID does not exist" Jul 14 17:34:51.220123 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.220034 2562 scope.go:117] "RemoveContainer" containerID="8f2a41cd79ab0381c72598412420403d55dee3c1308db49e44d26d37df3e5ba5" Jul 14 17:34:51.220438 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:34:51.220422 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8f2a41cd79ab0381c72598412420403d55dee3c1308db49e44d26d37df3e5ba5\": container with ID starting with 8f2a41cd79ab0381c72598412420403d55dee3c1308db49e44d26d37df3e5ba5 not found: ID does not exist" containerID="8f2a41cd79ab0381c72598412420403d55dee3c1308db49e44d26d37df3e5ba5" Jul 14 17:34:51.220479 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.220443 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f2a41cd79ab0381c72598412420403d55dee3c1308db49e44d26d37df3e5ba5"} err="failed to get container status \"8f2a41cd79ab0381c72598412420403d55dee3c1308db49e44d26d37df3e5ba5\": rpc error: code = NotFound desc = could not find container \"8f2a41cd79ab0381c72598412420403d55dee3c1308db49e44d26d37df3e5ba5\": container with ID starting with 8f2a41cd79ab0381c72598412420403d55dee3c1308db49e44d26d37df3e5ba5 not found: ID does not exist" Jul 14 17:34:51.220479 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.220457 2562 scope.go:117] "RemoveContainer" containerID="fdf56b3ca4c387e31fda8fe854447d4662fe6bd511a2627b68347b213f9d6f74" Jul 14 17:34:51.220689 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:34:51.220673 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fdf56b3ca4c387e31fda8fe854447d4662fe6bd511a2627b68347b213f9d6f74\": container with ID starting with fdf56b3ca4c387e31fda8fe854447d4662fe6bd511a2627b68347b213f9d6f74 not found: ID does not exist" containerID="fdf56b3ca4c387e31fda8fe854447d4662fe6bd511a2627b68347b213f9d6f74" Jul 14 17:34:51.220727 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.220695 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fdf56b3ca4c387e31fda8fe854447d4662fe6bd511a2627b68347b213f9d6f74"} err="failed to get container status \"fdf56b3ca4c387e31fda8fe854447d4662fe6bd511a2627b68347b213f9d6f74\": rpc error: code = NotFound desc = could not find container \"fdf56b3ca4c387e31fda8fe854447d4662fe6bd511a2627b68347b213f9d6f74\": container with ID starting with fdf56b3ca4c387e31fda8fe854447d4662fe6bd511a2627b68347b213f9d6f74 not found: ID does not exist" Jul 14 17:34:51.220727 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.220710 2562 scope.go:117] "RemoveContainer" containerID="1bd6e783c122326db01313d12d0bc98128bfdc4f28a87f0727315472c77140b1" Jul 14 17:34:51.220934 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:34:51.220918 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1bd6e783c122326db01313d12d0bc98128bfdc4f28a87f0727315472c77140b1\": container with ID starting with 1bd6e783c122326db01313d12d0bc98128bfdc4f28a87f0727315472c77140b1 not found: ID does not exist" containerID="1bd6e783c122326db01313d12d0bc98128bfdc4f28a87f0727315472c77140b1" Jul 14 17:34:51.220982 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.220937 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1bd6e783c122326db01313d12d0bc98128bfdc4f28a87f0727315472c77140b1"} err="failed to get container status \"1bd6e783c122326db01313d12d0bc98128bfdc4f28a87f0727315472c77140b1\": rpc error: code = NotFound desc = could not find container \"1bd6e783c122326db01313d12d0bc98128bfdc4f28a87f0727315472c77140b1\": container with ID starting with 1bd6e783c122326db01313d12d0bc98128bfdc4f28a87f0727315472c77140b1 not found: ID does not exist" Jul 14 17:34:51.220982 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.220950 2562 scope.go:117] "RemoveContainer" containerID="8987cb03f837b6062e4d3dcc65fe66a5031d9d4f5e3ec0bf8f599f27dc13152d" Jul 14 17:34:51.221133 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.221113 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8987cb03f837b6062e4d3dcc65fe66a5031d9d4f5e3ec0bf8f599f27dc13152d"} err="failed to get container status \"8987cb03f837b6062e4d3dcc65fe66a5031d9d4f5e3ec0bf8f599f27dc13152d\": rpc error: code = NotFound desc = could not find container \"8987cb03f837b6062e4d3dcc65fe66a5031d9d4f5e3ec0bf8f599f27dc13152d\": container with ID starting with 8987cb03f837b6062e4d3dcc65fe66a5031d9d4f5e3ec0bf8f599f27dc13152d not found: ID does not exist" Jul 14 17:34:51.221176 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.221134 2562 scope.go:117] "RemoveContainer" containerID="62b409308e2d72caabdd09589bd74653e7cb03756e5c043c14c88e5a327da2b1" Jul 14 17:34:51.221449 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.221434 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"62b409308e2d72caabdd09589bd74653e7cb03756e5c043c14c88e5a327da2b1"} err="failed to get container status \"62b409308e2d72caabdd09589bd74653e7cb03756e5c043c14c88e5a327da2b1\": rpc error: code = NotFound desc = could not find container \"62b409308e2d72caabdd09589bd74653e7cb03756e5c043c14c88e5a327da2b1\": container with ID starting with 62b409308e2d72caabdd09589bd74653e7cb03756e5c043c14c88e5a327da2b1 not found: ID does not exist" Jul 14 17:34:51.221505 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.221450 2562 scope.go:117] "RemoveContainer" containerID="ec6b33b846b1bc764e90c66f3e78dae7544b643009009d6a357ac49f378d58d8" Jul 14 17:34:51.221668 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.221652 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec6b33b846b1bc764e90c66f3e78dae7544b643009009d6a357ac49f378d58d8"} err="failed to get container status \"ec6b33b846b1bc764e90c66f3e78dae7544b643009009d6a357ac49f378d58d8\": rpc error: code = NotFound desc = could not find container \"ec6b33b846b1bc764e90c66f3e78dae7544b643009009d6a357ac49f378d58d8\": container with ID starting with ec6b33b846b1bc764e90c66f3e78dae7544b643009009d6a357ac49f378d58d8 not found: ID does not exist" Jul 14 17:34:51.221727 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.221669 2562 scope.go:117] "RemoveContainer" containerID="7394bf0460524ff21abefd0ec0d487e94747d5c33857dd64a8cd161e5562d7b4" Jul 14 17:34:51.221901 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.221886 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7394bf0460524ff21abefd0ec0d487e94747d5c33857dd64a8cd161e5562d7b4"} err="failed to get container status \"7394bf0460524ff21abefd0ec0d487e94747d5c33857dd64a8cd161e5562d7b4\": rpc error: code = NotFound desc = could not find container \"7394bf0460524ff21abefd0ec0d487e94747d5c33857dd64a8cd161e5562d7b4\": container with ID starting with 7394bf0460524ff21abefd0ec0d487e94747d5c33857dd64a8cd161e5562d7b4 not found: ID does not exist" Jul 14 17:34:51.221964 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.221901 2562 scope.go:117] "RemoveContainer" containerID="8f2a41cd79ab0381c72598412420403d55dee3c1308db49e44d26d37df3e5ba5" Jul 14 17:34:51.222175 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.222154 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f2a41cd79ab0381c72598412420403d55dee3c1308db49e44d26d37df3e5ba5"} err="failed to get container status \"8f2a41cd79ab0381c72598412420403d55dee3c1308db49e44d26d37df3e5ba5\": rpc error: code = NotFound desc = could not find container \"8f2a41cd79ab0381c72598412420403d55dee3c1308db49e44d26d37df3e5ba5\": container with ID starting with 8f2a41cd79ab0381c72598412420403d55dee3c1308db49e44d26d37df3e5ba5 not found: ID does not exist" Jul 14 17:34:51.222252 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.222198 2562 scope.go:117] "RemoveContainer" containerID="fdf56b3ca4c387e31fda8fe854447d4662fe6bd511a2627b68347b213f9d6f74" Jul 14 17:34:51.222420 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.222402 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fdf56b3ca4c387e31fda8fe854447d4662fe6bd511a2627b68347b213f9d6f74"} err="failed to get container status \"fdf56b3ca4c387e31fda8fe854447d4662fe6bd511a2627b68347b213f9d6f74\": rpc error: code = NotFound desc = could not find container \"fdf56b3ca4c387e31fda8fe854447d4662fe6bd511a2627b68347b213f9d6f74\": container with ID starting with fdf56b3ca4c387e31fda8fe854447d4662fe6bd511a2627b68347b213f9d6f74 not found: ID does not exist" Jul 14 17:34:51.222457 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.222421 2562 scope.go:117] "RemoveContainer" containerID="1bd6e783c122326db01313d12d0bc98128bfdc4f28a87f0727315472c77140b1" Jul 14 17:34:51.222611 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.222596 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1bd6e783c122326db01313d12d0bc98128bfdc4f28a87f0727315472c77140b1"} err="failed to get container status \"1bd6e783c122326db01313d12d0bc98128bfdc4f28a87f0727315472c77140b1\": rpc error: code = NotFound desc = could not find container \"1bd6e783c122326db01313d12d0bc98128bfdc4f28a87f0727315472c77140b1\": container with ID starting with 1bd6e783c122326db01313d12d0bc98128bfdc4f28a87f0727315472c77140b1 not found: ID does not exist" Jul 14 17:34:51.232045 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.232024 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 14 17:34:51.232298 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.232287 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="67e1d577-9cfd-475f-b758-4ef21fac1a15" containerName="console" Jul 14 17:34:51.232337 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.232300 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="67e1d577-9cfd-475f-b758-4ef21fac1a15" containerName="console" Jul 14 17:34:51.232337 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.232313 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9496f0bd-bcba-453e-b249-868c02fac411" containerName="alertmanager" Jul 14 17:34:51.232337 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.232319 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="9496f0bd-bcba-453e-b249-868c02fac411" containerName="alertmanager" Jul 14 17:34:51.232337 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.232326 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9496f0bd-bcba-453e-b249-868c02fac411" containerName="prom-label-proxy" Jul 14 17:34:51.232337 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.232331 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="9496f0bd-bcba-453e-b249-868c02fac411" containerName="prom-label-proxy" Jul 14 17:34:51.232337 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.232338 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9496f0bd-bcba-453e-b249-868c02fac411" containerName="init-config-reloader" Jul 14 17:34:51.232506 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.232343 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="9496f0bd-bcba-453e-b249-868c02fac411" containerName="init-config-reloader" Jul 14 17:34:51.232506 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.232349 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9496f0bd-bcba-453e-b249-868c02fac411" containerName="config-reloader" Jul 14 17:34:51.232506 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.232355 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="9496f0bd-bcba-453e-b249-868c02fac411" containerName="config-reloader" Jul 14 17:34:51.232506 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.232361 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c51ee65-7247-4790-885c-304ae178695c" containerName="console" Jul 14 17:34:51.232506 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.232366 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c51ee65-7247-4790-885c-304ae178695c" containerName="console" Jul 14 17:34:51.232506 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.232373 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b56ef57c-4634-40df-a68d-e4ae43d07c3d" containerName="console" Jul 14 17:34:51.232506 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.232378 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="b56ef57c-4634-40df-a68d-e4ae43d07c3d" containerName="console" Jul 14 17:34:51.232506 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.232385 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9496f0bd-bcba-453e-b249-868c02fac411" containerName="kube-rbac-proxy-web" Jul 14 17:34:51.232506 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.232390 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="9496f0bd-bcba-453e-b249-868c02fac411" containerName="kube-rbac-proxy-web" Jul 14 17:34:51.232506 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.232399 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9496f0bd-bcba-453e-b249-868c02fac411" containerName="kube-rbac-proxy" Jul 14 17:34:51.232506 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.232404 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="9496f0bd-bcba-453e-b249-868c02fac411" containerName="kube-rbac-proxy" Jul 14 17:34:51.232506 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.232412 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9496f0bd-bcba-453e-b249-868c02fac411" containerName="kube-rbac-proxy-metric" Jul 14 17:34:51.232506 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.232416 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="9496f0bd-bcba-453e-b249-868c02fac411" containerName="kube-rbac-proxy-metric" Jul 14 17:34:51.232506 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.232454 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="67e1d577-9cfd-475f-b758-4ef21fac1a15" containerName="console" Jul 14 17:34:51.232506 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.232461 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="9496f0bd-bcba-453e-b249-868c02fac411" containerName="kube-rbac-proxy" Jul 14 17:34:51.232506 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.232466 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="b56ef57c-4634-40df-a68d-e4ae43d07c3d" containerName="console" Jul 14 17:34:51.232506 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.232472 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="9496f0bd-bcba-453e-b249-868c02fac411" containerName="config-reloader" Jul 14 17:34:51.232506 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.232478 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="9496f0bd-bcba-453e-b249-868c02fac411" containerName="prom-label-proxy" Jul 14 17:34:51.232506 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.232484 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="9496f0bd-bcba-453e-b249-868c02fac411" containerName="alertmanager" Jul 14 17:34:51.232506 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.232489 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="9496f0bd-bcba-453e-b249-868c02fac411" containerName="kube-rbac-proxy-web" Jul 14 17:34:51.232506 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.232494 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="1c51ee65-7247-4790-885c-304ae178695c" containerName="console" Jul 14 17:34:51.232506 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.232499 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="9496f0bd-bcba-453e-b249-868c02fac411" containerName="kube-rbac-proxy-metric" Jul 14 17:34:51.236304 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.236289 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:34:51.238693 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.238666 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 14 17:34:51.238760 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.238707 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 14 17:34:51.238950 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.238935 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 14 17:34:51.239063 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.239047 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 14 17:34:51.239438 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.239418 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-bg757\"" Jul 14 17:34:51.239532 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.239484 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 14 17:34:51.239532 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.239493 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 14 17:34:51.239883 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.239864 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 14 17:34:51.240041 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.240023 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 14 17:34:51.244806 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.244786 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 14 17:34:51.279720 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.269625 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 14 17:34:51.329733 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.329701 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6f004740-868c-4ad2-9c9c-bf0d903bfa87-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"6f004740-868c-4ad2-9c9c-bf0d903bfa87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:34:51.329877 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.329738 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6f004740-868c-4ad2-9c9c-bf0d903bfa87-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"6f004740-868c-4ad2-9c9c-bf0d903bfa87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:34:51.329877 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.329757 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/6f004740-868c-4ad2-9c9c-bf0d903bfa87-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"6f004740-868c-4ad2-9c9c-bf0d903bfa87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:34:51.329877 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.329788 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rgnjx\" (UniqueName: \"kubernetes.io/projected/6f004740-868c-4ad2-9c9c-bf0d903bfa87-kube-api-access-rgnjx\") pod \"alertmanager-main-0\" (UID: \"6f004740-868c-4ad2-9c9c-bf0d903bfa87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:34:51.329877 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.329838 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6f004740-868c-4ad2-9c9c-bf0d903bfa87-config-out\") pod \"alertmanager-main-0\" (UID: \"6f004740-868c-4ad2-9c9c-bf0d903bfa87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:34:51.329877 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.329869 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6f004740-868c-4ad2-9c9c-bf0d903bfa87-tls-assets\") pod \"alertmanager-main-0\" (UID: \"6f004740-868c-4ad2-9c9c-bf0d903bfa87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:34:51.330046 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.329891 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6f004740-868c-4ad2-9c9c-bf0d903bfa87-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"6f004740-868c-4ad2-9c9c-bf0d903bfa87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:34:51.330046 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.329920 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6f004740-868c-4ad2-9c9c-bf0d903bfa87-web-config\") pod \"alertmanager-main-0\" (UID: \"6f004740-868c-4ad2-9c9c-bf0d903bfa87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:34:51.330046 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.329970 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/6f004740-868c-4ad2-9c9c-bf0d903bfa87-config-volume\") pod \"alertmanager-main-0\" (UID: \"6f004740-868c-4ad2-9c9c-bf0d903bfa87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:34:51.330046 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.330001 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6f004740-868c-4ad2-9c9c-bf0d903bfa87-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"6f004740-868c-4ad2-9c9c-bf0d903bfa87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:34:51.330046 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.330023 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/6f004740-868c-4ad2-9c9c-bf0d903bfa87-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"6f004740-868c-4ad2-9c9c-bf0d903bfa87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:34:51.330046 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.330042 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/6f004740-868c-4ad2-9c9c-bf0d903bfa87-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"6f004740-868c-4ad2-9c9c-bf0d903bfa87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:34:51.330259 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.330065 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/6f004740-868c-4ad2-9c9c-bf0d903bfa87-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"6f004740-868c-4ad2-9c9c-bf0d903bfa87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:34:51.431370 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.431302 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/6f004740-868c-4ad2-9c9c-bf0d903bfa87-config-volume\") pod \"alertmanager-main-0\" (UID: \"6f004740-868c-4ad2-9c9c-bf0d903bfa87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:34:51.431370 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.431334 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6f004740-868c-4ad2-9c9c-bf0d903bfa87-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"6f004740-868c-4ad2-9c9c-bf0d903bfa87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:34:51.431370 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.431355 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/6f004740-868c-4ad2-9c9c-bf0d903bfa87-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"6f004740-868c-4ad2-9c9c-bf0d903bfa87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:34:51.431618 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.431374 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/6f004740-868c-4ad2-9c9c-bf0d903bfa87-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"6f004740-868c-4ad2-9c9c-bf0d903bfa87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:34:51.431618 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.431397 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/6f004740-868c-4ad2-9c9c-bf0d903bfa87-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"6f004740-868c-4ad2-9c9c-bf0d903bfa87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:34:51.431618 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.431518 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6f004740-868c-4ad2-9c9c-bf0d903bfa87-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"6f004740-868c-4ad2-9c9c-bf0d903bfa87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:34:51.431618 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.431554 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6f004740-868c-4ad2-9c9c-bf0d903bfa87-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"6f004740-868c-4ad2-9c9c-bf0d903bfa87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:34:51.431618 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.431577 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/6f004740-868c-4ad2-9c9c-bf0d903bfa87-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"6f004740-868c-4ad2-9c9c-bf0d903bfa87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:34:51.431859 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.431635 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rgnjx\" (UniqueName: \"kubernetes.io/projected/6f004740-868c-4ad2-9c9c-bf0d903bfa87-kube-api-access-rgnjx\") pod \"alertmanager-main-0\" (UID: \"6f004740-868c-4ad2-9c9c-bf0d903bfa87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:34:51.431859 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.431680 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6f004740-868c-4ad2-9c9c-bf0d903bfa87-config-out\") pod \"alertmanager-main-0\" (UID: \"6f004740-868c-4ad2-9c9c-bf0d903bfa87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:34:51.431859 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.431717 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6f004740-868c-4ad2-9c9c-bf0d903bfa87-tls-assets\") pod \"alertmanager-main-0\" (UID: \"6f004740-868c-4ad2-9c9c-bf0d903bfa87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:34:51.431859 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.431750 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6f004740-868c-4ad2-9c9c-bf0d903bfa87-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"6f004740-868c-4ad2-9c9c-bf0d903bfa87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:34:51.431859 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.431785 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6f004740-868c-4ad2-9c9c-bf0d903bfa87-web-config\") pod \"alertmanager-main-0\" (UID: \"6f004740-868c-4ad2-9c9c-bf0d903bfa87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:34:51.431859 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.431802 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/6f004740-868c-4ad2-9c9c-bf0d903bfa87-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"6f004740-868c-4ad2-9c9c-bf0d903bfa87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:34:51.433014 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.432989 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6f004740-868c-4ad2-9c9c-bf0d903bfa87-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"6f004740-868c-4ad2-9c9c-bf0d903bfa87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:34:51.433804 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.433781 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6f004740-868c-4ad2-9c9c-bf0d903bfa87-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"6f004740-868c-4ad2-9c9c-bf0d903bfa87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:34:51.434724 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.434620 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/6f004740-868c-4ad2-9c9c-bf0d903bfa87-config-volume\") pod \"alertmanager-main-0\" (UID: \"6f004740-868c-4ad2-9c9c-bf0d903bfa87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:34:51.434724 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.434646 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/6f004740-868c-4ad2-9c9c-bf0d903bfa87-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"6f004740-868c-4ad2-9c9c-bf0d903bfa87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:34:51.434724 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.434680 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/6f004740-868c-4ad2-9c9c-bf0d903bfa87-config-out\") pod \"alertmanager-main-0\" (UID: \"6f004740-868c-4ad2-9c9c-bf0d903bfa87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:34:51.434864 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.434730 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/6f004740-868c-4ad2-9c9c-bf0d903bfa87-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"6f004740-868c-4ad2-9c9c-bf0d903bfa87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:34:51.434864 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.434730 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/6f004740-868c-4ad2-9c9c-bf0d903bfa87-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"6f004740-868c-4ad2-9c9c-bf0d903bfa87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:34:51.434864 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.434798 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/6f004740-868c-4ad2-9c9c-bf0d903bfa87-tls-assets\") pod \"alertmanager-main-0\" (UID: \"6f004740-868c-4ad2-9c9c-bf0d903bfa87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:34:51.435085 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.435067 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/6f004740-868c-4ad2-9c9c-bf0d903bfa87-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"6f004740-868c-4ad2-9c9c-bf0d903bfa87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:34:51.436419 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.436402 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/6f004740-868c-4ad2-9c9c-bf0d903bfa87-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"6f004740-868c-4ad2-9c9c-bf0d903bfa87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:34:51.436553 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.436538 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/6f004740-868c-4ad2-9c9c-bf0d903bfa87-web-config\") pod \"alertmanager-main-0\" (UID: \"6f004740-868c-4ad2-9c9c-bf0d903bfa87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:34:51.447737 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.447721 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rgnjx\" (UniqueName: \"kubernetes.io/projected/6f004740-868c-4ad2-9c9c-bf0d903bfa87-kube-api-access-rgnjx\") pod \"alertmanager-main-0\" (UID: \"6f004740-868c-4ad2-9c9c-bf0d903bfa87\") " pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:34:51.517926 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.517903 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9496f0bd-bcba-453e-b249-868c02fac411" path="/var/lib/kubelet/pods/9496f0bd-bcba-453e-b249-868c02fac411/volumes" Jul 14 17:34:51.546770 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.546750 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 14 17:34:51.686067 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:51.685968 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 14 17:34:51.688542 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:34:51.688514 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f004740_868c_4ad2_9c9c_bf0d903bfa87.slice/crio-6a11ab433046fc35a19fa6889768c0b68f324c5a9bc19a67a2a5a7fff85aa161 WatchSource:0}: Error finding container 6a11ab433046fc35a19fa6889768c0b68f324c5a9bc19a67a2a5a7fff85aa161: Status 404 returned error can't find the container with id 6a11ab433046fc35a19fa6889768c0b68f324c5a9bc19a67a2a5a7fff85aa161 Jul 14 17:34:52.171079 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:52.171040 2562 generic.go:356] "Generic (PLEG): container finished" podID="6f004740-868c-4ad2-9c9c-bf0d903bfa87" containerID="d1c17728b369a07e61ab4e5dca9df11faf4e3865c1f208ffaba204d071a34277" exitCode=0 Jul 14 17:34:52.171513 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:52.171131 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6f004740-868c-4ad2-9c9c-bf0d903bfa87","Type":"ContainerDied","Data":"d1c17728b369a07e61ab4e5dca9df11faf4e3865c1f208ffaba204d071a34277"} Jul 14 17:34:52.171513 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:52.171252 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6f004740-868c-4ad2-9c9c-bf0d903bfa87","Type":"ContainerStarted","Data":"6a11ab433046fc35a19fa6889768c0b68f324c5a9bc19a67a2a5a7fff85aa161"} Jul 14 17:34:53.179375 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:53.179343 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6f004740-868c-4ad2-9c9c-bf0d903bfa87","Type":"ContainerStarted","Data":"f5b8bc0fb39554c41340b5732b0544cd3cfd6196c2a5044b743dc1c58b132484"} Jul 14 17:34:53.179730 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:53.179405 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6f004740-868c-4ad2-9c9c-bf0d903bfa87","Type":"ContainerStarted","Data":"0e69f76ea5596c04b2fbcf8c4dfa3e24271ab3c417b4fcf817867d2b6d1bcb02"} Jul 14 17:34:53.179730 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:53.179431 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6f004740-868c-4ad2-9c9c-bf0d903bfa87","Type":"ContainerStarted","Data":"1c8b56aeb891042d632e5b17c1867162b1e42ef3c27da7751684ced590fccf19"} Jul 14 17:34:53.179730 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:53.179455 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6f004740-868c-4ad2-9c9c-bf0d903bfa87","Type":"ContainerStarted","Data":"685c3eb9b9025f31e3fc49758878e4ddb0291561c32cf24623f45d0417440adb"} Jul 14 17:34:53.179730 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:53.179478 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6f004740-868c-4ad2-9c9c-bf0d903bfa87","Type":"ContainerStarted","Data":"349ac69baef2b19fbb73c2ed2fc98d29793af8e0a62ae90536a1dcda962fcfcb"} Jul 14 17:34:53.179730 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:53.179505 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"6f004740-868c-4ad2-9c9c-bf0d903bfa87","Type":"ContainerStarted","Data":"6177c7342692ce1826797129921f67e8f1dd41820ebd2b0bb74ad19f91374b24"} Jul 14 17:34:53.220600 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:53.220548 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.205500678 podStartE2EDuration="2.220529326s" totalImagesPullingTime="0s" totalInitContainerRuntime="15.028648ms" isStatefulPod=false podCreationTimestamp="2026-07-14 17:34:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:34:53.215267214 +0000 UTC m=+142.291724753" watchObservedRunningTime="2026-07-14 17:34:53.220529326 +0000 UTC m=+142.296986867" Jul 14 17:34:53.976246 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:53.976209 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 14 17:34:53.976633 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:53.976530 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="162964a8-ba20-4887-b8b9-e98369835a0d" containerName="prometheus" containerID="cri-o://14eddb98ff4482142de2825a71c0306903354b4c989dba6b63e46f9981a1eb93" gracePeriod=600 Jul 14 17:34:53.976633 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:53.976560 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="162964a8-ba20-4887-b8b9-e98369835a0d" containerName="kube-rbac-proxy-thanos" containerID="cri-o://210019baefe3a41037b37b54a22651e239fb4c085cb7af1e3d44667fa347af67" gracePeriod=600 Jul 14 17:34:53.976633 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:53.976608 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="162964a8-ba20-4887-b8b9-e98369835a0d" containerName="thanos-sidecar" containerID="cri-o://303a1cededc32c93c1f7a013e72ede71495ef80842bd6ee6ddada0bc1f2c5b37" gracePeriod=600 Jul 14 17:34:53.976851 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:53.976618 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="162964a8-ba20-4887-b8b9-e98369835a0d" containerName="config-reloader" containerID="cri-o://f3f5cacddd0c2a27cdfc51d6e2022c88ec5b13d7d475353933eac7ffdee5a9ae" gracePeriod=600 Jul 14 17:34:53.976851 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:53.976572 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="162964a8-ba20-4887-b8b9-e98369835a0d" containerName="kube-rbac-proxy" containerID="cri-o://808f856eed06c1568c0bd1a7973a20d1113f8d7dee82f513131fff832f1ba39a" gracePeriod=600 Jul 14 17:34:53.976851 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:53.976568 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="162964a8-ba20-4887-b8b9-e98369835a0d" containerName="kube-rbac-proxy-web" containerID="cri-o://82df51680a9ebde69f9e3d98674a97daa5d013af7d99f3880fa05209ce71ed39" gracePeriod=600 Jul 14 17:34:54.188249 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:54.188218 2562 generic.go:356] "Generic (PLEG): container finished" podID="162964a8-ba20-4887-b8b9-e98369835a0d" containerID="210019baefe3a41037b37b54a22651e239fb4c085cb7af1e3d44667fa347af67" exitCode=0 Jul 14 17:34:54.188249 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:54.188245 2562 generic.go:356] "Generic (PLEG): container finished" podID="162964a8-ba20-4887-b8b9-e98369835a0d" containerID="808f856eed06c1568c0bd1a7973a20d1113f8d7dee82f513131fff832f1ba39a" exitCode=0 Jul 14 17:34:54.188249 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:54.188254 2562 generic.go:356] "Generic (PLEG): container finished" podID="162964a8-ba20-4887-b8b9-e98369835a0d" containerID="82df51680a9ebde69f9e3d98674a97daa5d013af7d99f3880fa05209ce71ed39" exitCode=0 Jul 14 17:34:54.188626 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:54.188262 2562 generic.go:356] "Generic (PLEG): container finished" podID="162964a8-ba20-4887-b8b9-e98369835a0d" containerID="303a1cededc32c93c1f7a013e72ede71495ef80842bd6ee6ddada0bc1f2c5b37" exitCode=0 Jul 14 17:34:54.188626 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:54.188270 2562 generic.go:356] "Generic (PLEG): container finished" podID="162964a8-ba20-4887-b8b9-e98369835a0d" containerID="f3f5cacddd0c2a27cdfc51d6e2022c88ec5b13d7d475353933eac7ffdee5a9ae" exitCode=0 Jul 14 17:34:54.188626 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:54.188278 2562 generic.go:356] "Generic (PLEG): container finished" podID="162964a8-ba20-4887-b8b9-e98369835a0d" containerID="14eddb98ff4482142de2825a71c0306903354b4c989dba6b63e46f9981a1eb93" exitCode=0 Jul 14 17:34:54.188626 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:54.188275 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"162964a8-ba20-4887-b8b9-e98369835a0d","Type":"ContainerDied","Data":"210019baefe3a41037b37b54a22651e239fb4c085cb7af1e3d44667fa347af67"} Jul 14 17:34:54.188626 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:54.188339 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"162964a8-ba20-4887-b8b9-e98369835a0d","Type":"ContainerDied","Data":"808f856eed06c1568c0bd1a7973a20d1113f8d7dee82f513131fff832f1ba39a"} Jul 14 17:34:54.188626 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:54.188385 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"162964a8-ba20-4887-b8b9-e98369835a0d","Type":"ContainerDied","Data":"82df51680a9ebde69f9e3d98674a97daa5d013af7d99f3880fa05209ce71ed39"} Jul 14 17:34:54.188626 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:54.188442 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"162964a8-ba20-4887-b8b9-e98369835a0d","Type":"ContainerDied","Data":"303a1cededc32c93c1f7a013e72ede71495ef80842bd6ee6ddada0bc1f2c5b37"} Jul 14 17:34:54.188626 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:54.188486 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"162964a8-ba20-4887-b8b9-e98369835a0d","Type":"ContainerDied","Data":"f3f5cacddd0c2a27cdfc51d6e2022c88ec5b13d7d475353933eac7ffdee5a9ae"} Jul 14 17:34:54.188626 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:54.188524 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"162964a8-ba20-4887-b8b9-e98369835a0d","Type":"ContainerDied","Data":"14eddb98ff4482142de2825a71c0306903354b4c989dba6b63e46f9981a1eb93"} Jul 14 17:34:54.214385 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:54.214365 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:34:54.254183 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:54.254054 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/162964a8-ba20-4887-b8b9-e98369835a0d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"162964a8-ba20-4887-b8b9-e98369835a0d\" (UID: \"162964a8-ba20-4887-b8b9-e98369835a0d\") " Jul 14 17:34:54.254183 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:54.254097 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/162964a8-ba20-4887-b8b9-e98369835a0d-secret-grpc-tls\") pod \"162964a8-ba20-4887-b8b9-e98369835a0d\" (UID: \"162964a8-ba20-4887-b8b9-e98369835a0d\") " Jul 14 17:34:54.254183 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:54.254131 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/162964a8-ba20-4887-b8b9-e98369835a0d-configmap-serving-certs-ca-bundle\") pod \"162964a8-ba20-4887-b8b9-e98369835a0d\" (UID: \"162964a8-ba20-4887-b8b9-e98369835a0d\") " Jul 14 17:34:54.254183 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:54.254159 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/162964a8-ba20-4887-b8b9-e98369835a0d-tls-assets\") pod \"162964a8-ba20-4887-b8b9-e98369835a0d\" (UID: \"162964a8-ba20-4887-b8b9-e98369835a0d\") " Jul 14 17:34:54.254491 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:54.254214 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/162964a8-ba20-4887-b8b9-e98369835a0d-thanos-prometheus-http-client-file\") pod \"162964a8-ba20-4887-b8b9-e98369835a0d\" (UID: \"162964a8-ba20-4887-b8b9-e98369835a0d\") " Jul 14 17:34:54.254491 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:54.254240 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/162964a8-ba20-4887-b8b9-e98369835a0d-prometheus-k8s-rulefiles-2\") pod \"162964a8-ba20-4887-b8b9-e98369835a0d\" (UID: \"162964a8-ba20-4887-b8b9-e98369835a0d\") " Jul 14 17:34:54.254491 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:54.254263 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/162964a8-ba20-4887-b8b9-e98369835a0d-web-config\") pod \"162964a8-ba20-4887-b8b9-e98369835a0d\" (UID: \"162964a8-ba20-4887-b8b9-e98369835a0d\") " Jul 14 17:34:54.254491 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:54.254284 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/162964a8-ba20-4887-b8b9-e98369835a0d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"162964a8-ba20-4887-b8b9-e98369835a0d\" (UID: \"162964a8-ba20-4887-b8b9-e98369835a0d\") " Jul 14 17:34:54.254491 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:54.254309 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/162964a8-ba20-4887-b8b9-e98369835a0d-prometheus-k8s-rulefiles-0\") pod \"162964a8-ba20-4887-b8b9-e98369835a0d\" (UID: \"162964a8-ba20-4887-b8b9-e98369835a0d\") " Jul 14 17:34:54.254491 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:54.254329 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/162964a8-ba20-4887-b8b9-e98369835a0d-config\") pod \"162964a8-ba20-4887-b8b9-e98369835a0d\" (UID: \"162964a8-ba20-4887-b8b9-e98369835a0d\") " Jul 14 17:34:54.254491 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:54.254343 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/162964a8-ba20-4887-b8b9-e98369835a0d-secret-prometheus-k8s-tls\") pod \"162964a8-ba20-4887-b8b9-e98369835a0d\" (UID: \"162964a8-ba20-4887-b8b9-e98369835a0d\") " Jul 14 17:34:54.254491 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:54.254378 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/162964a8-ba20-4887-b8b9-e98369835a0d-configmap-kubelet-serving-ca-bundle\") pod \"162964a8-ba20-4887-b8b9-e98369835a0d\" (UID: \"162964a8-ba20-4887-b8b9-e98369835a0d\") " Jul 14 17:34:54.254491 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:54.254418 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gwqft\" (UniqueName: \"kubernetes.io/projected/162964a8-ba20-4887-b8b9-e98369835a0d-kube-api-access-gwqft\") pod \"162964a8-ba20-4887-b8b9-e98369835a0d\" (UID: \"162964a8-ba20-4887-b8b9-e98369835a0d\") " Jul 14 17:34:54.254491 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:54.254440 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/162964a8-ba20-4887-b8b9-e98369835a0d-secret-metrics-client-certs\") pod \"162964a8-ba20-4887-b8b9-e98369835a0d\" (UID: \"162964a8-ba20-4887-b8b9-e98369835a0d\") " Jul 14 17:34:54.254491 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:54.254465 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/162964a8-ba20-4887-b8b9-e98369835a0d-configmap-metrics-client-ca\") pod \"162964a8-ba20-4887-b8b9-e98369835a0d\" (UID: \"162964a8-ba20-4887-b8b9-e98369835a0d\") " Jul 14 17:34:54.254491 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:54.254493 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/162964a8-ba20-4887-b8b9-e98369835a0d-prometheus-trusted-ca-bundle\") pod \"162964a8-ba20-4887-b8b9-e98369835a0d\" (UID: \"162964a8-ba20-4887-b8b9-e98369835a0d\") " Jul 14 17:34:54.259966 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:54.254519 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/162964a8-ba20-4887-b8b9-e98369835a0d-config-out\") pod \"162964a8-ba20-4887-b8b9-e98369835a0d\" (UID: \"162964a8-ba20-4887-b8b9-e98369835a0d\") " Jul 14 17:34:54.259966 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:54.254517 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/162964a8-ba20-4887-b8b9-e98369835a0d-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "162964a8-ba20-4887-b8b9-e98369835a0d" (UID: "162964a8-ba20-4887-b8b9-e98369835a0d"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 17:34:54.259966 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:54.254567 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/162964a8-ba20-4887-b8b9-e98369835a0d-prometheus-k8s-rulefiles-1\") pod \"162964a8-ba20-4887-b8b9-e98369835a0d\" (UID: \"162964a8-ba20-4887-b8b9-e98369835a0d\") " Jul 14 17:34:54.259966 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:54.254591 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/162964a8-ba20-4887-b8b9-e98369835a0d-secret-kube-rbac-proxy\") pod \"162964a8-ba20-4887-b8b9-e98369835a0d\" (UID: \"162964a8-ba20-4887-b8b9-e98369835a0d\") " Jul 14 17:34:54.259966 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:54.254620 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/162964a8-ba20-4887-b8b9-e98369835a0d-prometheus-k8s-db\") pod \"162964a8-ba20-4887-b8b9-e98369835a0d\" (UID: \"162964a8-ba20-4887-b8b9-e98369835a0d\") " Jul 14 17:34:54.259966 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:54.255031 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/162964a8-ba20-4887-b8b9-e98369835a0d-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "162964a8-ba20-4887-b8b9-e98369835a0d" (UID: "162964a8-ba20-4887-b8b9-e98369835a0d"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 17:34:54.259966 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:54.255089 2562 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/162964a8-ba20-4887-b8b9-e98369835a0d-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 17:34:54.259966 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:54.257719 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/162964a8-ba20-4887-b8b9-e98369835a0d-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "162964a8-ba20-4887-b8b9-e98369835a0d" (UID: "162964a8-ba20-4887-b8b9-e98369835a0d"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 17:34:54.259966 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:54.258261 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/162964a8-ba20-4887-b8b9-e98369835a0d-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "162964a8-ba20-4887-b8b9-e98369835a0d" (UID: "162964a8-ba20-4887-b8b9-e98369835a0d"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 17:34:54.259966 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:54.258862 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/162964a8-ba20-4887-b8b9-e98369835a0d-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "162964a8-ba20-4887-b8b9-e98369835a0d" (UID: "162964a8-ba20-4887-b8b9-e98369835a0d"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 17:34:54.259966 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:54.258938 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/162964a8-ba20-4887-b8b9-e98369835a0d-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "162964a8-ba20-4887-b8b9-e98369835a0d" (UID: "162964a8-ba20-4887-b8b9-e98369835a0d"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 17:34:54.259966 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:54.259727 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/162964a8-ba20-4887-b8b9-e98369835a0d-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "162964a8-ba20-4887-b8b9-e98369835a0d" (UID: "162964a8-ba20-4887-b8b9-e98369835a0d"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 17:34:54.260572 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:54.260168 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/162964a8-ba20-4887-b8b9-e98369835a0d-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "162964a8-ba20-4887-b8b9-e98369835a0d" (UID: "162964a8-ba20-4887-b8b9-e98369835a0d"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 17:34:54.261021 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:54.260992 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/162964a8-ba20-4887-b8b9-e98369835a0d-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "162964a8-ba20-4887-b8b9-e98369835a0d" (UID: "162964a8-ba20-4887-b8b9-e98369835a0d"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 17:34:54.261391 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:54.261348 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/162964a8-ba20-4887-b8b9-e98369835a0d-config-out" (OuterVolumeSpecName: "config-out") pod "162964a8-ba20-4887-b8b9-e98369835a0d" (UID: "162964a8-ba20-4887-b8b9-e98369835a0d"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 17:34:54.261596 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:54.261549 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/162964a8-ba20-4887-b8b9-e98369835a0d-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "162964a8-ba20-4887-b8b9-e98369835a0d" (UID: "162964a8-ba20-4887-b8b9-e98369835a0d"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 17:34:54.261674 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:54.261649 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/162964a8-ba20-4887-b8b9-e98369835a0d-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "162964a8-ba20-4887-b8b9-e98369835a0d" (UID: "162964a8-ba20-4887-b8b9-e98369835a0d"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 17:34:54.262063 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:54.262006 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/162964a8-ba20-4887-b8b9-e98369835a0d-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "162964a8-ba20-4887-b8b9-e98369835a0d" (UID: "162964a8-ba20-4887-b8b9-e98369835a0d"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 17:34:54.262063 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:54.262008 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/162964a8-ba20-4887-b8b9-e98369835a0d-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "162964a8-ba20-4887-b8b9-e98369835a0d" (UID: "162964a8-ba20-4887-b8b9-e98369835a0d"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 17:34:54.262311 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:54.262070 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/162964a8-ba20-4887-b8b9-e98369835a0d-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "162964a8-ba20-4887-b8b9-e98369835a0d" (UID: "162964a8-ba20-4887-b8b9-e98369835a0d"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 17:34:54.262508 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:54.262410 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/162964a8-ba20-4887-b8b9-e98369835a0d-config" (OuterVolumeSpecName: "config") pod "162964a8-ba20-4887-b8b9-e98369835a0d" (UID: "162964a8-ba20-4887-b8b9-e98369835a0d"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 17:34:54.262508 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:54.262485 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/162964a8-ba20-4887-b8b9-e98369835a0d-kube-api-access-gwqft" (OuterVolumeSpecName: "kube-api-access-gwqft") pod "162964a8-ba20-4887-b8b9-e98369835a0d" (UID: "162964a8-ba20-4887-b8b9-e98369835a0d"). InnerVolumeSpecName "kube-api-access-gwqft". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 17:34:54.263944 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:54.263919 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/162964a8-ba20-4887-b8b9-e98369835a0d-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "162964a8-ba20-4887-b8b9-e98369835a0d" (UID: "162964a8-ba20-4887-b8b9-e98369835a0d"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 17:34:54.265644 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:54.265619 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/162964a8-ba20-4887-b8b9-e98369835a0d-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "162964a8-ba20-4887-b8b9-e98369835a0d" (UID: "162964a8-ba20-4887-b8b9-e98369835a0d"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 17:34:54.272644 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:54.272618 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/162964a8-ba20-4887-b8b9-e98369835a0d-web-config" (OuterVolumeSpecName: "web-config") pod "162964a8-ba20-4887-b8b9-e98369835a0d" (UID: "162964a8-ba20-4887-b8b9-e98369835a0d"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 17:34:54.356228 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:54.356183 2562 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/162964a8-ba20-4887-b8b9-e98369835a0d-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 17:34:54.356228 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:54.356225 2562 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/162964a8-ba20-4887-b8b9-e98369835a0d-config\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 17:34:54.356364 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:54.356238 2562 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/162964a8-ba20-4887-b8b9-e98369835a0d-secret-prometheus-k8s-tls\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 17:34:54.356364 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:54.356249 2562 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/162964a8-ba20-4887-b8b9-e98369835a0d-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 17:34:54.356364 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:54.356260 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gwqft\" (UniqueName: \"kubernetes.io/projected/162964a8-ba20-4887-b8b9-e98369835a0d-kube-api-access-gwqft\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 17:34:54.356364 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:54.356268 2562 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/162964a8-ba20-4887-b8b9-e98369835a0d-secret-metrics-client-certs\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 17:34:54.356364 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:54.356277 2562 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/162964a8-ba20-4887-b8b9-e98369835a0d-configmap-metrics-client-ca\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 17:34:54.356364 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:54.356286 2562 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/162964a8-ba20-4887-b8b9-e98369835a0d-prometheus-trusted-ca-bundle\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 17:34:54.356364 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:54.356294 2562 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/162964a8-ba20-4887-b8b9-e98369835a0d-config-out\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 17:34:54.356364 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:54.356303 2562 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/162964a8-ba20-4887-b8b9-e98369835a0d-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 17:34:54.356364 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:54.356311 2562 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/162964a8-ba20-4887-b8b9-e98369835a0d-secret-kube-rbac-proxy\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 17:34:54.356364 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:54.356321 2562 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/162964a8-ba20-4887-b8b9-e98369835a0d-prometheus-k8s-db\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 17:34:54.356364 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:54.356330 2562 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/162964a8-ba20-4887-b8b9-e98369835a0d-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 17:34:54.356364 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:54.356338 2562 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/162964a8-ba20-4887-b8b9-e98369835a0d-secret-grpc-tls\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 17:34:54.356364 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:54.356346 2562 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/162964a8-ba20-4887-b8b9-e98369835a0d-tls-assets\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 17:34:54.356364 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:54.356353 2562 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/162964a8-ba20-4887-b8b9-e98369835a0d-thanos-prometheus-http-client-file\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 17:34:54.356364 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:54.356361 2562 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/162964a8-ba20-4887-b8b9-e98369835a0d-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 17:34:54.356364 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:54.356369 2562 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/162964a8-ba20-4887-b8b9-e98369835a0d-web-config\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 17:34:54.356790 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:54.356377 2562 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/162964a8-ba20-4887-b8b9-e98369835a0d-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 17:34:55.195953 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:55.195920 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"162964a8-ba20-4887-b8b9-e98369835a0d","Type":"ContainerDied","Data":"37490a16d2fd0352738080f29c2f3d8adf08c49dc5bbf83ac070ae6dd0939f37"} Jul 14 17:34:55.196398 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:55.195977 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:34:55.196398 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:55.195992 2562 scope.go:117] "RemoveContainer" containerID="210019baefe3a41037b37b54a22651e239fb4c085cb7af1e3d44667fa347af67" Jul 14 17:34:55.204490 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:55.204468 2562 scope.go:117] "RemoveContainer" containerID="808f856eed06c1568c0bd1a7973a20d1113f8d7dee82f513131fff832f1ba39a" Jul 14 17:34:55.211376 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:55.211359 2562 scope.go:117] "RemoveContainer" containerID="82df51680a9ebde69f9e3d98674a97daa5d013af7d99f3880fa05209ce71ed39" Jul 14 17:34:55.218261 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:55.218246 2562 scope.go:117] "RemoveContainer" containerID="303a1cededc32c93c1f7a013e72ede71495ef80842bd6ee6ddada0bc1f2c5b37" Jul 14 17:34:55.227027 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:55.226101 2562 scope.go:117] "RemoveContainer" containerID="f3f5cacddd0c2a27cdfc51d6e2022c88ec5b13d7d475353933eac7ffdee5a9ae" Jul 14 17:34:55.228772 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:55.228752 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 14 17:34:55.231371 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:55.231350 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 14 17:34:55.234872 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:55.234858 2562 scope.go:117] "RemoveContainer" containerID="14eddb98ff4482142de2825a71c0306903354b4c989dba6b63e46f9981a1eb93" Jul 14 17:34:55.241705 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:55.241689 2562 scope.go:117] "RemoveContainer" containerID="774ee46e9534c817adfd22129cf38b9a5069b761a444f0b1a6dc9eb5d1419954" Jul 14 17:34:55.270800 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:55.270780 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 14 17:34:55.271043 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:55.271031 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="162964a8-ba20-4887-b8b9-e98369835a0d" containerName="prometheus" Jul 14 17:34:55.271043 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:55.271043 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="162964a8-ba20-4887-b8b9-e98369835a0d" containerName="prometheus" Jul 14 17:34:55.271119 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:55.271052 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="162964a8-ba20-4887-b8b9-e98369835a0d" containerName="kube-rbac-proxy-thanos" Jul 14 17:34:55.271119 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:55.271058 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="162964a8-ba20-4887-b8b9-e98369835a0d" containerName="kube-rbac-proxy-thanos" Jul 14 17:34:55.271119 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:55.271067 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="162964a8-ba20-4887-b8b9-e98369835a0d" containerName="thanos-sidecar" Jul 14 17:34:55.271119 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:55.271072 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="162964a8-ba20-4887-b8b9-e98369835a0d" containerName="thanos-sidecar" Jul 14 17:34:55.271119 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:55.271080 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="162964a8-ba20-4887-b8b9-e98369835a0d" containerName="init-config-reloader" Jul 14 17:34:55.271119 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:55.271085 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="162964a8-ba20-4887-b8b9-e98369835a0d" containerName="init-config-reloader" Jul 14 17:34:55.271119 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:55.271091 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="162964a8-ba20-4887-b8b9-e98369835a0d" containerName="kube-rbac-proxy" Jul 14 17:34:55.271119 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:55.271096 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="162964a8-ba20-4887-b8b9-e98369835a0d" containerName="kube-rbac-proxy" Jul 14 17:34:55.271119 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:55.271105 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="162964a8-ba20-4887-b8b9-e98369835a0d" containerName="config-reloader" Jul 14 17:34:55.271119 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:55.271109 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="162964a8-ba20-4887-b8b9-e98369835a0d" containerName="config-reloader" Jul 14 17:34:55.271119 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:55.271120 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="162964a8-ba20-4887-b8b9-e98369835a0d" containerName="kube-rbac-proxy-web" Jul 14 17:34:55.271119 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:55.271126 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="162964a8-ba20-4887-b8b9-e98369835a0d" containerName="kube-rbac-proxy-web" Jul 14 17:34:55.271555 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:55.271167 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="162964a8-ba20-4887-b8b9-e98369835a0d" containerName="config-reloader" Jul 14 17:34:55.271555 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:55.271175 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="162964a8-ba20-4887-b8b9-e98369835a0d" containerName="kube-rbac-proxy-thanos" Jul 14 17:34:55.271555 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:55.271198 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="162964a8-ba20-4887-b8b9-e98369835a0d" containerName="prometheus" Jul 14 17:34:55.271555 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:55.271204 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="162964a8-ba20-4887-b8b9-e98369835a0d" containerName="kube-rbac-proxy-web" Jul 14 17:34:55.271555 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:55.271209 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="162964a8-ba20-4887-b8b9-e98369835a0d" containerName="thanos-sidecar" Jul 14 17:34:55.271555 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:55.271216 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="162964a8-ba20-4887-b8b9-e98369835a0d" containerName="kube-rbac-proxy" Jul 14 17:34:55.275252 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:55.275236 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:34:55.280518 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:55.280499 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 14 17:34:55.282284 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:55.282260 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 14 17:34:55.282624 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:55.282609 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 14 17:34:55.283828 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:55.283811 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-54mnitaep2gna\"" Jul 14 17:34:55.283897 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:55.283883 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 14 17:34:55.284372 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:55.284115 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 14 17:34:55.284372 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:55.284139 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 14 17:34:55.284372 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:55.284160 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 14 17:34:55.284372 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:55.284237 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 14 17:34:55.284372 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:55.284249 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 14 17:34:55.284372 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:55.284159 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 14 17:34:55.284372 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:55.284359 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 14 17:34:55.284731 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:55.284410 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 14 17:34:55.284731 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:55.284610 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-jrt8q\"" Jul 14 17:34:55.286941 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:55.286921 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 14 17:34:55.288180 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:55.288162 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 14 17:34:55.300935 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:55.300916 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 14 17:34:55.364509 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:55.364479 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/4a8f645b-43a7-4f0b-a290-3d4c92ae2499-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"4a8f645b-43a7-4f0b-a290-3d4c92ae2499\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:34:55.364617 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:55.364517 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4a8f645b-43a7-4f0b-a290-3d4c92ae2499-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"4a8f645b-43a7-4f0b-a290-3d4c92ae2499\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:34:55.364617 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:55.364534 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/4a8f645b-43a7-4f0b-a290-3d4c92ae2499-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"4a8f645b-43a7-4f0b-a290-3d4c92ae2499\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:34:55.364617 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:55.364582 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4a8f645b-43a7-4f0b-a290-3d4c92ae2499-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4a8f645b-43a7-4f0b-a290-3d4c92ae2499\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:34:55.364617 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:55.364610 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4a8f645b-43a7-4f0b-a290-3d4c92ae2499-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"4a8f645b-43a7-4f0b-a290-3d4c92ae2499\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:34:55.364737 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:55.364632 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4a8f645b-43a7-4f0b-a290-3d4c92ae2499-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4a8f645b-43a7-4f0b-a290-3d4c92ae2499\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:34:55.364737 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:55.364666 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/4a8f645b-43a7-4f0b-a290-3d4c92ae2499-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"4a8f645b-43a7-4f0b-a290-3d4c92ae2499\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:34:55.364737 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:55.364681 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4a8f645b-43a7-4f0b-a290-3d4c92ae2499-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"4a8f645b-43a7-4f0b-a290-3d4c92ae2499\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:34:55.364737 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:55.364697 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4a8f645b-43a7-4f0b-a290-3d4c92ae2499-config-out\") pod \"prometheus-k8s-0\" (UID: \"4a8f645b-43a7-4f0b-a290-3d4c92ae2499\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:34:55.364851 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:55.364752 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4a8f645b-43a7-4f0b-a290-3d4c92ae2499-web-config\") pod \"prometheus-k8s-0\" (UID: \"4a8f645b-43a7-4f0b-a290-3d4c92ae2499\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:34:55.364851 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:55.364782 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/4a8f645b-43a7-4f0b-a290-3d4c92ae2499-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"4a8f645b-43a7-4f0b-a290-3d4c92ae2499\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:34:55.364851 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:55.364817 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4s7lx\" (UniqueName: \"kubernetes.io/projected/4a8f645b-43a7-4f0b-a290-3d4c92ae2499-kube-api-access-4s7lx\") pod \"prometheus-k8s-0\" (UID: \"4a8f645b-43a7-4f0b-a290-3d4c92ae2499\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:34:55.364851 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:55.364835 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/4a8f645b-43a7-4f0b-a290-3d4c92ae2499-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"4a8f645b-43a7-4f0b-a290-3d4c92ae2499\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:34:55.365005 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:55.364851 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4a8f645b-43a7-4f0b-a290-3d4c92ae2499-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"4a8f645b-43a7-4f0b-a290-3d4c92ae2499\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:34:55.365005 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:55.364869 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/4a8f645b-43a7-4f0b-a290-3d4c92ae2499-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"4a8f645b-43a7-4f0b-a290-3d4c92ae2499\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:34:55.365005 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:55.364884 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/4a8f645b-43a7-4f0b-a290-3d4c92ae2499-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"4a8f645b-43a7-4f0b-a290-3d4c92ae2499\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:34:55.365005 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:55.364931 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4a8f645b-43a7-4f0b-a290-3d4c92ae2499-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"4a8f645b-43a7-4f0b-a290-3d4c92ae2499\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:34:55.365005 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:55.364962 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/4a8f645b-43a7-4f0b-a290-3d4c92ae2499-config\") pod \"prometheus-k8s-0\" (UID: \"4a8f645b-43a7-4f0b-a290-3d4c92ae2499\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:34:55.365005 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:55.364991 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/4a8f645b-43a7-4f0b-a290-3d4c92ae2499-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"4a8f645b-43a7-4f0b-a290-3d4c92ae2499\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:34:55.365177 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:55.365016 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4a8f645b-43a7-4f0b-a290-3d4c92ae2499-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4a8f645b-43a7-4f0b-a290-3d4c92ae2499\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:34:55.465540 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:55.465506 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/4a8f645b-43a7-4f0b-a290-3d4c92ae2499-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"4a8f645b-43a7-4f0b-a290-3d4c92ae2499\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:34:55.465630 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:55.465549 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4a8f645b-43a7-4f0b-a290-3d4c92ae2499-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4a8f645b-43a7-4f0b-a290-3d4c92ae2499\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:34:55.465630 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:55.465572 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/4a8f645b-43a7-4f0b-a290-3d4c92ae2499-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"4a8f645b-43a7-4f0b-a290-3d4c92ae2499\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:34:55.465704 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:55.465683 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4a8f645b-43a7-4f0b-a290-3d4c92ae2499-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"4a8f645b-43a7-4f0b-a290-3d4c92ae2499\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:34:55.465744 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:55.465710 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/4a8f645b-43a7-4f0b-a290-3d4c92ae2499-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"4a8f645b-43a7-4f0b-a290-3d4c92ae2499\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:34:55.465744 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:55.465733 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4a8f645b-43a7-4f0b-a290-3d4c92ae2499-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4a8f645b-43a7-4f0b-a290-3d4c92ae2499\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:34:55.465839 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:55.465752 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4a8f645b-43a7-4f0b-a290-3d4c92ae2499-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"4a8f645b-43a7-4f0b-a290-3d4c92ae2499\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:34:55.465839 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:55.465778 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4a8f645b-43a7-4f0b-a290-3d4c92ae2499-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4a8f645b-43a7-4f0b-a290-3d4c92ae2499\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:34:55.466163 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:55.466138 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/4a8f645b-43a7-4f0b-a290-3d4c92ae2499-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"4a8f645b-43a7-4f0b-a290-3d4c92ae2499\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:34:55.466289 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:55.466253 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/4a8f645b-43a7-4f0b-a290-3d4c92ae2499-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"4a8f645b-43a7-4f0b-a290-3d4c92ae2499\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:34:55.466344 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:55.466291 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4a8f645b-43a7-4f0b-a290-3d4c92ae2499-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"4a8f645b-43a7-4f0b-a290-3d4c92ae2499\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:34:55.466344 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:55.466321 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4a8f645b-43a7-4f0b-a290-3d4c92ae2499-config-out\") pod \"prometheus-k8s-0\" (UID: \"4a8f645b-43a7-4f0b-a290-3d4c92ae2499\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:34:55.466446 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:55.466367 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4a8f645b-43a7-4f0b-a290-3d4c92ae2499-web-config\") pod \"prometheus-k8s-0\" (UID: \"4a8f645b-43a7-4f0b-a290-3d4c92ae2499\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:34:55.466446 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:55.466395 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/4a8f645b-43a7-4f0b-a290-3d4c92ae2499-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"4a8f645b-43a7-4f0b-a290-3d4c92ae2499\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:34:55.466446 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:55.466430 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4s7lx\" (UniqueName: \"kubernetes.io/projected/4a8f645b-43a7-4f0b-a290-3d4c92ae2499-kube-api-access-4s7lx\") pod \"prometheus-k8s-0\" (UID: \"4a8f645b-43a7-4f0b-a290-3d4c92ae2499\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:34:55.466755 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:55.466456 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/4a8f645b-43a7-4f0b-a290-3d4c92ae2499-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"4a8f645b-43a7-4f0b-a290-3d4c92ae2499\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:34:55.466755 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:55.466478 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4a8f645b-43a7-4f0b-a290-3d4c92ae2499-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"4a8f645b-43a7-4f0b-a290-3d4c92ae2499\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:34:55.466755 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:55.466500 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/4a8f645b-43a7-4f0b-a290-3d4c92ae2499-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"4a8f645b-43a7-4f0b-a290-3d4c92ae2499\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:34:55.466755 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:55.466522 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/4a8f645b-43a7-4f0b-a290-3d4c92ae2499-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"4a8f645b-43a7-4f0b-a290-3d4c92ae2499\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:34:55.466755 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:55.466529 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4a8f645b-43a7-4f0b-a290-3d4c92ae2499-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4a8f645b-43a7-4f0b-a290-3d4c92ae2499\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:34:55.466755 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:55.466534 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4a8f645b-43a7-4f0b-a290-3d4c92ae2499-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4a8f645b-43a7-4f0b-a290-3d4c92ae2499\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:34:55.466755 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:55.466557 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4a8f645b-43a7-4f0b-a290-3d4c92ae2499-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"4a8f645b-43a7-4f0b-a290-3d4c92ae2499\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:34:55.469489 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:55.468146 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4a8f645b-43a7-4f0b-a290-3d4c92ae2499-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4a8f645b-43a7-4f0b-a290-3d4c92ae2499\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:34:55.469489 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:55.468361 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/4a8f645b-43a7-4f0b-a290-3d4c92ae2499-config\") pod \"prometheus-k8s-0\" (UID: \"4a8f645b-43a7-4f0b-a290-3d4c92ae2499\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:34:55.469489 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:55.468417 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4a8f645b-43a7-4f0b-a290-3d4c92ae2499-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"4a8f645b-43a7-4f0b-a290-3d4c92ae2499\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:34:55.469489 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:55.468998 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/4a8f645b-43a7-4f0b-a290-3d4c92ae2499-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"4a8f645b-43a7-4f0b-a290-3d4c92ae2499\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:34:55.469762 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:55.469438 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4a8f645b-43a7-4f0b-a290-3d4c92ae2499-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"4a8f645b-43a7-4f0b-a290-3d4c92ae2499\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:34:55.471206 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:55.471159 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/4a8f645b-43a7-4f0b-a290-3d4c92ae2499-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"4a8f645b-43a7-4f0b-a290-3d4c92ae2499\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:34:55.471491 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:55.471469 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/4a8f645b-43a7-4f0b-a290-3d4c92ae2499-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"4a8f645b-43a7-4f0b-a290-3d4c92ae2499\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:34:55.476261 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:55.472511 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4a8f645b-43a7-4f0b-a290-3d4c92ae2499-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"4a8f645b-43a7-4f0b-a290-3d4c92ae2499\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:34:55.476261 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:55.472627 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/4a8f645b-43a7-4f0b-a290-3d4c92ae2499-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"4a8f645b-43a7-4f0b-a290-3d4c92ae2499\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:34:55.476261 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:55.473177 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4a8f645b-43a7-4f0b-a290-3d4c92ae2499-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"4a8f645b-43a7-4f0b-a290-3d4c92ae2499\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:34:55.476261 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:55.473281 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/4a8f645b-43a7-4f0b-a290-3d4c92ae2499-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"4a8f645b-43a7-4f0b-a290-3d4c92ae2499\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:34:55.476261 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:55.473616 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/4a8f645b-43a7-4f0b-a290-3d4c92ae2499-config\") pod \"prometheus-k8s-0\" (UID: \"4a8f645b-43a7-4f0b-a290-3d4c92ae2499\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:34:55.476261 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:55.474139 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/4a8f645b-43a7-4f0b-a290-3d4c92ae2499-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"4a8f645b-43a7-4f0b-a290-3d4c92ae2499\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:34:55.476261 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:55.474819 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4a8f645b-43a7-4f0b-a290-3d4c92ae2499-web-config\") pod \"prometheus-k8s-0\" (UID: \"4a8f645b-43a7-4f0b-a290-3d4c92ae2499\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:34:55.477079 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:55.477052 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4a8f645b-43a7-4f0b-a290-3d4c92ae2499-config-out\") pod \"prometheus-k8s-0\" (UID: \"4a8f645b-43a7-4f0b-a290-3d4c92ae2499\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:34:55.477166 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:55.477117 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/4a8f645b-43a7-4f0b-a290-3d4c92ae2499-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"4a8f645b-43a7-4f0b-a290-3d4c92ae2499\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:34:55.477166 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:55.477156 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4a8f645b-43a7-4f0b-a290-3d4c92ae2499-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"4a8f645b-43a7-4f0b-a290-3d4c92ae2499\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:34:55.479145 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:55.479126 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4s7lx\" (UniqueName: \"kubernetes.io/projected/4a8f645b-43a7-4f0b-a290-3d4c92ae2499-kube-api-access-4s7lx\") pod \"prometheus-k8s-0\" (UID: \"4a8f645b-43a7-4f0b-a290-3d4c92ae2499\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:34:55.518539 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:55.518486 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="162964a8-ba20-4887-b8b9-e98369835a0d" path="/var/lib/kubelet/pods/162964a8-ba20-4887-b8b9-e98369835a0d/volumes" Jul 14 17:34:55.584396 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:55.584361 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:34:55.727734 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:55.727576 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 14 17:34:55.730148 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:34:55.730109 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a8f645b_43a7_4f0b_a290_3d4c92ae2499.slice/crio-e24ca6efb5220e4cb5f16b09db962ac1c42ae53320918789cefe1b13bedc02da WatchSource:0}: Error finding container e24ca6efb5220e4cb5f16b09db962ac1c42ae53320918789cefe1b13bedc02da: Status 404 returned error can't find the container with id e24ca6efb5220e4cb5f16b09db962ac1c42ae53320918789cefe1b13bedc02da Jul 14 17:34:56.201908 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:56.201866 2562 generic.go:356] "Generic (PLEG): container finished" podID="4a8f645b-43a7-4f0b-a290-3d4c92ae2499" containerID="42bced5446dd548457dbe241066ac1b793a6bdc08a89d1c220eb9aea20fe8235" exitCode=0 Jul 14 17:34:56.202280 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:56.201926 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4a8f645b-43a7-4f0b-a290-3d4c92ae2499","Type":"ContainerDied","Data":"42bced5446dd548457dbe241066ac1b793a6bdc08a89d1c220eb9aea20fe8235"} Jul 14 17:34:56.202280 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:56.201987 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4a8f645b-43a7-4f0b-a290-3d4c92ae2499","Type":"ContainerStarted","Data":"e24ca6efb5220e4cb5f16b09db962ac1c42ae53320918789cefe1b13bedc02da"} Jul 14 17:34:57.209750 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:57.209716 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4a8f645b-43a7-4f0b-a290-3d4c92ae2499","Type":"ContainerStarted","Data":"a4967c4ed46454626fd917d940fb15081c7b6195b614c3f93b4f0c4d44174698"} Jul 14 17:34:57.210134 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:57.209779 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4a8f645b-43a7-4f0b-a290-3d4c92ae2499","Type":"ContainerStarted","Data":"b8e26ab94710590074fc7e2a5ca4917297e2c8d3875e11c8e8d175c3fb80d2fb"} Jul 14 17:34:57.210134 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:57.209805 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4a8f645b-43a7-4f0b-a290-3d4c92ae2499","Type":"ContainerStarted","Data":"a32c2cb26532a05f15d6807cadf0f194713d6f33054d37d448ded8422a5c770d"} Jul 14 17:34:57.210134 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:57.209829 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4a8f645b-43a7-4f0b-a290-3d4c92ae2499","Type":"ContainerStarted","Data":"2ed1511b461d5cb78c17063385609181e0772dd5810c859d12d0a2791a7b8cde"} Jul 14 17:34:57.210134 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:57.209852 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4a8f645b-43a7-4f0b-a290-3d4c92ae2499","Type":"ContainerStarted","Data":"6ee61bcff2d396978e0039fd974534c3180385f67d2dcca8cf6675fbdd386b1f"} Jul 14 17:34:57.210134 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:57.209876 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4a8f645b-43a7-4f0b-a290-3d4c92ae2499","Type":"ContainerStarted","Data":"6a5618fdbf3db4236773b8f39e957e1da6751948af655d6d44432fbfa8d185eb"} Jul 14 17:34:57.253900 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:34:57.253852 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.235273694 podStartE2EDuration="2.253831329s" totalImagesPullingTime="0s" totalInitContainerRuntime="18.557635ms" isStatefulPod=false podCreationTimestamp="2026-07-14 17:34:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:34:57.251484269 +0000 UTC m=+146.327941807" watchObservedRunningTime="2026-07-14 17:34:57.253831329 +0000 UTC m=+146.330288868" Jul 14 17:35:00.585213 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:35:00.585162 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:35:10.185018 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:35:10.184976 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7dcb8b898d-zz5c2"] Jul 14 17:35:10.187613 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:35:10.187597 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7dcb8b898d-zz5c2" Jul 14 17:35:10.207478 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:35:10.207455 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7dcb8b898d-zz5c2"] Jul 14 17:35:10.287328 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:35:10.287308 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9139bbbf-3fd7-48de-9172-9b3551b89b44-trusted-ca-bundle\") pod \"console-7dcb8b898d-zz5c2\" (UID: \"9139bbbf-3fd7-48de-9172-9b3551b89b44\") " pod="openshift-console/console-7dcb8b898d-zz5c2" Jul 14 17:35:10.287444 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:35:10.287353 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9139bbbf-3fd7-48de-9172-9b3551b89b44-console-config\") pod \"console-7dcb8b898d-zz5c2\" (UID: \"9139bbbf-3fd7-48de-9172-9b3551b89b44\") " pod="openshift-console/console-7dcb8b898d-zz5c2" Jul 14 17:35:10.287444 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:35:10.287378 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9139bbbf-3fd7-48de-9172-9b3551b89b44-console-serving-cert\") pod \"console-7dcb8b898d-zz5c2\" (UID: \"9139bbbf-3fd7-48de-9172-9b3551b89b44\") " pod="openshift-console/console-7dcb8b898d-zz5c2" Jul 14 17:35:10.287532 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:35:10.287436 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9139bbbf-3fd7-48de-9172-9b3551b89b44-service-ca\") pod \"console-7dcb8b898d-zz5c2\" (UID: \"9139bbbf-3fd7-48de-9172-9b3551b89b44\") " pod="openshift-console/console-7dcb8b898d-zz5c2" Jul 14 17:35:10.287532 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:35:10.287482 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9139bbbf-3fd7-48de-9172-9b3551b89b44-oauth-serving-cert\") pod \"console-7dcb8b898d-zz5c2\" (UID: \"9139bbbf-3fd7-48de-9172-9b3551b89b44\") " pod="openshift-console/console-7dcb8b898d-zz5c2" Jul 14 17:35:10.287532 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:35:10.287501 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-27hf6\" (UniqueName: \"kubernetes.io/projected/9139bbbf-3fd7-48de-9172-9b3551b89b44-kube-api-access-27hf6\") pod \"console-7dcb8b898d-zz5c2\" (UID: \"9139bbbf-3fd7-48de-9172-9b3551b89b44\") " pod="openshift-console/console-7dcb8b898d-zz5c2" Jul 14 17:35:10.287532 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:35:10.287528 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9139bbbf-3fd7-48de-9172-9b3551b89b44-console-oauth-config\") pod \"console-7dcb8b898d-zz5c2\" (UID: \"9139bbbf-3fd7-48de-9172-9b3551b89b44\") " pod="openshift-console/console-7dcb8b898d-zz5c2" Jul 14 17:35:10.388221 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:35:10.388171 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9139bbbf-3fd7-48de-9172-9b3551b89b44-console-config\") pod \"console-7dcb8b898d-zz5c2\" (UID: \"9139bbbf-3fd7-48de-9172-9b3551b89b44\") " pod="openshift-console/console-7dcb8b898d-zz5c2" Jul 14 17:35:10.388316 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:35:10.388231 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9139bbbf-3fd7-48de-9172-9b3551b89b44-console-serving-cert\") pod \"console-7dcb8b898d-zz5c2\" (UID: \"9139bbbf-3fd7-48de-9172-9b3551b89b44\") " pod="openshift-console/console-7dcb8b898d-zz5c2" Jul 14 17:35:10.388316 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:35:10.388252 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9139bbbf-3fd7-48de-9172-9b3551b89b44-service-ca\") pod \"console-7dcb8b898d-zz5c2\" (UID: \"9139bbbf-3fd7-48de-9172-9b3551b89b44\") " pod="openshift-console/console-7dcb8b898d-zz5c2" Jul 14 17:35:10.388316 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:35:10.388276 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9139bbbf-3fd7-48de-9172-9b3551b89b44-oauth-serving-cert\") pod \"console-7dcb8b898d-zz5c2\" (UID: \"9139bbbf-3fd7-48de-9172-9b3551b89b44\") " pod="openshift-console/console-7dcb8b898d-zz5c2" Jul 14 17:35:10.388316 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:35:10.388291 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-27hf6\" (UniqueName: \"kubernetes.io/projected/9139bbbf-3fd7-48de-9172-9b3551b89b44-kube-api-access-27hf6\") pod \"console-7dcb8b898d-zz5c2\" (UID: \"9139bbbf-3fd7-48de-9172-9b3551b89b44\") " pod="openshift-console/console-7dcb8b898d-zz5c2" Jul 14 17:35:10.388316 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:35:10.388316 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9139bbbf-3fd7-48de-9172-9b3551b89b44-console-oauth-config\") pod \"console-7dcb8b898d-zz5c2\" (UID: \"9139bbbf-3fd7-48de-9172-9b3551b89b44\") " pod="openshift-console/console-7dcb8b898d-zz5c2" Jul 14 17:35:10.388559 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:35:10.388339 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9139bbbf-3fd7-48de-9172-9b3551b89b44-trusted-ca-bundle\") pod \"console-7dcb8b898d-zz5c2\" (UID: \"9139bbbf-3fd7-48de-9172-9b3551b89b44\") " pod="openshift-console/console-7dcb8b898d-zz5c2" Jul 14 17:35:10.388942 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:35:10.388917 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9139bbbf-3fd7-48de-9172-9b3551b89b44-service-ca\") pod \"console-7dcb8b898d-zz5c2\" (UID: \"9139bbbf-3fd7-48de-9172-9b3551b89b44\") " pod="openshift-console/console-7dcb8b898d-zz5c2" Jul 14 17:35:10.389051 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:35:10.388955 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9139bbbf-3fd7-48de-9172-9b3551b89b44-console-config\") pod \"console-7dcb8b898d-zz5c2\" (UID: \"9139bbbf-3fd7-48de-9172-9b3551b89b44\") " pod="openshift-console/console-7dcb8b898d-zz5c2" Jul 14 17:35:10.389255 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:35:10.389232 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9139bbbf-3fd7-48de-9172-9b3551b89b44-oauth-serving-cert\") pod \"console-7dcb8b898d-zz5c2\" (UID: \"9139bbbf-3fd7-48de-9172-9b3551b89b44\") " pod="openshift-console/console-7dcb8b898d-zz5c2" Jul 14 17:35:10.389319 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:35:10.389302 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9139bbbf-3fd7-48de-9172-9b3551b89b44-trusted-ca-bundle\") pod \"console-7dcb8b898d-zz5c2\" (UID: \"9139bbbf-3fd7-48de-9172-9b3551b89b44\") " pod="openshift-console/console-7dcb8b898d-zz5c2" Jul 14 17:35:10.390815 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:35:10.390789 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9139bbbf-3fd7-48de-9172-9b3551b89b44-console-oauth-config\") pod \"console-7dcb8b898d-zz5c2\" (UID: \"9139bbbf-3fd7-48de-9172-9b3551b89b44\") " pod="openshift-console/console-7dcb8b898d-zz5c2" Jul 14 17:35:10.391019 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:35:10.390999 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9139bbbf-3fd7-48de-9172-9b3551b89b44-console-serving-cert\") pod \"console-7dcb8b898d-zz5c2\" (UID: \"9139bbbf-3fd7-48de-9172-9b3551b89b44\") " pod="openshift-console/console-7dcb8b898d-zz5c2" Jul 14 17:35:10.398106 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:35:10.398083 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-27hf6\" (UniqueName: \"kubernetes.io/projected/9139bbbf-3fd7-48de-9172-9b3551b89b44-kube-api-access-27hf6\") pod \"console-7dcb8b898d-zz5c2\" (UID: \"9139bbbf-3fd7-48de-9172-9b3551b89b44\") " pod="openshift-console/console-7dcb8b898d-zz5c2" Jul 14 17:35:10.496949 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:35:10.496874 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7dcb8b898d-zz5c2" Jul 14 17:35:10.635025 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:35:10.634997 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7dcb8b898d-zz5c2"] Jul 14 17:35:10.636812 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:35:10.636782 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9139bbbf_3fd7_48de_9172_9b3551b89b44.slice/crio-290c69b7b3238152823211f8e7e04fcbd22be2aedf4f13eccf258d071fe59058 WatchSource:0}: Error finding container 290c69b7b3238152823211f8e7e04fcbd22be2aedf4f13eccf258d071fe59058: Status 404 returned error can't find the container with id 290c69b7b3238152823211f8e7e04fcbd22be2aedf4f13eccf258d071fe59058 Jul 14 17:35:11.269494 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:35:11.269456 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7dcb8b898d-zz5c2" event={"ID":"9139bbbf-3fd7-48de-9172-9b3551b89b44","Type":"ContainerStarted","Data":"b869d5b2a89b372cb7b39f0475c6f2d6d044bdfe7f4b99c1a0b3ad4156b897b8"} Jul 14 17:35:11.269970 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:35:11.269511 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7dcb8b898d-zz5c2" event={"ID":"9139bbbf-3fd7-48de-9172-9b3551b89b44","Type":"ContainerStarted","Data":"290c69b7b3238152823211f8e7e04fcbd22be2aedf4f13eccf258d071fe59058"} Jul 14 17:35:11.290903 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:35:11.290844 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7dcb8b898d-zz5c2" podStartSLOduration=1.29082224 podStartE2EDuration="1.29082224s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:35:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:35:11.2896589 +0000 UTC m=+160.366116438" watchObservedRunningTime="2026-07-14 17:35:11.29082224 +0000 UTC m=+160.367279783" Jul 14 17:35:20.497642 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:35:20.497607 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7dcb8b898d-zz5c2" Jul 14 17:35:20.498089 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:35:20.497672 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7dcb8b898d-zz5c2" Jul 14 17:35:20.502054 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:35:20.502034 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7dcb8b898d-zz5c2" Jul 14 17:35:21.311585 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:35:21.311558 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7dcb8b898d-zz5c2" Jul 14 17:35:21.398464 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:35:21.398428 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-67dfb76bf5-hvq26"] Jul 14 17:35:46.425535 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:35:46.425480 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-67dfb76bf5-hvq26" podUID="e190d466-c58f-4604-bf7f-77d668a5cba4" containerName="console" containerID="cri-o://c6b9ce4131cc409da2587e5370555046abc8501d0b8c73efe7aa8c76335a0558" gracePeriod=15 Jul 14 17:35:46.663486 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:35:46.663457 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-67dfb76bf5-hvq26" Jul 14 17:35:46.789163 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:35:46.789084 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e190d466-c58f-4604-bf7f-77d668a5cba4-console-config\") pod \"e190d466-c58f-4604-bf7f-77d668a5cba4\" (UID: \"e190d466-c58f-4604-bf7f-77d668a5cba4\") " Jul 14 17:35:46.789163 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:35:46.789154 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e190d466-c58f-4604-bf7f-77d668a5cba4-console-serving-cert\") pod \"e190d466-c58f-4604-bf7f-77d668a5cba4\" (UID: \"e190d466-c58f-4604-bf7f-77d668a5cba4\") " Jul 14 17:35:46.789384 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:35:46.789171 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e190d466-c58f-4604-bf7f-77d668a5cba4-oauth-serving-cert\") pod \"e190d466-c58f-4604-bf7f-77d668a5cba4\" (UID: \"e190d466-c58f-4604-bf7f-77d668a5cba4\") " Jul 14 17:35:46.789384 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:35:46.789242 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e190d466-c58f-4604-bf7f-77d668a5cba4-console-oauth-config\") pod \"e190d466-c58f-4604-bf7f-77d668a5cba4\" (UID: \"e190d466-c58f-4604-bf7f-77d668a5cba4\") " Jul 14 17:35:46.789384 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:35:46.789268 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e190d466-c58f-4604-bf7f-77d668a5cba4-trusted-ca-bundle\") pod \"e190d466-c58f-4604-bf7f-77d668a5cba4\" (UID: \"e190d466-c58f-4604-bf7f-77d668a5cba4\") " Jul 14 17:35:46.789384 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:35:46.789293 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e190d466-c58f-4604-bf7f-77d668a5cba4-service-ca\") pod \"e190d466-c58f-4604-bf7f-77d668a5cba4\" (UID: \"e190d466-c58f-4604-bf7f-77d668a5cba4\") " Jul 14 17:35:46.789384 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:35:46.789341 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hgqvd\" (UniqueName: \"kubernetes.io/projected/e190d466-c58f-4604-bf7f-77d668a5cba4-kube-api-access-hgqvd\") pod \"e190d466-c58f-4604-bf7f-77d668a5cba4\" (UID: \"e190d466-c58f-4604-bf7f-77d668a5cba4\") " Jul 14 17:35:46.789620 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:35:46.789596 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e190d466-c58f-4604-bf7f-77d668a5cba4-console-config" (OuterVolumeSpecName: "console-config") pod "e190d466-c58f-4604-bf7f-77d668a5cba4" (UID: "e190d466-c58f-4604-bf7f-77d668a5cba4"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 17:35:46.789752 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:35:46.789715 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e190d466-c58f-4604-bf7f-77d668a5cba4-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "e190d466-c58f-4604-bf7f-77d668a5cba4" (UID: "e190d466-c58f-4604-bf7f-77d668a5cba4"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 17:35:46.789851 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:35:46.789734 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e190d466-c58f-4604-bf7f-77d668a5cba4-service-ca" (OuterVolumeSpecName: "service-ca") pod "e190d466-c58f-4604-bf7f-77d668a5cba4" (UID: "e190d466-c58f-4604-bf7f-77d668a5cba4"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 17:35:46.789851 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:35:46.789737 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e190d466-c58f-4604-bf7f-77d668a5cba4-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "e190d466-c58f-4604-bf7f-77d668a5cba4" (UID: "e190d466-c58f-4604-bf7f-77d668a5cba4"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 17:35:46.791524 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:35:46.791492 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e190d466-c58f-4604-bf7f-77d668a5cba4-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "e190d466-c58f-4604-bf7f-77d668a5cba4" (UID: "e190d466-c58f-4604-bf7f-77d668a5cba4"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 17:35:46.791700 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:35:46.791676 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e190d466-c58f-4604-bf7f-77d668a5cba4-kube-api-access-hgqvd" (OuterVolumeSpecName: "kube-api-access-hgqvd") pod "e190d466-c58f-4604-bf7f-77d668a5cba4" (UID: "e190d466-c58f-4604-bf7f-77d668a5cba4"). InnerVolumeSpecName "kube-api-access-hgqvd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 17:35:46.791749 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:35:46.791704 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e190d466-c58f-4604-bf7f-77d668a5cba4-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "e190d466-c58f-4604-bf7f-77d668a5cba4" (UID: "e190d466-c58f-4604-bf7f-77d668a5cba4"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 17:35:46.890387 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:35:46.890353 2562 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e190d466-c58f-4604-bf7f-77d668a5cba4-console-serving-cert\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 17:35:46.890387 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:35:46.890379 2562 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e190d466-c58f-4604-bf7f-77d668a5cba4-oauth-serving-cert\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 17:35:46.890387 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:35:46.890388 2562 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e190d466-c58f-4604-bf7f-77d668a5cba4-console-oauth-config\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 17:35:46.890387 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:35:46.890396 2562 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e190d466-c58f-4604-bf7f-77d668a5cba4-trusted-ca-bundle\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 17:35:46.890618 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:35:46.890405 2562 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e190d466-c58f-4604-bf7f-77d668a5cba4-service-ca\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 17:35:46.890618 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:35:46.890414 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hgqvd\" (UniqueName: \"kubernetes.io/projected/e190d466-c58f-4604-bf7f-77d668a5cba4-kube-api-access-hgqvd\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 17:35:46.890618 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:35:46.890422 2562 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e190d466-c58f-4604-bf7f-77d668a5cba4-console-config\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 17:35:47.420713 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:35:47.420674 2562 generic.go:356] "Generic (PLEG): container finished" podID="e190d466-c58f-4604-bf7f-77d668a5cba4" containerID="c6b9ce4131cc409da2587e5370555046abc8501d0b8c73efe7aa8c76335a0558" exitCode=0 Jul 14 17:35:47.420911 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:35:47.420728 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-67dfb76bf5-hvq26" Jul 14 17:35:47.420911 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:35:47.420750 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-67dfb76bf5-hvq26" event={"ID":"e190d466-c58f-4604-bf7f-77d668a5cba4","Type":"ContainerDied","Data":"c6b9ce4131cc409da2587e5370555046abc8501d0b8c73efe7aa8c76335a0558"} Jul 14 17:35:47.420911 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:35:47.420815 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-67dfb76bf5-hvq26" event={"ID":"e190d466-c58f-4604-bf7f-77d668a5cba4","Type":"ContainerDied","Data":"65107d5dc1432806e0fd19cdb76dacb0d2ef2227dea691b15a54e8d363994ab1"} Jul 14 17:35:47.420911 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:35:47.420837 2562 scope.go:117] "RemoveContainer" containerID="c6b9ce4131cc409da2587e5370555046abc8501d0b8c73efe7aa8c76335a0558" Jul 14 17:35:47.429367 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:35:47.429178 2562 scope.go:117] "RemoveContainer" containerID="c6b9ce4131cc409da2587e5370555046abc8501d0b8c73efe7aa8c76335a0558" Jul 14 17:35:47.429577 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:35:47.429424 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c6b9ce4131cc409da2587e5370555046abc8501d0b8c73efe7aa8c76335a0558\": container with ID starting with c6b9ce4131cc409da2587e5370555046abc8501d0b8c73efe7aa8c76335a0558 not found: ID does not exist" containerID="c6b9ce4131cc409da2587e5370555046abc8501d0b8c73efe7aa8c76335a0558" Jul 14 17:35:47.429577 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:35:47.429456 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c6b9ce4131cc409da2587e5370555046abc8501d0b8c73efe7aa8c76335a0558"} err="failed to get container status \"c6b9ce4131cc409da2587e5370555046abc8501d0b8c73efe7aa8c76335a0558\": rpc error: code = NotFound desc = could not find container \"c6b9ce4131cc409da2587e5370555046abc8501d0b8c73efe7aa8c76335a0558\": container with ID starting with c6b9ce4131cc409da2587e5370555046abc8501d0b8c73efe7aa8c76335a0558 not found: ID does not exist" Jul 14 17:35:47.444918 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:35:47.444894 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-67dfb76bf5-hvq26"] Jul 14 17:35:47.449262 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:35:47.449243 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-67dfb76bf5-hvq26"] Jul 14 17:35:47.518465 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:35:47.518440 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e190d466-c58f-4604-bf7f-77d668a5cba4" path="/var/lib/kubelet/pods/e190d466-c58f-4604-bf7f-77d668a5cba4/volumes" Jul 14 17:35:55.585355 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:35:55.585323 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:35:55.600910 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:35:55.600885 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:35:56.468754 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:35:56.468729 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 14 17:36:52.150044 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:36:52.149968 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-tbssn"] Jul 14 17:36:52.150460 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:36:52.150303 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e190d466-c58f-4604-bf7f-77d668a5cba4" containerName="console" Jul 14 17:36:52.150460 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:36:52.150316 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="e190d466-c58f-4604-bf7f-77d668a5cba4" containerName="console" Jul 14 17:36:52.150460 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:36:52.150387 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="e190d466-c58f-4604-bf7f-77d668a5cba4" containerName="console" Jul 14 17:36:52.153023 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:36:52.153007 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-tbssn" Jul 14 17:36:52.155891 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:36:52.155871 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 14 17:36:52.156673 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:36:52.156659 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 14 17:36:52.156673 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:36:52.156667 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-zlkb7\"" Jul 14 17:36:52.166205 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:36:52.166151 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-tbssn"] Jul 14 17:36:52.213021 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:36:52.212998 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s8zdb\" (UniqueName: \"kubernetes.io/projected/0a7c0111-57fe-466b-8fc7-1bf171c9b25d-kube-api-access-s8zdb\") pod \"cert-manager-55b4d954d5-tbssn\" (UID: \"0a7c0111-57fe-466b-8fc7-1bf171c9b25d\") " pod="cert-manager/cert-manager-55b4d954d5-tbssn" Jul 14 17:36:52.213112 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:36:52.213034 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0a7c0111-57fe-466b-8fc7-1bf171c9b25d-bound-sa-token\") pod \"cert-manager-55b4d954d5-tbssn\" (UID: \"0a7c0111-57fe-466b-8fc7-1bf171c9b25d\") " pod="cert-manager/cert-manager-55b4d954d5-tbssn" Jul 14 17:36:52.313819 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:36:52.313795 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s8zdb\" (UniqueName: \"kubernetes.io/projected/0a7c0111-57fe-466b-8fc7-1bf171c9b25d-kube-api-access-s8zdb\") pod \"cert-manager-55b4d954d5-tbssn\" (UID: \"0a7c0111-57fe-466b-8fc7-1bf171c9b25d\") " pod="cert-manager/cert-manager-55b4d954d5-tbssn" Jul 14 17:36:52.313918 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:36:52.313835 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0a7c0111-57fe-466b-8fc7-1bf171c9b25d-bound-sa-token\") pod \"cert-manager-55b4d954d5-tbssn\" (UID: \"0a7c0111-57fe-466b-8fc7-1bf171c9b25d\") " pod="cert-manager/cert-manager-55b4d954d5-tbssn" Jul 14 17:36:52.323833 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:36:52.323806 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s8zdb\" (UniqueName: \"kubernetes.io/projected/0a7c0111-57fe-466b-8fc7-1bf171c9b25d-kube-api-access-s8zdb\") pod \"cert-manager-55b4d954d5-tbssn\" (UID: \"0a7c0111-57fe-466b-8fc7-1bf171c9b25d\") " pod="cert-manager/cert-manager-55b4d954d5-tbssn" Jul 14 17:36:52.324047 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:36:52.324029 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0a7c0111-57fe-466b-8fc7-1bf171c9b25d-bound-sa-token\") pod \"cert-manager-55b4d954d5-tbssn\" (UID: \"0a7c0111-57fe-466b-8fc7-1bf171c9b25d\") " pod="cert-manager/cert-manager-55b4d954d5-tbssn" Jul 14 17:36:52.482451 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:36:52.482380 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-tbssn" Jul 14 17:36:52.631610 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:36:52.631575 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-tbssn"] Jul 14 17:36:52.635688 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:36:52.635652 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0a7c0111_57fe_466b_8fc7_1bf171c9b25d.slice/crio-4c2d6e563bd61c2e16dc31cf7d8db3cb5aa32843c0529e073e7f22726ac1a6ce WatchSource:0}: Error finding container 4c2d6e563bd61c2e16dc31cf7d8db3cb5aa32843c0529e073e7f22726ac1a6ce: Status 404 returned error can't find the container with id 4c2d6e563bd61c2e16dc31cf7d8db3cb5aa32843c0529e073e7f22726ac1a6ce Jul 14 17:36:52.658060 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:36:52.658035 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-tbssn" event={"ID":"0a7c0111-57fe-466b-8fc7-1bf171c9b25d","Type":"ContainerStarted","Data":"4c2d6e563bd61c2e16dc31cf7d8db3cb5aa32843c0529e073e7f22726ac1a6ce"} Jul 14 17:36:57.679905 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:36:57.679868 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-tbssn" event={"ID":"0a7c0111-57fe-466b-8fc7-1bf171c9b25d","Type":"ContainerStarted","Data":"41ce0550c3bdf780b0c5ce1c1bd974ae1eab7138a7935b7b12ecefbfa361b7e7"} Jul 14 17:36:57.710711 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:36:57.710660 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-tbssn" podStartSLOduration=1.575526333 podStartE2EDuration="5.710644123s" totalImagesPullingTime="4.13511779s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:36:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:36:57.709593492 +0000 UTC m=+266.786051056" watchObservedRunningTime="2026-07-14 17:36:57.710644123 +0000 UTC m=+266.787101662" Jul 14 17:37:19.183429 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:19.183395 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7558c48c44-gnp72"] Jul 14 17:37:19.188665 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:19.188643 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7558c48c44-gnp72" Jul 14 17:37:19.192428 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:19.192317 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 14 17:37:19.192837 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:19.192811 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-xk54r\"" Jul 14 17:37:19.193016 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:19.192989 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 14 17:37:19.193242 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:19.193222 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 14 17:37:19.197035 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:19.197019 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 14 17:37:19.231887 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:19.231865 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7558c48c44-gnp72"] Jul 14 17:37:19.235672 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:19.235655 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3f0ef414-c4bb-492c-98d8-9bc1f07a9b6f-tmp\") pod \"opendatahub-operator-controller-manager-7558c48c44-gnp72\" (UID: \"3f0ef414-c4bb-492c-98d8-9bc1f07a9b6f\") " pod="opendatahub/opendatahub-operator-controller-manager-7558c48c44-gnp72" Jul 14 17:37:19.235727 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:19.235695 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/3f0ef414-c4bb-492c-98d8-9bc1f07a9b6f-manifests\") pod \"opendatahub-operator-controller-manager-7558c48c44-gnp72\" (UID: \"3f0ef414-c4bb-492c-98d8-9bc1f07a9b6f\") " pod="opendatahub/opendatahub-operator-controller-manager-7558c48c44-gnp72" Jul 14 17:37:19.235727 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:19.235716 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kbw2n\" (UniqueName: \"kubernetes.io/projected/3f0ef414-c4bb-492c-98d8-9bc1f07a9b6f-kube-api-access-kbw2n\") pod \"opendatahub-operator-controller-manager-7558c48c44-gnp72\" (UID: \"3f0ef414-c4bb-492c-98d8-9bc1f07a9b6f\") " pod="opendatahub/opendatahub-operator-controller-manager-7558c48c44-gnp72" Jul 14 17:37:19.235804 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:19.235746 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/3f0ef414-c4bb-492c-98d8-9bc1f07a9b6f-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7558c48c44-gnp72\" (UID: \"3f0ef414-c4bb-492c-98d8-9bc1f07a9b6f\") " pod="opendatahub/opendatahub-operator-controller-manager-7558c48c44-gnp72" Jul 14 17:37:19.235804 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:19.235765 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3f0ef414-c4bb-492c-98d8-9bc1f07a9b6f-webhook-cert\") pod \"opendatahub-operator-controller-manager-7558c48c44-gnp72\" (UID: \"3f0ef414-c4bb-492c-98d8-9bc1f07a9b6f\") " pod="opendatahub/opendatahub-operator-controller-manager-7558c48c44-gnp72" Jul 14 17:37:19.336118 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:19.336089 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3f0ef414-c4bb-492c-98d8-9bc1f07a9b6f-tmp\") pod \"opendatahub-operator-controller-manager-7558c48c44-gnp72\" (UID: \"3f0ef414-c4bb-492c-98d8-9bc1f07a9b6f\") " pod="opendatahub/opendatahub-operator-controller-manager-7558c48c44-gnp72" Jul 14 17:37:19.336286 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:19.336130 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/3f0ef414-c4bb-492c-98d8-9bc1f07a9b6f-manifests\") pod \"opendatahub-operator-controller-manager-7558c48c44-gnp72\" (UID: \"3f0ef414-c4bb-492c-98d8-9bc1f07a9b6f\") " pod="opendatahub/opendatahub-operator-controller-manager-7558c48c44-gnp72" Jul 14 17:37:19.336286 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:19.336150 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kbw2n\" (UniqueName: \"kubernetes.io/projected/3f0ef414-c4bb-492c-98d8-9bc1f07a9b6f-kube-api-access-kbw2n\") pod \"opendatahub-operator-controller-manager-7558c48c44-gnp72\" (UID: \"3f0ef414-c4bb-492c-98d8-9bc1f07a9b6f\") " pod="opendatahub/opendatahub-operator-controller-manager-7558c48c44-gnp72" Jul 14 17:37:19.336286 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:19.336170 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/3f0ef414-c4bb-492c-98d8-9bc1f07a9b6f-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7558c48c44-gnp72\" (UID: \"3f0ef414-c4bb-492c-98d8-9bc1f07a9b6f\") " pod="opendatahub/opendatahub-operator-controller-manager-7558c48c44-gnp72" Jul 14 17:37:19.336286 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:19.336247 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3f0ef414-c4bb-492c-98d8-9bc1f07a9b6f-webhook-cert\") pod \"opendatahub-operator-controller-manager-7558c48c44-gnp72\" (UID: \"3f0ef414-c4bb-492c-98d8-9bc1f07a9b6f\") " pod="opendatahub/opendatahub-operator-controller-manager-7558c48c44-gnp72" Jul 14 17:37:19.336632 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:19.336603 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/3f0ef414-c4bb-492c-98d8-9bc1f07a9b6f-manifests\") pod \"opendatahub-operator-controller-manager-7558c48c44-gnp72\" (UID: \"3f0ef414-c4bb-492c-98d8-9bc1f07a9b6f\") " pod="opendatahub/opendatahub-operator-controller-manager-7558c48c44-gnp72" Jul 14 17:37:19.338433 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:19.338405 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3f0ef414-c4bb-492c-98d8-9bc1f07a9b6f-tmp\") pod \"opendatahub-operator-controller-manager-7558c48c44-gnp72\" (UID: \"3f0ef414-c4bb-492c-98d8-9bc1f07a9b6f\") " pod="opendatahub/opendatahub-operator-controller-manager-7558c48c44-gnp72" Jul 14 17:37:19.338681 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:19.338661 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/3f0ef414-c4bb-492c-98d8-9bc1f07a9b6f-webhook-cert\") pod \"opendatahub-operator-controller-manager-7558c48c44-gnp72\" (UID: \"3f0ef414-c4bb-492c-98d8-9bc1f07a9b6f\") " pod="opendatahub/opendatahub-operator-controller-manager-7558c48c44-gnp72" Jul 14 17:37:19.338785 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:19.338765 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/3f0ef414-c4bb-492c-98d8-9bc1f07a9b6f-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7558c48c44-gnp72\" (UID: \"3f0ef414-c4bb-492c-98d8-9bc1f07a9b6f\") " pod="opendatahub/opendatahub-operator-controller-manager-7558c48c44-gnp72" Jul 14 17:37:19.363334 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:19.363316 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kbw2n\" (UniqueName: \"kubernetes.io/projected/3f0ef414-c4bb-492c-98d8-9bc1f07a9b6f-kube-api-access-kbw2n\") pod \"opendatahub-operator-controller-manager-7558c48c44-gnp72\" (UID: \"3f0ef414-c4bb-492c-98d8-9bc1f07a9b6f\") " pod="opendatahub/opendatahub-operator-controller-manager-7558c48c44-gnp72" Jul 14 17:37:19.500068 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:19.499980 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7558c48c44-gnp72" Jul 14 17:37:19.652687 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:19.652597 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7558c48c44-gnp72"] Jul 14 17:37:19.656453 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:37:19.656428 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f0ef414_c4bb_492c_98d8_9bc1f07a9b6f.slice/crio-5a9728e1c3d1ba12ffda3d0f7f899f8ee1b1d0218aef7e098c36bb2ea153e3de WatchSource:0}: Error finding container 5a9728e1c3d1ba12ffda3d0f7f899f8ee1b1d0218aef7e098c36bb2ea153e3de: Status 404 returned error can't find the container with id 5a9728e1c3d1ba12ffda3d0f7f899f8ee1b1d0218aef7e098c36bb2ea153e3de Jul 14 17:37:19.760716 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:19.760651 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7558c48c44-gnp72" event={"ID":"3f0ef414-c4bb-492c-98d8-9bc1f07a9b6f","Type":"ContainerStarted","Data":"5a9728e1c3d1ba12ffda3d0f7f899f8ee1b1d0218aef7e098c36bb2ea153e3de"} Jul 14 17:37:23.785671 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:23.785636 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7558c48c44-gnp72" event={"ID":"3f0ef414-c4bb-492c-98d8-9bc1f07a9b6f","Type":"ContainerStarted","Data":"16036853bfffb5a3a239674e606a2e4c00165eddbf1365a4b8f355ef6c390c9c"} Jul 14 17:37:24.790872 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:24.790832 2562 generic.go:356] "Generic (PLEG): container finished" podID="3f0ef414-c4bb-492c-98d8-9bc1f07a9b6f" containerID="16036853bfffb5a3a239674e606a2e4c00165eddbf1365a4b8f355ef6c390c9c" exitCode=0 Jul 14 17:37:24.791274 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:24.790916 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7558c48c44-gnp72" event={"ID":"3f0ef414-c4bb-492c-98d8-9bc1f07a9b6f","Type":"ContainerDied","Data":"16036853bfffb5a3a239674e606a2e4c00165eddbf1365a4b8f355ef6c390c9c"} Jul 14 17:37:25.797248 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:25.797178 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7558c48c44-gnp72" event={"ID":"3f0ef414-c4bb-492c-98d8-9bc1f07a9b6f","Type":"ContainerStarted","Data":"ea9eaa41bf5a7a8038ebb7c1515f0a17fe998b2c9a4e4569633332298b76d230"} Jul 14 17:37:25.797677 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:25.797316 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7558c48c44-gnp72" Jul 14 17:37:25.835758 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:25.835710 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7558c48c44-gnp72" podStartSLOduration=2.340884801 podStartE2EDuration="6.83569511s" totalImagesPullingTime="3.471961874s" totalInitContainerRuntime="1.022848435s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:37:19 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:37:25.834995319 +0000 UTC m=+294.911452874" watchObservedRunningTime="2026-07-14 17:37:25.83569511 +0000 UTC m=+294.912152650" Jul 14 17:37:31.338785 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:31.338754 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2sbww_c3d00b23-d0d5-4d18-a722-464d7b114745/ovn-acl-logging/0.log" Jul 14 17:37:31.342356 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:31.342331 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2sbww_c3d00b23-d0d5-4d18-a722-464d7b114745/ovn-acl-logging/0.log" Jul 14 17:37:31.344867 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:31.344844 2562 kubelet.go:1648] "Image garbage collection succeeded" Jul 14 17:37:36.802915 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:36.802884 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7558c48c44-gnp72" Jul 14 17:37:40.752693 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:40.752659 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-fd75767d5-gwn8q"] Jul 14 17:37:40.755957 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:40.755940 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-fd75767d5-gwn8q" Jul 14 17:37:40.772299 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:40.772273 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 14 17:37:40.772299 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:40.772277 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 14 17:37:40.772489 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:40.772466 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 14 17:37:40.784818 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:40.784795 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-fd75767d5-gwn8q"] Jul 14 17:37:40.787782 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:40.787763 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 14 17:37:40.788026 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:40.788011 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-rp55r\"" Jul 14 17:37:40.801003 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:40.800984 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vnwws\" (UniqueName: \"kubernetes.io/projected/ece6bedb-d95d-4355-8108-b4dba7886c3b-kube-api-access-vnwws\") pod \"kube-auth-proxy-fd75767d5-gwn8q\" (UID: \"ece6bedb-d95d-4355-8108-b4dba7886c3b\") " pod="openshift-ingress/kube-auth-proxy-fd75767d5-gwn8q" Jul 14 17:37:40.801090 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:40.801014 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ece6bedb-d95d-4355-8108-b4dba7886c3b-tmp\") pod \"kube-auth-proxy-fd75767d5-gwn8q\" (UID: \"ece6bedb-d95d-4355-8108-b4dba7886c3b\") " pod="openshift-ingress/kube-auth-proxy-fd75767d5-gwn8q" Jul 14 17:37:40.801090 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:40.801047 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ece6bedb-d95d-4355-8108-b4dba7886c3b-tls-certs\") pod \"kube-auth-proxy-fd75767d5-gwn8q\" (UID: \"ece6bedb-d95d-4355-8108-b4dba7886c3b\") " pod="openshift-ingress/kube-auth-proxy-fd75767d5-gwn8q" Jul 14 17:37:40.902239 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:40.902208 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ece6bedb-d95d-4355-8108-b4dba7886c3b-tls-certs\") pod \"kube-auth-proxy-fd75767d5-gwn8q\" (UID: \"ece6bedb-d95d-4355-8108-b4dba7886c3b\") " pod="openshift-ingress/kube-auth-proxy-fd75767d5-gwn8q" Jul 14 17:37:40.902405 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:40.902289 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vnwws\" (UniqueName: \"kubernetes.io/projected/ece6bedb-d95d-4355-8108-b4dba7886c3b-kube-api-access-vnwws\") pod \"kube-auth-proxy-fd75767d5-gwn8q\" (UID: \"ece6bedb-d95d-4355-8108-b4dba7886c3b\") " pod="openshift-ingress/kube-auth-proxy-fd75767d5-gwn8q" Jul 14 17:37:40.902405 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:40.902314 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ece6bedb-d95d-4355-8108-b4dba7886c3b-tmp\") pod \"kube-auth-proxy-fd75767d5-gwn8q\" (UID: \"ece6bedb-d95d-4355-8108-b4dba7886c3b\") " pod="openshift-ingress/kube-auth-proxy-fd75767d5-gwn8q" Jul 14 17:37:40.904656 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:40.904634 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ece6bedb-d95d-4355-8108-b4dba7886c3b-tmp\") pod \"kube-auth-proxy-fd75767d5-gwn8q\" (UID: \"ece6bedb-d95d-4355-8108-b4dba7886c3b\") " pod="openshift-ingress/kube-auth-proxy-fd75767d5-gwn8q" Jul 14 17:37:40.904776 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:40.904758 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ece6bedb-d95d-4355-8108-b4dba7886c3b-tls-certs\") pod \"kube-auth-proxy-fd75767d5-gwn8q\" (UID: \"ece6bedb-d95d-4355-8108-b4dba7886c3b\") " pod="openshift-ingress/kube-auth-proxy-fd75767d5-gwn8q" Jul 14 17:37:40.913019 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:40.912997 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vnwws\" (UniqueName: \"kubernetes.io/projected/ece6bedb-d95d-4355-8108-b4dba7886c3b-kube-api-access-vnwws\") pod \"kube-auth-proxy-fd75767d5-gwn8q\" (UID: \"ece6bedb-d95d-4355-8108-b4dba7886c3b\") " pod="openshift-ingress/kube-auth-proxy-fd75767d5-gwn8q" Jul 14 17:37:41.065831 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:41.065753 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-fd75767d5-gwn8q" Jul 14 17:37:41.207228 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:41.207178 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-fd75767d5-gwn8q"] Jul 14 17:37:41.212024 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:37:41.211997 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podece6bedb_d95d_4355_8108_b4dba7886c3b.slice/crio-3eec44bbed63a93083aff53bac45abfa6127d286877208887e0a72364f5bdd24 WatchSource:0}: Error finding container 3eec44bbed63a93083aff53bac45abfa6127d286877208887e0a72364f5bdd24: Status 404 returned error can't find the container with id 3eec44bbed63a93083aff53bac45abfa6127d286877208887e0a72364f5bdd24 Jul 14 17:37:41.214312 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:41.214291 2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 14 17:37:41.859088 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:41.859051 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-fd75767d5-gwn8q" event={"ID":"ece6bedb-d95d-4355-8108-b4dba7886c3b","Type":"ContainerStarted","Data":"3eec44bbed63a93083aff53bac45abfa6127d286877208887e0a72364f5bdd24"} Jul 14 17:37:45.876468 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:45.876432 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-fd75767d5-gwn8q" event={"ID":"ece6bedb-d95d-4355-8108-b4dba7886c3b","Type":"ContainerStarted","Data":"f0127f95451fb1fdeb61fcaa580229f9f8e16f91ea0a38331523c05ad626097d"} Jul 14 17:37:45.906969 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:45.906920 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-fd75767d5-gwn8q" podStartSLOduration=1.9653522159999999 podStartE2EDuration="5.906902545s" totalImagesPullingTime="3.941550329s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:37:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:37:45.906732042 +0000 UTC m=+314.983189591" watchObservedRunningTime="2026-07-14 17:37:45.906902545 +0000 UTC m=+314.983360062" Jul 14 17:37:48.578718 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:48.578687 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46rt48"] Jul 14 17:37:48.581828 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:48.581812 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46rt48" Jul 14 17:37:48.587137 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:48.587116 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 14 17:37:48.587270 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:48.587175 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-dzmd2\"" Jul 14 17:37:48.624050 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:48.624028 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46rt48"] Jul 14 17:37:48.670393 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:48.670362 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1a4c56d5-9623-4732-9bbd-d6d539fe8049-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46rt48\" (UID: \"1a4c56d5-9623-4732-9bbd-d6d539fe8049\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46rt48" Jul 14 17:37:48.670528 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:48.670400 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1a4c56d5-9623-4732-9bbd-d6d539fe8049-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46rt48\" (UID: \"1a4c56d5-9623-4732-9bbd-d6d539fe8049\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46rt48" Jul 14 17:37:48.670528 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:48.670441 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a4c56d5-9623-4732-9bbd-d6d539fe8049-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46rt48\" (UID: \"1a4c56d5-9623-4732-9bbd-d6d539fe8049\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46rt48" Jul 14 17:37:48.670528 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:48.670507 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1a4c56d5-9623-4732-9bbd-d6d539fe8049-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46rt48\" (UID: \"1a4c56d5-9623-4732-9bbd-d6d539fe8049\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46rt48" Jul 14 17:37:48.670676 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:48.670554 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1a4c56d5-9623-4732-9bbd-d6d539fe8049-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46rt48\" (UID: \"1a4c56d5-9623-4732-9bbd-d6d539fe8049\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46rt48" Jul 14 17:37:48.670676 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:48.670589 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jjbgt\" (UniqueName: \"kubernetes.io/projected/1a4c56d5-9623-4732-9bbd-d6d539fe8049-kube-api-access-jjbgt\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46rt48\" (UID: \"1a4c56d5-9623-4732-9bbd-d6d539fe8049\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46rt48" Jul 14 17:37:48.670754 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:48.670693 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1a4c56d5-9623-4732-9bbd-d6d539fe8049-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46rt48\" (UID: \"1a4c56d5-9623-4732-9bbd-d6d539fe8049\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46rt48" Jul 14 17:37:48.670795 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:48.670774 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1a4c56d5-9623-4732-9bbd-d6d539fe8049-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46rt48\" (UID: \"1a4c56d5-9623-4732-9bbd-d6d539fe8049\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46rt48" Jul 14 17:37:48.670837 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:48.670811 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a4c56d5-9623-4732-9bbd-d6d539fe8049-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46rt48\" (UID: \"1a4c56d5-9623-4732-9bbd-d6d539fe8049\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46rt48" Jul 14 17:37:48.771380 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:48.771352 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1a4c56d5-9623-4732-9bbd-d6d539fe8049-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46rt48\" (UID: \"1a4c56d5-9623-4732-9bbd-d6d539fe8049\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46rt48" Jul 14 17:37:48.771505 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:48.771390 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jjbgt\" (UniqueName: \"kubernetes.io/projected/1a4c56d5-9623-4732-9bbd-d6d539fe8049-kube-api-access-jjbgt\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46rt48\" (UID: \"1a4c56d5-9623-4732-9bbd-d6d539fe8049\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46rt48" Jul 14 17:37:48.771505 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:48.771428 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1a4c56d5-9623-4732-9bbd-d6d539fe8049-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46rt48\" (UID: \"1a4c56d5-9623-4732-9bbd-d6d539fe8049\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46rt48" Jul 14 17:37:48.771505 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:48.771456 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1a4c56d5-9623-4732-9bbd-d6d539fe8049-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46rt48\" (UID: \"1a4c56d5-9623-4732-9bbd-d6d539fe8049\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46rt48" Jul 14 17:37:48.771505 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:48.771475 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a4c56d5-9623-4732-9bbd-d6d539fe8049-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46rt48\" (UID: \"1a4c56d5-9623-4732-9bbd-d6d539fe8049\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46rt48" Jul 14 17:37:48.771692 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:48.771515 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1a4c56d5-9623-4732-9bbd-d6d539fe8049-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46rt48\" (UID: \"1a4c56d5-9623-4732-9bbd-d6d539fe8049\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46rt48" Jul 14 17:37:48.771692 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:48.771537 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1a4c56d5-9623-4732-9bbd-d6d539fe8049-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46rt48\" (UID: \"1a4c56d5-9623-4732-9bbd-d6d539fe8049\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46rt48" Jul 14 17:37:48.771692 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:48.771569 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a4c56d5-9623-4732-9bbd-d6d539fe8049-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46rt48\" (UID: \"1a4c56d5-9623-4732-9bbd-d6d539fe8049\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46rt48" Jul 14 17:37:48.771692 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:48.771609 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1a4c56d5-9623-4732-9bbd-d6d539fe8049-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46rt48\" (UID: \"1a4c56d5-9623-4732-9bbd-d6d539fe8049\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46rt48" Jul 14 17:37:48.771939 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:48.771919 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1a4c56d5-9623-4732-9bbd-d6d539fe8049-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46rt48\" (UID: \"1a4c56d5-9623-4732-9bbd-d6d539fe8049\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46rt48" Jul 14 17:37:48.772017 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:48.771951 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a4c56d5-9623-4732-9bbd-d6d539fe8049-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46rt48\" (UID: \"1a4c56d5-9623-4732-9bbd-d6d539fe8049\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46rt48" Jul 14 17:37:48.772055 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:48.772015 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1a4c56d5-9623-4732-9bbd-d6d539fe8049-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46rt48\" (UID: \"1a4c56d5-9623-4732-9bbd-d6d539fe8049\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46rt48" Jul 14 17:37:48.772097 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:48.772049 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a4c56d5-9623-4732-9bbd-d6d539fe8049-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46rt48\" (UID: \"1a4c56d5-9623-4732-9bbd-d6d539fe8049\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46rt48" Jul 14 17:37:48.772451 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:48.772433 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1a4c56d5-9623-4732-9bbd-d6d539fe8049-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46rt48\" (UID: \"1a4c56d5-9623-4732-9bbd-d6d539fe8049\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46rt48" Jul 14 17:37:48.773699 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:48.773680 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1a4c56d5-9623-4732-9bbd-d6d539fe8049-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46rt48\" (UID: \"1a4c56d5-9623-4732-9bbd-d6d539fe8049\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46rt48" Jul 14 17:37:48.774166 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:48.774144 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1a4c56d5-9623-4732-9bbd-d6d539fe8049-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46rt48\" (UID: \"1a4c56d5-9623-4732-9bbd-d6d539fe8049\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46rt48" Jul 14 17:37:48.785249 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:48.785226 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1a4c56d5-9623-4732-9bbd-d6d539fe8049-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46rt48\" (UID: \"1a4c56d5-9623-4732-9bbd-d6d539fe8049\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46rt48" Jul 14 17:37:48.797357 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:48.797338 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jjbgt\" (UniqueName: \"kubernetes.io/projected/1a4c56d5-9623-4732-9bbd-d6d539fe8049-kube-api-access-jjbgt\") pod \"data-science-gateway-data-science-gateway-class-6576859cf46rt48\" (UID: \"1a4c56d5-9623-4732-9bbd-d6d539fe8049\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46rt48" Jul 14 17:37:48.891166 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:48.891139 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46rt48" Jul 14 17:37:49.082879 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:49.082852 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46rt48"] Jul 14 17:37:49.084131 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:37:49.084091 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a4c56d5_9623_4732_9bbd_d6d539fe8049.slice/crio-d89b96f8bb2985a1b69e40489d7cf0dd30db2d6e0d686e5e811905e4cdc3f003 WatchSource:0}: Error finding container d89b96f8bb2985a1b69e40489d7cf0dd30db2d6e0d686e5e811905e4cdc3f003: Status 404 returned error can't find the container with id d89b96f8bb2985a1b69e40489d7cf0dd30db2d6e0d686e5e811905e4cdc3f003 Jul 14 17:37:49.872448 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:49.872417 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ln9vg"] Jul 14 17:37:49.876921 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:49.876904 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ln9vg" Jul 14 17:37:49.883743 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:49.883720 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 14 17:37:49.883850 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:49.883744 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 14 17:37:49.883850 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:49.883769 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-n8766\"" Jul 14 17:37:49.883850 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:49.883720 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 14 17:37:49.896989 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:49.896958 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46rt48" event={"ID":"1a4c56d5-9623-4732-9bbd-d6d539fe8049","Type":"ContainerStarted","Data":"d89b96f8bb2985a1b69e40489d7cf0dd30db2d6e0d686e5e811905e4cdc3f003"} Jul 14 17:37:49.981797 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:49.981775 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e6e12fb7-7384-4f6d-9f71-8fb407254355-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-ln9vg\" (UID: \"e6e12fb7-7384-4f6d-9f71-8fb407254355\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ln9vg" Jul 14 17:37:49.981895 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:49.981805 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e6e12fb7-7384-4f6d-9f71-8fb407254355-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-ln9vg\" (UID: \"e6e12fb7-7384-4f6d-9f71-8fb407254355\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ln9vg" Jul 14 17:37:49.981895 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:49.981854 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-56w7d\" (UniqueName: \"kubernetes.io/projected/e6e12fb7-7384-4f6d-9f71-8fb407254355-kube-api-access-56w7d\") pod \"istiod-openshift-gateway-94bb8fbfd-ln9vg\" (UID: \"e6e12fb7-7384-4f6d-9f71-8fb407254355\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ln9vg" Jul 14 17:37:49.981969 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:49.981910 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e6e12fb7-7384-4f6d-9f71-8fb407254355-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-ln9vg\" (UID: \"e6e12fb7-7384-4f6d-9f71-8fb407254355\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ln9vg" Jul 14 17:37:49.981969 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:49.981942 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e6e12fb7-7384-4f6d-9f71-8fb407254355-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-ln9vg\" (UID: \"e6e12fb7-7384-4f6d-9f71-8fb407254355\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ln9vg" Jul 14 17:37:49.981969 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:49.981960 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e6e12fb7-7384-4f6d-9f71-8fb407254355-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-ln9vg\" (UID: \"e6e12fb7-7384-4f6d-9f71-8fb407254355\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ln9vg" Jul 14 17:37:49.982053 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:49.981983 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e6e12fb7-7384-4f6d-9f71-8fb407254355-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-ln9vg\" (UID: \"e6e12fb7-7384-4f6d-9f71-8fb407254355\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ln9vg" Jul 14 17:37:50.008255 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:50.008181 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ln9vg"] Jul 14 17:37:50.082921 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:50.082894 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-56w7d\" (UniqueName: \"kubernetes.io/projected/e6e12fb7-7384-4f6d-9f71-8fb407254355-kube-api-access-56w7d\") pod \"istiod-openshift-gateway-94bb8fbfd-ln9vg\" (UID: \"e6e12fb7-7384-4f6d-9f71-8fb407254355\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ln9vg" Jul 14 17:37:50.083031 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:50.082930 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e6e12fb7-7384-4f6d-9f71-8fb407254355-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-ln9vg\" (UID: \"e6e12fb7-7384-4f6d-9f71-8fb407254355\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ln9vg" Jul 14 17:37:50.083031 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:50.082952 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e6e12fb7-7384-4f6d-9f71-8fb407254355-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-ln9vg\" (UID: \"e6e12fb7-7384-4f6d-9f71-8fb407254355\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ln9vg" Jul 14 17:37:50.083031 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:50.082967 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e6e12fb7-7384-4f6d-9f71-8fb407254355-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-ln9vg\" (UID: \"e6e12fb7-7384-4f6d-9f71-8fb407254355\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ln9vg" Jul 14 17:37:50.083031 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:50.082987 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e6e12fb7-7384-4f6d-9f71-8fb407254355-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-ln9vg\" (UID: \"e6e12fb7-7384-4f6d-9f71-8fb407254355\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ln9vg" Jul 14 17:37:50.083245 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:50.083032 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e6e12fb7-7384-4f6d-9f71-8fb407254355-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-ln9vg\" (UID: \"e6e12fb7-7384-4f6d-9f71-8fb407254355\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ln9vg" Jul 14 17:37:50.083245 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:50.083048 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e6e12fb7-7384-4f6d-9f71-8fb407254355-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-ln9vg\" (UID: \"e6e12fb7-7384-4f6d-9f71-8fb407254355\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ln9vg" Jul 14 17:37:50.083964 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:50.083936 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e6e12fb7-7384-4f6d-9f71-8fb407254355-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-ln9vg\" (UID: \"e6e12fb7-7384-4f6d-9f71-8fb407254355\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ln9vg" Jul 14 17:37:50.085472 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:50.085443 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e6e12fb7-7384-4f6d-9f71-8fb407254355-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-ln9vg\" (UID: \"e6e12fb7-7384-4f6d-9f71-8fb407254355\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ln9vg" Jul 14 17:37:50.085472 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:50.085461 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e6e12fb7-7384-4f6d-9f71-8fb407254355-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-ln9vg\" (UID: \"e6e12fb7-7384-4f6d-9f71-8fb407254355\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ln9vg" Jul 14 17:37:50.085581 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:50.085485 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e6e12fb7-7384-4f6d-9f71-8fb407254355-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-ln9vg\" (UID: \"e6e12fb7-7384-4f6d-9f71-8fb407254355\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ln9vg" Jul 14 17:37:50.085581 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:50.085566 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e6e12fb7-7384-4f6d-9f71-8fb407254355-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-ln9vg\" (UID: \"e6e12fb7-7384-4f6d-9f71-8fb407254355\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ln9vg" Jul 14 17:37:50.100762 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:50.100741 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-56w7d\" (UniqueName: \"kubernetes.io/projected/e6e12fb7-7384-4f6d-9f71-8fb407254355-kube-api-access-56w7d\") pod \"istiod-openshift-gateway-94bb8fbfd-ln9vg\" (UID: \"e6e12fb7-7384-4f6d-9f71-8fb407254355\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ln9vg" Jul 14 17:37:50.101183 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:50.101166 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e6e12fb7-7384-4f6d-9f71-8fb407254355-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-ln9vg\" (UID: \"e6e12fb7-7384-4f6d-9f71-8fb407254355\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ln9vg" Jul 14 17:37:50.186393 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:50.186313 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ln9vg" Jul 14 17:37:50.382302 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:50.382263 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ln9vg"] Jul 14 17:37:50.385856 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:37:50.385829 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6e12fb7_7384_4f6d_9f71_8fb407254355.slice/crio-ce128b9874550e6857caaf921ea04224516d899030e8245ca52da81e1b2974e2 WatchSource:0}: Error finding container ce128b9874550e6857caaf921ea04224516d899030e8245ca52da81e1b2974e2: Status 404 returned error can't find the container with id ce128b9874550e6857caaf921ea04224516d899030e8245ca52da81e1b2974e2 Jul 14 17:37:50.904146 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:50.904095 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ln9vg" event={"ID":"e6e12fb7-7384-4f6d-9f71-8fb407254355","Type":"ContainerStarted","Data":"ce128b9874550e6857caaf921ea04224516d899030e8245ca52da81e1b2974e2"} Jul 14 17:37:52.286644 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:52.286609 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-8448dcd647-77wvx"] Jul 14 17:37:52.295114 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:52.295086 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-8448dcd647-77wvx" Jul 14 17:37:52.305843 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:52.305823 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 14 17:37:52.306943 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:52.306925 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 14 17:37:52.358440 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:52.358413 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 14 17:37:52.360965 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:52.360937 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-8448dcd647-77wvx"] Jul 14 17:37:52.372639 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:52.372613 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 14 17:37:52.377892 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:52.377868 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-kbnwl\"" Jul 14 17:37:52.388685 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:52.388662 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 14 17:37:52.405058 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:52.405032 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/d1441010-f67e-4b0a-b81e-ef383d4d05a8-metrics-cert\") pod \"lws-controller-manager-8448dcd647-77wvx\" (UID: \"d1441010-f67e-4b0a-b81e-ef383d4d05a8\") " pod="openshift-lws-operator/lws-controller-manager-8448dcd647-77wvx" Jul 14 17:37:52.405207 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:52.405166 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d1441010-f67e-4b0a-b81e-ef383d4d05a8-cert\") pod \"lws-controller-manager-8448dcd647-77wvx\" (UID: \"d1441010-f67e-4b0a-b81e-ef383d4d05a8\") " pod="openshift-lws-operator/lws-controller-manager-8448dcd647-77wvx" Jul 14 17:37:52.405293 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:52.405271 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sgjvw\" (UniqueName: \"kubernetes.io/projected/d1441010-f67e-4b0a-b81e-ef383d4d05a8-kube-api-access-sgjvw\") pod \"lws-controller-manager-8448dcd647-77wvx\" (UID: \"d1441010-f67e-4b0a-b81e-ef383d4d05a8\") " pod="openshift-lws-operator/lws-controller-manager-8448dcd647-77wvx" Jul 14 17:37:52.405357 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:52.405326 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/d1441010-f67e-4b0a-b81e-ef383d4d05a8-manager-config\") pod \"lws-controller-manager-8448dcd647-77wvx\" (UID: \"d1441010-f67e-4b0a-b81e-ef383d4d05a8\") " pod="openshift-lws-operator/lws-controller-manager-8448dcd647-77wvx" Jul 14 17:37:52.505874 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:52.505836 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d1441010-f67e-4b0a-b81e-ef383d4d05a8-cert\") pod \"lws-controller-manager-8448dcd647-77wvx\" (UID: \"d1441010-f67e-4b0a-b81e-ef383d4d05a8\") " pod="openshift-lws-operator/lws-controller-manager-8448dcd647-77wvx" Jul 14 17:37:52.506047 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:52.505916 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sgjvw\" (UniqueName: \"kubernetes.io/projected/d1441010-f67e-4b0a-b81e-ef383d4d05a8-kube-api-access-sgjvw\") pod \"lws-controller-manager-8448dcd647-77wvx\" (UID: \"d1441010-f67e-4b0a-b81e-ef383d4d05a8\") " pod="openshift-lws-operator/lws-controller-manager-8448dcd647-77wvx" Jul 14 17:37:52.506047 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:52.505953 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/d1441010-f67e-4b0a-b81e-ef383d4d05a8-manager-config\") pod \"lws-controller-manager-8448dcd647-77wvx\" (UID: \"d1441010-f67e-4b0a-b81e-ef383d4d05a8\") " pod="openshift-lws-operator/lws-controller-manager-8448dcd647-77wvx" Jul 14 17:37:52.506047 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:52.505995 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/d1441010-f67e-4b0a-b81e-ef383d4d05a8-metrics-cert\") pod \"lws-controller-manager-8448dcd647-77wvx\" (UID: \"d1441010-f67e-4b0a-b81e-ef383d4d05a8\") " pod="openshift-lws-operator/lws-controller-manager-8448dcd647-77wvx" Jul 14 17:37:52.506751 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:52.506721 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/d1441010-f67e-4b0a-b81e-ef383d4d05a8-manager-config\") pod \"lws-controller-manager-8448dcd647-77wvx\" (UID: \"d1441010-f67e-4b0a-b81e-ef383d4d05a8\") " pod="openshift-lws-operator/lws-controller-manager-8448dcd647-77wvx" Jul 14 17:37:52.508820 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:52.508798 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d1441010-f67e-4b0a-b81e-ef383d4d05a8-cert\") pod \"lws-controller-manager-8448dcd647-77wvx\" (UID: \"d1441010-f67e-4b0a-b81e-ef383d4d05a8\") " pod="openshift-lws-operator/lws-controller-manager-8448dcd647-77wvx" Jul 14 17:37:52.508925 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:52.508864 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/d1441010-f67e-4b0a-b81e-ef383d4d05a8-metrics-cert\") pod \"lws-controller-manager-8448dcd647-77wvx\" (UID: \"d1441010-f67e-4b0a-b81e-ef383d4d05a8\") " pod="openshift-lws-operator/lws-controller-manager-8448dcd647-77wvx" Jul 14 17:37:52.541584 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:52.541477 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sgjvw\" (UniqueName: \"kubernetes.io/projected/d1441010-f67e-4b0a-b81e-ef383d4d05a8-kube-api-access-sgjvw\") pod \"lws-controller-manager-8448dcd647-77wvx\" (UID: \"d1441010-f67e-4b0a-b81e-ef383d4d05a8\") " pod="openshift-lws-operator/lws-controller-manager-8448dcd647-77wvx" Jul 14 17:37:52.607138 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:52.607105 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-8448dcd647-77wvx" Jul 14 17:37:53.262349 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:53.262304 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 14 17:37:53.262483 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:53.262397 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 14 17:37:53.262483 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:53.262442 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 14 17:37:53.266294 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:53.266252 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 14 17:37:53.266379 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:53.266326 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 14 17:37:53.424230 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:53.424177 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-8448dcd647-77wvx"] Jul 14 17:37:53.427992 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:37:53.427965 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1441010_f67e_4b0a_b81e_ef383d4d05a8.slice/crio-d88835e478ac7e44d70a907e2a60499e99ff004acbd02727a12c550b0cc3de76 WatchSource:0}: Error finding container d88835e478ac7e44d70a907e2a60499e99ff004acbd02727a12c550b0cc3de76: Status 404 returned error can't find the container with id d88835e478ac7e44d70a907e2a60499e99ff004acbd02727a12c550b0cc3de76 Jul 14 17:37:53.918268 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:53.918222 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-8448dcd647-77wvx" event={"ID":"d1441010-f67e-4b0a-b81e-ef383d4d05a8","Type":"ContainerStarted","Data":"d88835e478ac7e44d70a907e2a60499e99ff004acbd02727a12c550b0cc3de76"} Jul 14 17:37:53.919939 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:53.919912 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ln9vg" event={"ID":"e6e12fb7-7384-4f6d-9f71-8fb407254355","Type":"ContainerStarted","Data":"7860d4666c20b306729c06f4b487a0696fd4dc829e34881cf1543b208b4ec629"} Jul 14 17:37:53.920097 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:53.919965 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ln9vg" Jul 14 17:37:53.921417 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:53.921397 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46rt48" event={"ID":"1a4c56d5-9623-4732-9bbd-d6d539fe8049","Type":"ContainerStarted","Data":"d0bd2b81a7e3d541096fca8efca9e23aea9e72f1683fa7ec3e9674dc0fd189e4"} Jul 14 17:37:54.007141 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:54.007093 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ln9vg" podStartSLOduration=2.129588144 podStartE2EDuration="5.007077924s" totalImagesPullingTime="2.87748978s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:37:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:37:54.00540929 +0000 UTC m=+323.081866849" watchObservedRunningTime="2026-07-14 17:37:54.007077924 +0000 UTC m=+323.083535464" Jul 14 17:37:54.060819 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:54.060761 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46rt48" podStartSLOduration=1.884936731 podStartE2EDuration="6.060737072s" totalImagesPullingTime="4.175800341s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:37:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:37:54.058151229 +0000 UTC m=+323.134608803" watchObservedRunningTime="2026-07-14 17:37:54.060737072 +0000 UTC m=+323.137194611" Jul 14 17:37:54.891797 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:54.891767 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46rt48" Jul 14 17:37:54.897415 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:54.897389 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46rt48" Jul 14 17:37:54.927896 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:54.927832 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46rt48" Jul 14 17:37:54.928675 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:54.928646 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf46rt48" Jul 14 17:37:54.929480 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:54.929459 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ln9vg" Jul 14 17:37:55.941895 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:55.941780 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-8448dcd647-77wvx" event={"ID":"d1441010-f67e-4b0a-b81e-ef383d4d05a8","Type":"ContainerStarted","Data":"357e6d07b9d1ffc4f0f8258f5682bc98ccbeb19461a9a7eff7dc14a11fa2925d"} Jul 14 17:37:55.942762 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:55.942727 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-8448dcd647-77wvx" Jul 14 17:37:55.980514 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:37:55.980461 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-8448dcd647-77wvx" podStartSLOduration=1.5761374049999999 podStartE2EDuration="3.980445115s" totalImagesPullingTime="2.40430771s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:37:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:37:55.977408484 +0000 UTC m=+325.053866024" watchObservedRunningTime="2026-07-14 17:37:55.980445115 +0000 UTC m=+325.056902654" Jul 14 17:38:07.951432 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:38:07.951395 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-8448dcd647-77wvx" Jul 14 17:38:57.603814 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:38:57.603781 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7dcb8b898d-zz5c2"] Jul 14 17:39:22.623902 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:39:22.623849 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7dcb8b898d-zz5c2" podUID="9139bbbf-3fd7-48de-9172-9b3551b89b44" containerName="console" containerID="cri-o://b869d5b2a89b372cb7b39f0475c6f2d6d044bdfe7f4b99c1a0b3ad4156b897b8" gracePeriod=15 Jul 14 17:39:22.869049 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:39:22.869024 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7dcb8b898d-zz5c2" Jul 14 17:39:22.979375 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:39:22.979347 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9139bbbf-3fd7-48de-9172-9b3551b89b44-trusted-ca-bundle\") pod \"9139bbbf-3fd7-48de-9172-9b3551b89b44\" (UID: \"9139bbbf-3fd7-48de-9172-9b3551b89b44\") " Jul 14 17:39:22.979514 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:39:22.979384 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9139bbbf-3fd7-48de-9172-9b3551b89b44-console-config\") pod \"9139bbbf-3fd7-48de-9172-9b3551b89b44\" (UID: \"9139bbbf-3fd7-48de-9172-9b3551b89b44\") " Jul 14 17:39:22.979514 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:39:22.979410 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9139bbbf-3fd7-48de-9172-9b3551b89b44-service-ca\") pod \"9139bbbf-3fd7-48de-9172-9b3551b89b44\" (UID: \"9139bbbf-3fd7-48de-9172-9b3551b89b44\") " Jul 14 17:39:22.979514 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:39:22.979439 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9139bbbf-3fd7-48de-9172-9b3551b89b44-oauth-serving-cert\") pod \"9139bbbf-3fd7-48de-9172-9b3551b89b44\" (UID: \"9139bbbf-3fd7-48de-9172-9b3551b89b44\") " Jul 14 17:39:22.979667 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:39:22.979561 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9139bbbf-3fd7-48de-9172-9b3551b89b44-console-oauth-config\") pod \"9139bbbf-3fd7-48de-9172-9b3551b89b44\" (UID: \"9139bbbf-3fd7-48de-9172-9b3551b89b44\") " Jul 14 17:39:22.979667 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:39:22.979610 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-27hf6\" (UniqueName: \"kubernetes.io/projected/9139bbbf-3fd7-48de-9172-9b3551b89b44-kube-api-access-27hf6\") pod \"9139bbbf-3fd7-48de-9172-9b3551b89b44\" (UID: \"9139bbbf-3fd7-48de-9172-9b3551b89b44\") " Jul 14 17:39:22.979667 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:39:22.979631 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9139bbbf-3fd7-48de-9172-9b3551b89b44-console-serving-cert\") pod \"9139bbbf-3fd7-48de-9172-9b3551b89b44\" (UID: \"9139bbbf-3fd7-48de-9172-9b3551b89b44\") " Jul 14 17:39:22.979825 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:39:22.979763 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9139bbbf-3fd7-48de-9172-9b3551b89b44-service-ca" (OuterVolumeSpecName: "service-ca") pod "9139bbbf-3fd7-48de-9172-9b3551b89b44" (UID: "9139bbbf-3fd7-48de-9172-9b3551b89b44"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 17:39:22.979879 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:39:22.979826 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9139bbbf-3fd7-48de-9172-9b3551b89b44-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "9139bbbf-3fd7-48de-9172-9b3551b89b44" (UID: "9139bbbf-3fd7-48de-9172-9b3551b89b44"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 17:39:22.979879 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:39:22.979832 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9139bbbf-3fd7-48de-9172-9b3551b89b44-console-config" (OuterVolumeSpecName: "console-config") pod "9139bbbf-3fd7-48de-9172-9b3551b89b44" (UID: "9139bbbf-3fd7-48de-9172-9b3551b89b44"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 17:39:22.979879 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:39:22.979844 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9139bbbf-3fd7-48de-9172-9b3551b89b44-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "9139bbbf-3fd7-48de-9172-9b3551b89b44" (UID: "9139bbbf-3fd7-48de-9172-9b3551b89b44"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 17:39:22.980017 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:39:22.979940 2562 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9139bbbf-3fd7-48de-9172-9b3551b89b44-trusted-ca-bundle\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 17:39:22.980017 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:39:22.979960 2562 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9139bbbf-3fd7-48de-9172-9b3551b89b44-console-config\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 17:39:22.980017 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:39:22.979974 2562 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9139bbbf-3fd7-48de-9172-9b3551b89b44-service-ca\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 17:39:22.980017 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:39:22.979986 2562 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9139bbbf-3fd7-48de-9172-9b3551b89b44-oauth-serving-cert\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 17:39:22.981784 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:39:22.981761 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9139bbbf-3fd7-48de-9172-9b3551b89b44-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "9139bbbf-3fd7-48de-9172-9b3551b89b44" (UID: "9139bbbf-3fd7-48de-9172-9b3551b89b44"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 17:39:22.981878 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:39:22.981781 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9139bbbf-3fd7-48de-9172-9b3551b89b44-kube-api-access-27hf6" (OuterVolumeSpecName: "kube-api-access-27hf6") pod "9139bbbf-3fd7-48de-9172-9b3551b89b44" (UID: "9139bbbf-3fd7-48de-9172-9b3551b89b44"). InnerVolumeSpecName "kube-api-access-27hf6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 17:39:22.981878 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:39:22.981788 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9139bbbf-3fd7-48de-9172-9b3551b89b44-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "9139bbbf-3fd7-48de-9172-9b3551b89b44" (UID: "9139bbbf-3fd7-48de-9172-9b3551b89b44"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 14 17:39:23.080947 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:39:23.080922 2562 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9139bbbf-3fd7-48de-9172-9b3551b89b44-console-oauth-config\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 17:39:23.080947 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:39:23.080944 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-27hf6\" (UniqueName: \"kubernetes.io/projected/9139bbbf-3fd7-48de-9172-9b3551b89b44-kube-api-access-27hf6\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 17:39:23.081083 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:39:23.080954 2562 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9139bbbf-3fd7-48de-9172-9b3551b89b44-console-serving-cert\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 17:39:23.323318 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:39:23.323242 2562 generic.go:356] "Generic (PLEG): container finished" podID="9139bbbf-3fd7-48de-9172-9b3551b89b44" containerID="b869d5b2a89b372cb7b39f0475c6f2d6d044bdfe7f4b99c1a0b3ad4156b897b8" exitCode=0 Jul 14 17:39:23.323318 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:39:23.323308 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7dcb8b898d-zz5c2" Jul 14 17:39:23.323318 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:39:23.323308 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7dcb8b898d-zz5c2" event={"ID":"9139bbbf-3fd7-48de-9172-9b3551b89b44","Type":"ContainerDied","Data":"b869d5b2a89b372cb7b39f0475c6f2d6d044bdfe7f4b99c1a0b3ad4156b897b8"} Jul 14 17:39:23.323496 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:39:23.323385 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7dcb8b898d-zz5c2" event={"ID":"9139bbbf-3fd7-48de-9172-9b3551b89b44","Type":"ContainerDied","Data":"290c69b7b3238152823211f8e7e04fcbd22be2aedf4f13eccf258d071fe59058"} Jul 14 17:39:23.323496 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:39:23.323413 2562 scope.go:117] "RemoveContainer" containerID="b869d5b2a89b372cb7b39f0475c6f2d6d044bdfe7f4b99c1a0b3ad4156b897b8" Jul 14 17:39:23.331920 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:39:23.331905 2562 scope.go:117] "RemoveContainer" containerID="b869d5b2a89b372cb7b39f0475c6f2d6d044bdfe7f4b99c1a0b3ad4156b897b8" Jul 14 17:39:23.332156 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:39:23.332134 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b869d5b2a89b372cb7b39f0475c6f2d6d044bdfe7f4b99c1a0b3ad4156b897b8\": container with ID starting with b869d5b2a89b372cb7b39f0475c6f2d6d044bdfe7f4b99c1a0b3ad4156b897b8 not found: ID does not exist" containerID="b869d5b2a89b372cb7b39f0475c6f2d6d044bdfe7f4b99c1a0b3ad4156b897b8" Jul 14 17:39:23.332221 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:39:23.332167 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b869d5b2a89b372cb7b39f0475c6f2d6d044bdfe7f4b99c1a0b3ad4156b897b8"} err="failed to get container status \"b869d5b2a89b372cb7b39f0475c6f2d6d044bdfe7f4b99c1a0b3ad4156b897b8\": rpc error: code = NotFound desc = could not find container \"b869d5b2a89b372cb7b39f0475c6f2d6d044bdfe7f4b99c1a0b3ad4156b897b8\": container with ID starting with b869d5b2a89b372cb7b39f0475c6f2d6d044bdfe7f4b99c1a0b3ad4156b897b8 not found: ID does not exist" Jul 14 17:39:23.367336 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:39:23.367315 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7dcb8b898d-zz5c2"] Jul 14 17:39:23.372606 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:39:23.372587 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7dcb8b898d-zz5c2"] Jul 14 17:39:23.518939 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:39:23.518911 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9139bbbf-3fd7-48de-9172-9b3551b89b44" path="/var/lib/kubelet/pods/9139bbbf-3fd7-48de-9172-9b3551b89b44/volumes" Jul 14 17:42:31.367502 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:42:31.367468 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2sbww_c3d00b23-d0d5-4d18-a722-464d7b114745/ovn-acl-logging/0.log" Jul 14 17:42:31.370445 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:42:31.370415 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2sbww_c3d00b23-d0d5-4d18-a722-464d7b114745/ovn-acl-logging/0.log" Jul 14 17:47:31.394419 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:47:31.394331 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2sbww_c3d00b23-d0d5-4d18-a722-464d7b114745/ovn-acl-logging/0.log" Jul 14 17:47:31.395667 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:47:31.395642 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2sbww_c3d00b23-d0d5-4d18-a722-464d7b114745/ovn-acl-logging/0.log" Jul 14 17:52:31.420868 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:52:31.420836 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2sbww_c3d00b23-d0d5-4d18-a722-464d7b114745/ovn-acl-logging/0.log" Jul 14 17:52:31.422364 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:52:31.422345 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2sbww_c3d00b23-d0d5-4d18-a722-464d7b114745/ovn-acl-logging/0.log" Jul 14 17:57:25.074863 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:57:25.074831 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-09df0cc0-openshift-default-647f78d5b5-5ts4d"] Jul 14 17:57:25.075283 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:57:25.075202 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9139bbbf-3fd7-48de-9172-9b3551b89b44" containerName="console" Jul 14 17:57:25.075283 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:57:25.075215 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="9139bbbf-3fd7-48de-9172-9b3551b89b44" containerName="console" Jul 14 17:57:25.075283 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:57:25.075269 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="9139bbbf-3fd7-48de-9172-9b3551b89b44" containerName="console" Jul 14 17:57:25.078181 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:57:25.078157 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-09df0cc0-openshift-default-647f78d5b5-5ts4d" Jul 14 17:57:25.081549 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:57:25.081530 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-09df0cc0-openshift-default-dockercfg-fvbnq\"" Jul 14 17:57:25.100312 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:57:25.100283 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-09df0cc0-openshift-default-647f78d5b5-5ts4d"] Jul 14 17:57:25.184149 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:57:25.184120 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b147d938-359e-423d-a33b-b9b6fd5269fa-istio-podinfo\") pod \"e2e-ait-09df0cc0-openshift-default-647f78d5b5-5ts4d\" (UID: \"b147d938-359e-423d-a33b-b9b6fd5269fa\") " pod="openshift-ingress/e2e-ait-09df0cc0-openshift-default-647f78d5b5-5ts4d" Jul 14 17:57:25.184318 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:57:25.184168 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b147d938-359e-423d-a33b-b9b6fd5269fa-workload-socket\") pod \"e2e-ait-09df0cc0-openshift-default-647f78d5b5-5ts4d\" (UID: \"b147d938-359e-423d-a33b-b9b6fd5269fa\") " pod="openshift-ingress/e2e-ait-09df0cc0-openshift-default-647f78d5b5-5ts4d" Jul 14 17:57:25.184318 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:57:25.184219 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b147d938-359e-423d-a33b-b9b6fd5269fa-istio-token\") pod \"e2e-ait-09df0cc0-openshift-default-647f78d5b5-5ts4d\" (UID: \"b147d938-359e-423d-a33b-b9b6fd5269fa\") " pod="openshift-ingress/e2e-ait-09df0cc0-openshift-default-647f78d5b5-5ts4d" Jul 14 17:57:25.184318 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:57:25.184266 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b147d938-359e-423d-a33b-b9b6fd5269fa-workload-certs\") pod \"e2e-ait-09df0cc0-openshift-default-647f78d5b5-5ts4d\" (UID: \"b147d938-359e-423d-a33b-b9b6fd5269fa\") " pod="openshift-ingress/e2e-ait-09df0cc0-openshift-default-647f78d5b5-5ts4d" Jul 14 17:57:25.184318 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:57:25.184299 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b147d938-359e-423d-a33b-b9b6fd5269fa-istio-envoy\") pod \"e2e-ait-09df0cc0-openshift-default-647f78d5b5-5ts4d\" (UID: \"b147d938-359e-423d-a33b-b9b6fd5269fa\") " pod="openshift-ingress/e2e-ait-09df0cc0-openshift-default-647f78d5b5-5ts4d" Jul 14 17:57:25.184460 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:57:25.184328 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b147d938-359e-423d-a33b-b9b6fd5269fa-credential-socket\") pod \"e2e-ait-09df0cc0-openshift-default-647f78d5b5-5ts4d\" (UID: \"b147d938-359e-423d-a33b-b9b6fd5269fa\") " pod="openshift-ingress/e2e-ait-09df0cc0-openshift-default-647f78d5b5-5ts4d" Jul 14 17:57:25.184493 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:57:25.184406 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b147d938-359e-423d-a33b-b9b6fd5269fa-istiod-ca-cert\") pod \"e2e-ait-09df0cc0-openshift-default-647f78d5b5-5ts4d\" (UID: \"b147d938-359e-423d-a33b-b9b6fd5269fa\") " pod="openshift-ingress/e2e-ait-09df0cc0-openshift-default-647f78d5b5-5ts4d" Jul 14 17:57:25.184532 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:57:25.184523 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b147d938-359e-423d-a33b-b9b6fd5269fa-istio-data\") pod \"e2e-ait-09df0cc0-openshift-default-647f78d5b5-5ts4d\" (UID: \"b147d938-359e-423d-a33b-b9b6fd5269fa\") " pod="openshift-ingress/e2e-ait-09df0cc0-openshift-default-647f78d5b5-5ts4d" Jul 14 17:57:25.184568 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:57:25.184542 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vm62t\" (UniqueName: \"kubernetes.io/projected/b147d938-359e-423d-a33b-b9b6fd5269fa-kube-api-access-vm62t\") pod \"e2e-ait-09df0cc0-openshift-default-647f78d5b5-5ts4d\" (UID: \"b147d938-359e-423d-a33b-b9b6fd5269fa\") " pod="openshift-ingress/e2e-ait-09df0cc0-openshift-default-647f78d5b5-5ts4d" Jul 14 17:57:25.285050 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:57:25.284983 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b147d938-359e-423d-a33b-b9b6fd5269fa-istio-podinfo\") pod \"e2e-ait-09df0cc0-openshift-default-647f78d5b5-5ts4d\" (UID: \"b147d938-359e-423d-a33b-b9b6fd5269fa\") " pod="openshift-ingress/e2e-ait-09df0cc0-openshift-default-647f78d5b5-5ts4d" Jul 14 17:57:25.285050 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:57:25.285017 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b147d938-359e-423d-a33b-b9b6fd5269fa-workload-socket\") pod \"e2e-ait-09df0cc0-openshift-default-647f78d5b5-5ts4d\" (UID: \"b147d938-359e-423d-a33b-b9b6fd5269fa\") " pod="openshift-ingress/e2e-ait-09df0cc0-openshift-default-647f78d5b5-5ts4d" Jul 14 17:57:25.285050 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:57:25.285033 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b147d938-359e-423d-a33b-b9b6fd5269fa-istio-token\") pod \"e2e-ait-09df0cc0-openshift-default-647f78d5b5-5ts4d\" (UID: \"b147d938-359e-423d-a33b-b9b6fd5269fa\") " pod="openshift-ingress/e2e-ait-09df0cc0-openshift-default-647f78d5b5-5ts4d" Jul 14 17:57:25.285313 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:57:25.285054 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b147d938-359e-423d-a33b-b9b6fd5269fa-workload-certs\") pod \"e2e-ait-09df0cc0-openshift-default-647f78d5b5-5ts4d\" (UID: \"b147d938-359e-423d-a33b-b9b6fd5269fa\") " pod="openshift-ingress/e2e-ait-09df0cc0-openshift-default-647f78d5b5-5ts4d" Jul 14 17:57:25.285313 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:57:25.285109 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b147d938-359e-423d-a33b-b9b6fd5269fa-istio-envoy\") pod \"e2e-ait-09df0cc0-openshift-default-647f78d5b5-5ts4d\" (UID: \"b147d938-359e-423d-a33b-b9b6fd5269fa\") " pod="openshift-ingress/e2e-ait-09df0cc0-openshift-default-647f78d5b5-5ts4d" Jul 14 17:57:25.285313 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:57:25.285154 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b147d938-359e-423d-a33b-b9b6fd5269fa-credential-socket\") pod \"e2e-ait-09df0cc0-openshift-default-647f78d5b5-5ts4d\" (UID: \"b147d938-359e-423d-a33b-b9b6fd5269fa\") " pod="openshift-ingress/e2e-ait-09df0cc0-openshift-default-647f78d5b5-5ts4d" Jul 14 17:57:25.285313 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:57:25.285218 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b147d938-359e-423d-a33b-b9b6fd5269fa-istiod-ca-cert\") pod \"e2e-ait-09df0cc0-openshift-default-647f78d5b5-5ts4d\" (UID: \"b147d938-359e-423d-a33b-b9b6fd5269fa\") " pod="openshift-ingress/e2e-ait-09df0cc0-openshift-default-647f78d5b5-5ts4d" Jul 14 17:57:25.285313 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:57:25.285265 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b147d938-359e-423d-a33b-b9b6fd5269fa-istio-data\") pod \"e2e-ait-09df0cc0-openshift-default-647f78d5b5-5ts4d\" (UID: \"b147d938-359e-423d-a33b-b9b6fd5269fa\") " pod="openshift-ingress/e2e-ait-09df0cc0-openshift-default-647f78d5b5-5ts4d" Jul 14 17:57:25.285561 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:57:25.285355 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vm62t\" (UniqueName: \"kubernetes.io/projected/b147d938-359e-423d-a33b-b9b6fd5269fa-kube-api-access-vm62t\") pod \"e2e-ait-09df0cc0-openshift-default-647f78d5b5-5ts4d\" (UID: \"b147d938-359e-423d-a33b-b9b6fd5269fa\") " pod="openshift-ingress/e2e-ait-09df0cc0-openshift-default-647f78d5b5-5ts4d" Jul 14 17:57:25.285561 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:57:25.285437 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b147d938-359e-423d-a33b-b9b6fd5269fa-workload-certs\") pod \"e2e-ait-09df0cc0-openshift-default-647f78d5b5-5ts4d\" (UID: \"b147d938-359e-423d-a33b-b9b6fd5269fa\") " pod="openshift-ingress/e2e-ait-09df0cc0-openshift-default-647f78d5b5-5ts4d" Jul 14 17:57:25.285652 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:57:25.285559 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b147d938-359e-423d-a33b-b9b6fd5269fa-workload-socket\") pod \"e2e-ait-09df0cc0-openshift-default-647f78d5b5-5ts4d\" (UID: \"b147d938-359e-423d-a33b-b9b6fd5269fa\") " pod="openshift-ingress/e2e-ait-09df0cc0-openshift-default-647f78d5b5-5ts4d" Jul 14 17:57:25.285690 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:57:25.285646 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b147d938-359e-423d-a33b-b9b6fd5269fa-credential-socket\") pod \"e2e-ait-09df0cc0-openshift-default-647f78d5b5-5ts4d\" (UID: \"b147d938-359e-423d-a33b-b9b6fd5269fa\") " pod="openshift-ingress/e2e-ait-09df0cc0-openshift-default-647f78d5b5-5ts4d" Jul 14 17:57:25.285872 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:57:25.285850 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b147d938-359e-423d-a33b-b9b6fd5269fa-istio-data\") pod \"e2e-ait-09df0cc0-openshift-default-647f78d5b5-5ts4d\" (UID: \"b147d938-359e-423d-a33b-b9b6fd5269fa\") " pod="openshift-ingress/e2e-ait-09df0cc0-openshift-default-647f78d5b5-5ts4d" Jul 14 17:57:25.285969 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:57:25.285954 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b147d938-359e-423d-a33b-b9b6fd5269fa-istiod-ca-cert\") pod \"e2e-ait-09df0cc0-openshift-default-647f78d5b5-5ts4d\" (UID: \"b147d938-359e-423d-a33b-b9b6fd5269fa\") " pod="openshift-ingress/e2e-ait-09df0cc0-openshift-default-647f78d5b5-5ts4d" Jul 14 17:57:25.287481 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:57:25.287460 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b147d938-359e-423d-a33b-b9b6fd5269fa-istio-envoy\") pod \"e2e-ait-09df0cc0-openshift-default-647f78d5b5-5ts4d\" (UID: \"b147d938-359e-423d-a33b-b9b6fd5269fa\") " pod="openshift-ingress/e2e-ait-09df0cc0-openshift-default-647f78d5b5-5ts4d" Jul 14 17:57:25.287610 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:57:25.287593 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b147d938-359e-423d-a33b-b9b6fd5269fa-istio-podinfo\") pod \"e2e-ait-09df0cc0-openshift-default-647f78d5b5-5ts4d\" (UID: \"b147d938-359e-423d-a33b-b9b6fd5269fa\") " pod="openshift-ingress/e2e-ait-09df0cc0-openshift-default-647f78d5b5-5ts4d" Jul 14 17:57:25.295424 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:57:25.295396 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vm62t\" (UniqueName: \"kubernetes.io/projected/b147d938-359e-423d-a33b-b9b6fd5269fa-kube-api-access-vm62t\") pod \"e2e-ait-09df0cc0-openshift-default-647f78d5b5-5ts4d\" (UID: \"b147d938-359e-423d-a33b-b9b6fd5269fa\") " pod="openshift-ingress/e2e-ait-09df0cc0-openshift-default-647f78d5b5-5ts4d" Jul 14 17:57:25.296281 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:57:25.296256 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b147d938-359e-423d-a33b-b9b6fd5269fa-istio-token\") pod \"e2e-ait-09df0cc0-openshift-default-647f78d5b5-5ts4d\" (UID: \"b147d938-359e-423d-a33b-b9b6fd5269fa\") " pod="openshift-ingress/e2e-ait-09df0cc0-openshift-default-647f78d5b5-5ts4d" Jul 14 17:57:25.388353 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:57:25.388326 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-09df0cc0-openshift-default-647f78d5b5-5ts4d" Jul 14 17:57:25.530863 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:57:25.530843 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-09df0cc0-openshift-default-647f78d5b5-5ts4d"] Jul 14 17:57:25.533139 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:57:25.533109 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb147d938_359e_423d_a33b_b9b6fd5269fa.slice/crio-4eb76d68e9c46dad587e95cda25935cb1a741ead81b402331e4e422b60105898 WatchSource:0}: Error finding container 4eb76d68e9c46dad587e95cda25935cb1a741ead81b402331e4e422b60105898: Status 404 returned error can't find the container with id 4eb76d68e9c46dad587e95cda25935cb1a741ead81b402331e4e422b60105898 Jul 14 17:57:25.535237 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:57:25.535161 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 14 17:57:25.535323 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:57:25.535249 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 14 17:57:25.535323 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:57:25.535276 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 14 17:57:25.829641 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:57:25.829555 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-09df0cc0-openshift-default-647f78d5b5-5ts4d" event={"ID":"b147d938-359e-423d-a33b-b9b6fd5269fa","Type":"ContainerStarted","Data":"43666d352986abd8a5e747d6a5ec9fbb03a744114bb6bce8f3b645be5af32500"} Jul 14 17:57:25.829641 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:57:25.829604 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-09df0cc0-openshift-default-647f78d5b5-5ts4d" event={"ID":"b147d938-359e-423d-a33b-b9b6fd5269fa","Type":"ContainerStarted","Data":"4eb76d68e9c46dad587e95cda25935cb1a741ead81b402331e4e422b60105898"} Jul 14 17:57:25.858271 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:57:25.858219 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-09df0cc0-openshift-default-647f78d5b5-5ts4d" podStartSLOduration=0.858164196 podStartE2EDuration="858.164196ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:57:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:57:25.857329443 +0000 UTC m=+1494.933787014" watchObservedRunningTime="2026-07-14 17:57:25.858164196 +0000 UTC m=+1494.934621775" Jul 14 17:57:26.389078 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:57:26.389041 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-09df0cc0-openshift-default-647f78d5b5-5ts4d" Jul 14 17:57:26.393797 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:57:26.393776 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-09df0cc0-openshift-default-647f78d5b5-5ts4d" Jul 14 17:57:26.834958 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:57:26.834873 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-09df0cc0-openshift-default-647f78d5b5-5ts4d" Jul 14 17:57:26.835819 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:57:26.835799 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-09df0cc0-openshift-default-647f78d5b5-5ts4d" Jul 14 17:57:31.448058 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:57:31.448033 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2sbww_c3d00b23-d0d5-4d18-a722-464d7b114745/ovn-acl-logging/0.log" Jul 14 17:57:31.450040 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:57:31.450016 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2sbww_c3d00b23-d0d5-4d18-a722-464d7b114745/ovn-acl-logging/0.log" Jul 14 17:57:41.836080 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:57:41.836043 2562 patch_prober.go:28] interesting pod/e2e-ait-09df0cc0-openshift-default-647f78d5b5-5ts4d container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused" start-of-body= Jul 14 17:57:41.836517 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:57:41.836101 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-09df0cc0-openshift-default-647f78d5b5-5ts4d" podUID="b147d938-359e-423d-a33b-b9b6fd5269fa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused" Jul 14 17:57:41.908332 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:57:41.908300 2562 generic.go:356] "Generic (PLEG): container finished" podID="b147d938-359e-423d-a33b-b9b6fd5269fa" containerID="43666d352986abd8a5e747d6a5ec9fbb03a744114bb6bce8f3b645be5af32500" exitCode=0 Jul 14 17:57:41.908507 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:57:41.908377 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-09df0cc0-openshift-default-647f78d5b5-5ts4d" event={"ID":"b147d938-359e-423d-a33b-b9b6fd5269fa","Type":"ContainerDied","Data":"43666d352986abd8a5e747d6a5ec9fbb03a744114bb6bce8f3b645be5af32500"} Jul 14 17:57:41.908835 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:57:41.908816 2562 scope.go:117] "RemoveContainer" containerID="43666d352986abd8a5e747d6a5ec9fbb03a744114bb6bce8f3b645be5af32500" Jul 14 17:57:41.909838 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:57:41.909812 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 14 17:57:41.909925 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:57:41.909877 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 14 17:57:41.909925 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:57:41.909909 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 14 17:57:42.914276 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:57:42.914241 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-09df0cc0-openshift-default-647f78d5b5-5ts4d" event={"ID":"b147d938-359e-423d-a33b-b9b6fd5269fa","Type":"ContainerStarted","Data":"209f1873e2f190f8453f9696c73858c9957badb9d06aee87f5e0bd0e03c730af"} Jul 14 17:57:43.388678 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:57:43.388648 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-09df0cc0-openshift-default-647f78d5b5-5ts4d" Jul 14 17:57:43.388968 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:57:43.388947 2562 patch_prober.go:28] interesting pod/e2e-ait-09df0cc0-openshift-default-647f78d5b5-5ts4d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused" start-of-body= Jul 14 17:57:43.389012 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:57:43.388997 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-09df0cc0-openshift-default-647f78d5b5-5ts4d" podUID="b147d938-359e-423d-a33b-b9b6fd5269fa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused" Jul 14 17:57:44.389337 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:57:44.389304 2562 patch_prober.go:28] interesting pod/e2e-ait-09df0cc0-openshift-default-647f78d5b5-5ts4d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused" start-of-body= Jul 14 17:57:44.389725 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:57:44.389358 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-09df0cc0-openshift-default-647f78d5b5-5ts4d" podUID="b147d938-359e-423d-a33b-b9b6fd5269fa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused" Jul 14 17:57:45.388733 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:57:45.388698 2562 patch_prober.go:28] interesting pod/e2e-ait-09df0cc0-openshift-default-647f78d5b5-5ts4d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused" start-of-body= Jul 14 17:57:45.388908 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:57:45.388782 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-09df0cc0-openshift-default-647f78d5b5-5ts4d" podUID="b147d938-359e-423d-a33b-b9b6fd5269fa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused" Jul 14 17:57:46.389064 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:57:46.389025 2562 patch_prober.go:28] interesting pod/e2e-ait-09df0cc0-openshift-default-647f78d5b5-5ts4d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused" start-of-body= Jul 14 17:57:46.389500 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:57:46.389086 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-09df0cc0-openshift-default-647f78d5b5-5ts4d" podUID="b147d938-359e-423d-a33b-b9b6fd5269fa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused" Jul 14 17:57:47.389321 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:57:47.389285 2562 patch_prober.go:28] interesting pod/e2e-ait-09df0cc0-openshift-default-647f78d5b5-5ts4d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused" start-of-body= Jul 14 17:57:47.389708 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:57:47.389342 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-09df0cc0-openshift-default-647f78d5b5-5ts4d" podUID="b147d938-359e-423d-a33b-b9b6fd5269fa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused" Jul 14 17:57:48.389428 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:57:48.389390 2562 patch_prober.go:28] interesting pod/e2e-ait-09df0cc0-openshift-default-647f78d5b5-5ts4d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused" start-of-body= Jul 14 17:57:48.389895 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:57:48.389456 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-09df0cc0-openshift-default-647f78d5b5-5ts4d" podUID="b147d938-359e-423d-a33b-b9b6fd5269fa" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.27:15021/healthz/ready\": dial tcp 10.134.0.27:15021: connect: connection refused" Jul 14 17:57:49.392438 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:57:49.392409 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-09df0cc0-openshift-default-647f78d5b5-5ts4d" Jul 14 17:57:49.392838 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:57:49.392759 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-09df0cc0-openshift-default-647f78d5b5-5ts4d" Jul 14 17:57:49.393362 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:57:49.393347 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-09df0cc0-openshift-default-647f78d5b5-5ts4d" Jul 14 17:58:06.562638 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:58:06.562556 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-09df0cc0-openshift-default-647f78d5b5-5ts4d"] Jul 14 17:58:06.563147 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:58:06.562739 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-09df0cc0-openshift-default-647f78d5b5-5ts4d" podUID="b147d938-359e-423d-a33b-b9b6fd5269fa" containerName="istio-proxy" containerID="cri-o://209f1873e2f190f8453f9696c73858c9957badb9d06aee87f5e0bd0e03c730af" gracePeriod=30 Jul 14 17:58:07.146352 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:58:07.146311 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-b28b559b-gw-openshift-default-68489c4759-z9m8c"] Jul 14 17:58:07.148759 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:58:07.148738 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-b28b559b-gw-openshift-default-68489c4759-z9m8c" Jul 14 17:58:07.151400 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:58:07.151377 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-b28b559b-gw-openshift-default-dockercfg-qt25p\"" Jul 14 17:58:07.166123 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:58:07.166100 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-b28b559b-gw-openshift-default-68489c4759-z9m8c"] Jul 14 17:58:07.261489 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:58:07.261450 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b8561883-c29f-403d-aaef-2bd9318a9eb3-istiod-ca-cert\") pod \"e2e-migrate-b28b559b-gw-openshift-default-68489c4759-z9m8c\" (UID: \"b8561883-c29f-403d-aaef-2bd9318a9eb3\") " pod="openshift-ingress/e2e-migrate-b28b559b-gw-openshift-default-68489c4759-z9m8c" Jul 14 17:58:07.261676 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:58:07.261574 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b8561883-c29f-403d-aaef-2bd9318a9eb3-istio-token\") pod \"e2e-migrate-b28b559b-gw-openshift-default-68489c4759-z9m8c\" (UID: \"b8561883-c29f-403d-aaef-2bd9318a9eb3\") " pod="openshift-ingress/e2e-migrate-b28b559b-gw-openshift-default-68489c4759-z9m8c" Jul 14 17:58:07.261676 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:58:07.261616 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b8561883-c29f-403d-aaef-2bd9318a9eb3-workload-socket\") pod \"e2e-migrate-b28b559b-gw-openshift-default-68489c4759-z9m8c\" (UID: \"b8561883-c29f-403d-aaef-2bd9318a9eb3\") " pod="openshift-ingress/e2e-migrate-b28b559b-gw-openshift-default-68489c4759-z9m8c" Jul 14 17:58:07.261809 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:58:07.261678 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b8561883-c29f-403d-aaef-2bd9318a9eb3-credential-socket\") pod \"e2e-migrate-b28b559b-gw-openshift-default-68489c4759-z9m8c\" (UID: \"b8561883-c29f-403d-aaef-2bd9318a9eb3\") " pod="openshift-ingress/e2e-migrate-b28b559b-gw-openshift-default-68489c4759-z9m8c" Jul 14 17:58:07.261809 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:58:07.261732 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b8561883-c29f-403d-aaef-2bd9318a9eb3-istio-podinfo\") pod \"e2e-migrate-b28b559b-gw-openshift-default-68489c4759-z9m8c\" (UID: \"b8561883-c29f-403d-aaef-2bd9318a9eb3\") " pod="openshift-ingress/e2e-migrate-b28b559b-gw-openshift-default-68489c4759-z9m8c" Jul 14 17:58:07.261809 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:58:07.261769 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b8561883-c29f-403d-aaef-2bd9318a9eb3-istio-envoy\") pod \"e2e-migrate-b28b559b-gw-openshift-default-68489c4759-z9m8c\" (UID: \"b8561883-c29f-403d-aaef-2bd9318a9eb3\") " pod="openshift-ingress/e2e-migrate-b28b559b-gw-openshift-default-68489c4759-z9m8c" Jul 14 17:58:07.261970 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:58:07.261860 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b8561883-c29f-403d-aaef-2bd9318a9eb3-workload-certs\") pod \"e2e-migrate-b28b559b-gw-openshift-default-68489c4759-z9m8c\" (UID: \"b8561883-c29f-403d-aaef-2bd9318a9eb3\") " pod="openshift-ingress/e2e-migrate-b28b559b-gw-openshift-default-68489c4759-z9m8c" Jul 14 17:58:07.261970 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:58:07.261895 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6pxzk\" (UniqueName: \"kubernetes.io/projected/b8561883-c29f-403d-aaef-2bd9318a9eb3-kube-api-access-6pxzk\") pod \"e2e-migrate-b28b559b-gw-openshift-default-68489c4759-z9m8c\" (UID: \"b8561883-c29f-403d-aaef-2bd9318a9eb3\") " pod="openshift-ingress/e2e-migrate-b28b559b-gw-openshift-default-68489c4759-z9m8c" Jul 14 17:58:07.261970 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:58:07.261926 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b8561883-c29f-403d-aaef-2bd9318a9eb3-istio-data\") pod \"e2e-migrate-b28b559b-gw-openshift-default-68489c4759-z9m8c\" (UID: \"b8561883-c29f-403d-aaef-2bd9318a9eb3\") " pod="openshift-ingress/e2e-migrate-b28b559b-gw-openshift-default-68489c4759-z9m8c" Jul 14 17:58:07.362713 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:58:07.362681 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b8561883-c29f-403d-aaef-2bd9318a9eb3-istio-token\") pod \"e2e-migrate-b28b559b-gw-openshift-default-68489c4759-z9m8c\" (UID: \"b8561883-c29f-403d-aaef-2bd9318a9eb3\") " pod="openshift-ingress/e2e-migrate-b28b559b-gw-openshift-default-68489c4759-z9m8c" Jul 14 17:58:07.362880 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:58:07.362719 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b8561883-c29f-403d-aaef-2bd9318a9eb3-workload-socket\") pod \"e2e-migrate-b28b559b-gw-openshift-default-68489c4759-z9m8c\" (UID: \"b8561883-c29f-403d-aaef-2bd9318a9eb3\") " pod="openshift-ingress/e2e-migrate-b28b559b-gw-openshift-default-68489c4759-z9m8c" Jul 14 17:58:07.362880 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:58:07.362746 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b8561883-c29f-403d-aaef-2bd9318a9eb3-credential-socket\") pod \"e2e-migrate-b28b559b-gw-openshift-default-68489c4759-z9m8c\" (UID: \"b8561883-c29f-403d-aaef-2bd9318a9eb3\") " pod="openshift-ingress/e2e-migrate-b28b559b-gw-openshift-default-68489c4759-z9m8c" Jul 14 17:58:07.362880 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:58:07.362765 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b8561883-c29f-403d-aaef-2bd9318a9eb3-istio-podinfo\") pod \"e2e-migrate-b28b559b-gw-openshift-default-68489c4759-z9m8c\" (UID: \"b8561883-c29f-403d-aaef-2bd9318a9eb3\") " pod="openshift-ingress/e2e-migrate-b28b559b-gw-openshift-default-68489c4759-z9m8c" Jul 14 17:58:07.362880 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:58:07.362780 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b8561883-c29f-403d-aaef-2bd9318a9eb3-istio-envoy\") pod \"e2e-migrate-b28b559b-gw-openshift-default-68489c4759-z9m8c\" (UID: \"b8561883-c29f-403d-aaef-2bd9318a9eb3\") " pod="openshift-ingress/e2e-migrate-b28b559b-gw-openshift-default-68489c4759-z9m8c" Jul 14 17:58:07.362880 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:58:07.362816 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b8561883-c29f-403d-aaef-2bd9318a9eb3-workload-certs\") pod \"e2e-migrate-b28b559b-gw-openshift-default-68489c4759-z9m8c\" (UID: \"b8561883-c29f-403d-aaef-2bd9318a9eb3\") " pod="openshift-ingress/e2e-migrate-b28b559b-gw-openshift-default-68489c4759-z9m8c" Jul 14 17:58:07.362880 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:58:07.362831 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6pxzk\" (UniqueName: \"kubernetes.io/projected/b8561883-c29f-403d-aaef-2bd9318a9eb3-kube-api-access-6pxzk\") pod \"e2e-migrate-b28b559b-gw-openshift-default-68489c4759-z9m8c\" (UID: \"b8561883-c29f-403d-aaef-2bd9318a9eb3\") " pod="openshift-ingress/e2e-migrate-b28b559b-gw-openshift-default-68489c4759-z9m8c" Jul 14 17:58:07.362880 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:58:07.362848 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b8561883-c29f-403d-aaef-2bd9318a9eb3-istio-data\") pod \"e2e-migrate-b28b559b-gw-openshift-default-68489c4759-z9m8c\" (UID: \"b8561883-c29f-403d-aaef-2bd9318a9eb3\") " pod="openshift-ingress/e2e-migrate-b28b559b-gw-openshift-default-68489c4759-z9m8c" Jul 14 17:58:07.362880 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:58:07.362872 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b8561883-c29f-403d-aaef-2bd9318a9eb3-istiod-ca-cert\") pod \"e2e-migrate-b28b559b-gw-openshift-default-68489c4759-z9m8c\" (UID: \"b8561883-c29f-403d-aaef-2bd9318a9eb3\") " pod="openshift-ingress/e2e-migrate-b28b559b-gw-openshift-default-68489c4759-z9m8c" Jul 14 17:58:07.363340 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:58:07.363163 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b8561883-c29f-403d-aaef-2bd9318a9eb3-workload-socket\") pod \"e2e-migrate-b28b559b-gw-openshift-default-68489c4759-z9m8c\" (UID: \"b8561883-c29f-403d-aaef-2bd9318a9eb3\") " pod="openshift-ingress/e2e-migrate-b28b559b-gw-openshift-default-68489c4759-z9m8c" Jul 14 17:58:07.363340 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:58:07.363176 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b8561883-c29f-403d-aaef-2bd9318a9eb3-workload-certs\") pod \"e2e-migrate-b28b559b-gw-openshift-default-68489c4759-z9m8c\" (UID: \"b8561883-c29f-403d-aaef-2bd9318a9eb3\") " pod="openshift-ingress/e2e-migrate-b28b559b-gw-openshift-default-68489c4759-z9m8c" Jul 14 17:58:07.363504 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:58:07.363483 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b8561883-c29f-403d-aaef-2bd9318a9eb3-credential-socket\") pod \"e2e-migrate-b28b559b-gw-openshift-default-68489c4759-z9m8c\" (UID: \"b8561883-c29f-403d-aaef-2bd9318a9eb3\") " pod="openshift-ingress/e2e-migrate-b28b559b-gw-openshift-default-68489c4759-z9m8c" Jul 14 17:58:07.363585 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:58:07.363552 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b8561883-c29f-403d-aaef-2bd9318a9eb3-istiod-ca-cert\") pod \"e2e-migrate-b28b559b-gw-openshift-default-68489c4759-z9m8c\" (UID: \"b8561883-c29f-403d-aaef-2bd9318a9eb3\") " pod="openshift-ingress/e2e-migrate-b28b559b-gw-openshift-default-68489c4759-z9m8c" Jul 14 17:58:07.363821 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:58:07.363800 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b8561883-c29f-403d-aaef-2bd9318a9eb3-istio-data\") pod \"e2e-migrate-b28b559b-gw-openshift-default-68489c4759-z9m8c\" (UID: \"b8561883-c29f-403d-aaef-2bd9318a9eb3\") " pod="openshift-ingress/e2e-migrate-b28b559b-gw-openshift-default-68489c4759-z9m8c" Jul 14 17:58:07.365395 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:58:07.365375 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b8561883-c29f-403d-aaef-2bd9318a9eb3-istio-envoy\") pod \"e2e-migrate-b28b559b-gw-openshift-default-68489c4759-z9m8c\" (UID: \"b8561883-c29f-403d-aaef-2bd9318a9eb3\") " pod="openshift-ingress/e2e-migrate-b28b559b-gw-openshift-default-68489c4759-z9m8c" Jul 14 17:58:07.365484 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:58:07.365409 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b8561883-c29f-403d-aaef-2bd9318a9eb3-istio-podinfo\") pod \"e2e-migrate-b28b559b-gw-openshift-default-68489c4759-z9m8c\" (UID: \"b8561883-c29f-403d-aaef-2bd9318a9eb3\") " pod="openshift-ingress/e2e-migrate-b28b559b-gw-openshift-default-68489c4759-z9m8c" Jul 14 17:58:07.379683 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:58:07.379651 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b8561883-c29f-403d-aaef-2bd9318a9eb3-istio-token\") pod \"e2e-migrate-b28b559b-gw-openshift-default-68489c4759-z9m8c\" (UID: \"b8561883-c29f-403d-aaef-2bd9318a9eb3\") " pod="openshift-ingress/e2e-migrate-b28b559b-gw-openshift-default-68489c4759-z9m8c" Jul 14 17:58:07.382032 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:58:07.382011 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6pxzk\" (UniqueName: \"kubernetes.io/projected/b8561883-c29f-403d-aaef-2bd9318a9eb3-kube-api-access-6pxzk\") pod \"e2e-migrate-b28b559b-gw-openshift-default-68489c4759-z9m8c\" (UID: \"b8561883-c29f-403d-aaef-2bd9318a9eb3\") " pod="openshift-ingress/e2e-migrate-b28b559b-gw-openshift-default-68489c4759-z9m8c" Jul 14 17:58:07.459520 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:58:07.459454 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-b28b559b-gw-openshift-default-68489c4759-z9m8c" Jul 14 17:58:07.619759 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:58:07.619737 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-b28b559b-gw-openshift-default-68489c4759-z9m8c"] Jul 14 17:58:07.622042 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:58:07.622014 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8561883_c29f_403d_aaef_2bd9318a9eb3.slice/crio-297d278c5329a69869a9642fc727d37fb6c9788b910d0077840e8a3892e41fc1 WatchSource:0}: Error finding container 297d278c5329a69869a9642fc727d37fb6c9788b910d0077840e8a3892e41fc1: Status 404 returned error can't find the container with id 297d278c5329a69869a9642fc727d37fb6c9788b910d0077840e8a3892e41fc1 Jul 14 17:58:07.623998 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:58:07.623967 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 14 17:58:07.624107 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:58:07.624043 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 14 17:58:07.624107 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:58:07.624086 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 14 17:58:08.023988 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:58:08.023951 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-b28b559b-gw-openshift-default-68489c4759-z9m8c" event={"ID":"b8561883-c29f-403d-aaef-2bd9318a9eb3","Type":"ContainerStarted","Data":"84671d46dda8e16bb3462715018eca426b7a66f81053c016069b097899063a7d"} Jul 14 17:58:08.024149 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:58:08.024018 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-b28b559b-gw-openshift-default-68489c4759-z9m8c" event={"ID":"b8561883-c29f-403d-aaef-2bd9318a9eb3","Type":"ContainerStarted","Data":"297d278c5329a69869a9642fc727d37fb6c9788b910d0077840e8a3892e41fc1"} Jul 14 17:58:08.055779 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:58:08.055731 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-b28b559b-gw-openshift-default-68489c4759-z9m8c" podStartSLOduration=1.055717296 podStartE2EDuration="1.055717296s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:58:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:58:08.054632392 +0000 UTC m=+1537.131089932" watchObservedRunningTime="2026-07-14 17:58:08.055717296 +0000 UTC m=+1537.132174835" Jul 14 17:58:08.460088 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:58:08.460055 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-b28b559b-gw-openshift-default-68489c4759-z9m8c" Jul 14 17:58:08.464647 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:58:08.464626 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-b28b559b-gw-openshift-default-68489c4759-z9m8c" Jul 14 17:58:09.029344 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:58:09.029309 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-b28b559b-gw-openshift-default-68489c4759-z9m8c" Jul 14 17:58:09.030359 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:58:09.030343 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-b28b559b-gw-openshift-default-68489c4759-z9m8c" Jul 14 17:58:11.811402 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:58:11.811380 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-09df0cc0-openshift-default-647f78d5b5-5ts4d" Jul 14 17:58:12.003806 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:58:12.003775 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b147d938-359e-423d-a33b-b9b6fd5269fa-istio-podinfo\") pod \"b147d938-359e-423d-a33b-b9b6fd5269fa\" (UID: \"b147d938-359e-423d-a33b-b9b6fd5269fa\") " Jul 14 17:58:12.003958 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:58:12.003832 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vm62t\" (UniqueName: \"kubernetes.io/projected/b147d938-359e-423d-a33b-b9b6fd5269fa-kube-api-access-vm62t\") pod \"b147d938-359e-423d-a33b-b9b6fd5269fa\" (UID: \"b147d938-359e-423d-a33b-b9b6fd5269fa\") " Jul 14 17:58:12.003998 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:58:12.003948 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b147d938-359e-423d-a33b-b9b6fd5269fa-workload-certs\") pod \"b147d938-359e-423d-a33b-b9b6fd5269fa\" (UID: \"b147d938-359e-423d-a33b-b9b6fd5269fa\") " Jul 14 17:58:12.003998 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:58:12.003987 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b147d938-359e-423d-a33b-b9b6fd5269fa-istio-data\") pod \"b147d938-359e-423d-a33b-b9b6fd5269fa\" (UID: \"b147d938-359e-423d-a33b-b9b6fd5269fa\") " Jul 14 17:58:12.004060 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:58:12.004035 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b147d938-359e-423d-a33b-b9b6fd5269fa-workload-socket\") pod \"b147d938-359e-423d-a33b-b9b6fd5269fa\" (UID: \"b147d938-359e-423d-a33b-b9b6fd5269fa\") " Jul 14 17:58:12.004105 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:58:12.004085 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b147d938-359e-423d-a33b-b9b6fd5269fa-istio-token\") pod \"b147d938-359e-423d-a33b-b9b6fd5269fa\" (UID: \"b147d938-359e-423d-a33b-b9b6fd5269fa\") " Jul 14 17:58:12.004156 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:58:12.004131 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b147d938-359e-423d-a33b-b9b6fd5269fa-credential-socket\") pod \"b147d938-359e-423d-a33b-b9b6fd5269fa\" (UID: \"b147d938-359e-423d-a33b-b9b6fd5269fa\") " Jul 14 17:58:12.004231 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:58:12.004155 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b147d938-359e-423d-a33b-b9b6fd5269fa-istiod-ca-cert\") pod \"b147d938-359e-423d-a33b-b9b6fd5269fa\" (UID: \"b147d938-359e-423d-a33b-b9b6fd5269fa\") " Jul 14 17:58:12.004231 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:58:12.004213 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b147d938-359e-423d-a33b-b9b6fd5269fa-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b147d938-359e-423d-a33b-b9b6fd5269fa" (UID: "b147d938-359e-423d-a33b-b9b6fd5269fa"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 17:58:12.004331 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:58:12.004266 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b147d938-359e-423d-a33b-b9b6fd5269fa-istio-envoy\") pod \"b147d938-359e-423d-a33b-b9b6fd5269fa\" (UID: \"b147d938-359e-423d-a33b-b9b6fd5269fa\") " Jul 14 17:58:12.004428 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:58:12.004385 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b147d938-359e-423d-a33b-b9b6fd5269fa-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b147d938-359e-423d-a33b-b9b6fd5269fa" (UID: "b147d938-359e-423d-a33b-b9b6fd5269fa"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 17:58:12.004428 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:58:12.004397 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b147d938-359e-423d-a33b-b9b6fd5269fa-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b147d938-359e-423d-a33b-b9b6fd5269fa" (UID: "b147d938-359e-423d-a33b-b9b6fd5269fa"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 17:58:12.004688 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:58:12.004645 2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b147d938-359e-423d-a33b-b9b6fd5269fa-workload-certs\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 17:58:12.004688 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:58:12.004669 2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b147d938-359e-423d-a33b-b9b6fd5269fa-workload-socket\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 17:58:12.004688 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:58:12.004682 2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b147d938-359e-423d-a33b-b9b6fd5269fa-credential-socket\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 17:58:12.005048 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:58:12.004689 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b147d938-359e-423d-a33b-b9b6fd5269fa-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b147d938-359e-423d-a33b-b9b6fd5269fa" (UID: "b147d938-359e-423d-a33b-b9b6fd5269fa"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 17:58:12.005142 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:58:12.005110 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b147d938-359e-423d-a33b-b9b6fd5269fa-istio-data" (OuterVolumeSpecName: "istio-data") pod "b147d938-359e-423d-a33b-b9b6fd5269fa" (UID: "b147d938-359e-423d-a33b-b9b6fd5269fa"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 17:58:12.006358 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:58:12.006331 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b147d938-359e-423d-a33b-b9b6fd5269fa-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b147d938-359e-423d-a33b-b9b6fd5269fa" (UID: "b147d938-359e-423d-a33b-b9b6fd5269fa"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 14 17:58:12.006457 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:58:12.006427 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b147d938-359e-423d-a33b-b9b6fd5269fa-istio-token" (OuterVolumeSpecName: "istio-token") pod "b147d938-359e-423d-a33b-b9b6fd5269fa" (UID: "b147d938-359e-423d-a33b-b9b6fd5269fa"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 17:58:12.006513 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:58:12.006447 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b147d938-359e-423d-a33b-b9b6fd5269fa-kube-api-access-vm62t" (OuterVolumeSpecName: "kube-api-access-vm62t") pod "b147d938-359e-423d-a33b-b9b6fd5269fa" (UID: "b147d938-359e-423d-a33b-b9b6fd5269fa"). InnerVolumeSpecName "kube-api-access-vm62t". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 17:58:12.006513 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:58:12.006499 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b147d938-359e-423d-a33b-b9b6fd5269fa-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b147d938-359e-423d-a33b-b9b6fd5269fa" (UID: "b147d938-359e-423d-a33b-b9b6fd5269fa"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 17:58:12.042963 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:58:12.042942 2562 generic.go:356] "Generic (PLEG): container finished" podID="b147d938-359e-423d-a33b-b9b6fd5269fa" containerID="209f1873e2f190f8453f9696c73858c9957badb9d06aee87f5e0bd0e03c730af" exitCode=0 Jul 14 17:58:12.043056 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:58:12.042998 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-09df0cc0-openshift-default-647f78d5b5-5ts4d" Jul 14 17:58:12.043056 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:58:12.043024 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-09df0cc0-openshift-default-647f78d5b5-5ts4d" event={"ID":"b147d938-359e-423d-a33b-b9b6fd5269fa","Type":"ContainerDied","Data":"209f1873e2f190f8453f9696c73858c9957badb9d06aee87f5e0bd0e03c730af"} Jul 14 17:58:12.043129 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:58:12.043090 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-09df0cc0-openshift-default-647f78d5b5-5ts4d" event={"ID":"b147d938-359e-423d-a33b-b9b6fd5269fa","Type":"ContainerDied","Data":"4eb76d68e9c46dad587e95cda25935cb1a741ead81b402331e4e422b60105898"} Jul 14 17:58:12.043129 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:58:12.043112 2562 scope.go:117] "RemoveContainer" containerID="209f1873e2f190f8453f9696c73858c9957badb9d06aee87f5e0bd0e03c730af" Jul 14 17:58:12.053594 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:58:12.053575 2562 scope.go:117] "RemoveContainer" containerID="43666d352986abd8a5e747d6a5ec9fbb03a744114bb6bce8f3b645be5af32500" Jul 14 17:58:12.061467 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:58:12.061450 2562 scope.go:117] "RemoveContainer" containerID="209f1873e2f190f8453f9696c73858c9957badb9d06aee87f5e0bd0e03c730af" Jul 14 17:58:12.061696 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:58:12.061676 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"209f1873e2f190f8453f9696c73858c9957badb9d06aee87f5e0bd0e03c730af\": container with ID starting with 209f1873e2f190f8453f9696c73858c9957badb9d06aee87f5e0bd0e03c730af not found: ID does not exist" containerID="209f1873e2f190f8453f9696c73858c9957badb9d06aee87f5e0bd0e03c730af" Jul 14 17:58:12.061752 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:58:12.061710 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"209f1873e2f190f8453f9696c73858c9957badb9d06aee87f5e0bd0e03c730af"} err="failed to get container status \"209f1873e2f190f8453f9696c73858c9957badb9d06aee87f5e0bd0e03c730af\": rpc error: code = NotFound desc = could not find container \"209f1873e2f190f8453f9696c73858c9957badb9d06aee87f5e0bd0e03c730af\": container with ID starting with 209f1873e2f190f8453f9696c73858c9957badb9d06aee87f5e0bd0e03c730af not found: ID does not exist" Jul 14 17:58:12.061752 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:58:12.061733 2562 scope.go:117] "RemoveContainer" containerID="43666d352986abd8a5e747d6a5ec9fbb03a744114bb6bce8f3b645be5af32500" Jul 14 17:58:12.061938 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:58:12.061923 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"43666d352986abd8a5e747d6a5ec9fbb03a744114bb6bce8f3b645be5af32500\": container with ID starting with 43666d352986abd8a5e747d6a5ec9fbb03a744114bb6bce8f3b645be5af32500 not found: ID does not exist" containerID="43666d352986abd8a5e747d6a5ec9fbb03a744114bb6bce8f3b645be5af32500" Jul 14 17:58:12.061977 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:58:12.061943 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"43666d352986abd8a5e747d6a5ec9fbb03a744114bb6bce8f3b645be5af32500"} err="failed to get container status \"43666d352986abd8a5e747d6a5ec9fbb03a744114bb6bce8f3b645be5af32500\": rpc error: code = NotFound desc = could not find container \"43666d352986abd8a5e747d6a5ec9fbb03a744114bb6bce8f3b645be5af32500\": container with ID starting with 43666d352986abd8a5e747d6a5ec9fbb03a744114bb6bce8f3b645be5af32500 not found: ID does not exist" Jul 14 17:58:12.068351 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:58:12.068330 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-09df0cc0-openshift-default-647f78d5b5-5ts4d"] Jul 14 17:58:12.074551 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:58:12.074532 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-09df0cc0-openshift-default-647f78d5b5-5ts4d"] Jul 14 17:58:12.105990 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:58:12.105968 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vm62t\" (UniqueName: \"kubernetes.io/projected/b147d938-359e-423d-a33b-b9b6fd5269fa-kube-api-access-vm62t\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 17:58:12.106067 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:58:12.105993 2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b147d938-359e-423d-a33b-b9b6fd5269fa-istio-data\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 17:58:12.106067 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:58:12.106003 2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b147d938-359e-423d-a33b-b9b6fd5269fa-istio-token\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 17:58:12.106067 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:58:12.106012 2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b147d938-359e-423d-a33b-b9b6fd5269fa-istiod-ca-cert\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 17:58:12.106067 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:58:12.106019 2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b147d938-359e-423d-a33b-b9b6fd5269fa-istio-envoy\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 17:58:12.106067 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:58:12.106027 2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b147d938-359e-423d-a33b-b9b6fd5269fa-istio-podinfo\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 17:58:13.518877 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:58:13.518843 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b147d938-359e-423d-a33b-b9b6fd5269fa" path="/var/lib/kubelet/pods/b147d938-359e-423d-a33b-b9b6fd5269fa/volumes" Jul 14 17:58:25.071347 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:58:25.071300 2562 patch_prober.go:28] interesting pod/e2e-migrate-b28b559b-gw-openshift-default-68489c4759-z9m8c container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.28:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 14 17:58:25.071818 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:58:25.071382 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-b28b559b-gw-openshift-default-68489c4759-z9m8c" podUID="b8561883-c29f-403d-aaef-2bd9318a9eb3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.28:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 14 17:58:26.107353 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:58:26.107288 2562 generic.go:356] "Generic (PLEG): container finished" podID="b8561883-c29f-403d-aaef-2bd9318a9eb3" containerID="84671d46dda8e16bb3462715018eca426b7a66f81053c016069b097899063a7d" exitCode=0 Jul 14 17:58:26.107353 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:58:26.107341 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-b28b559b-gw-openshift-default-68489c4759-z9m8c" event={"ID":"b8561883-c29f-403d-aaef-2bd9318a9eb3","Type":"ContainerDied","Data":"84671d46dda8e16bb3462715018eca426b7a66f81053c016069b097899063a7d"} Jul 14 17:58:26.107717 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:58:26.107658 2562 scope.go:117] "RemoveContainer" containerID="84671d46dda8e16bb3462715018eca426b7a66f81053c016069b097899063a7d" Jul 14 17:58:26.108633 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:58:26.108608 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 14 17:58:26.108695 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:58:26.108676 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 14 17:58:26.108729 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:58:26.108708 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 14 17:58:27.113838 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:58:27.113798 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-b28b559b-gw-openshift-default-68489c4759-z9m8c" event={"ID":"b8561883-c29f-403d-aaef-2bd9318a9eb3","Type":"ContainerStarted","Data":"a680240a930c9d0a3d1bcbedf05542ce02f28f69ddee7c93910c3780145e1e59"} Jul 14 17:58:27.460470 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:58:27.460363 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-b28b559b-gw-openshift-default-68489c4759-z9m8c" Jul 14 17:58:27.460677 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:58:27.460501 2562 patch_prober.go:28] interesting pod/e2e-migrate-b28b559b-gw-openshift-default-68489c4759-z9m8c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.28:15021/healthz/ready\": dial tcp 10.134.0.28:15021: connect: connection refused" start-of-body= Jul 14 17:58:27.460677 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:58:27.460542 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-b28b559b-gw-openshift-default-68489c4759-z9m8c" podUID="b8561883-c29f-403d-aaef-2bd9318a9eb3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.28:15021/healthz/ready\": dial tcp 10.134.0.28:15021: connect: connection refused" Jul 14 17:58:28.460430 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:58:28.460395 2562 patch_prober.go:28] interesting pod/e2e-migrate-b28b559b-gw-openshift-default-68489c4759-z9m8c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.28:15021/healthz/ready\": dial tcp 10.134.0.28:15021: connect: connection refused" start-of-body= Jul 14 17:58:28.461415 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:58:28.460984 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-b28b559b-gw-openshift-default-68489c4759-z9m8c" podUID="b8561883-c29f-403d-aaef-2bd9318a9eb3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.28:15021/healthz/ready\": dial tcp 10.134.0.28:15021: connect: connection refused" Jul 14 17:58:29.460256 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:58:29.460219 2562 patch_prober.go:28] interesting pod/e2e-migrate-b28b559b-gw-openshift-default-68489c4759-z9m8c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.28:15021/healthz/ready\": dial tcp 10.134.0.28:15021: connect: connection refused" start-of-body= Jul 14 17:58:29.460524 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:58:29.460498 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-b28b559b-gw-openshift-default-68489c4759-z9m8c" podUID="b8561883-c29f-403d-aaef-2bd9318a9eb3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.28:15021/healthz/ready\": dial tcp 10.134.0.28:15021: connect: connection refused" Jul 14 17:58:30.460742 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:58:30.460696 2562 patch_prober.go:28] interesting pod/e2e-migrate-b28b559b-gw-openshift-default-68489c4759-z9m8c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.28:15021/healthz/ready\": dial tcp 10.134.0.28:15021: connect: connection refused" start-of-body= Jul 14 17:58:30.461218 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:58:30.460769 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-b28b559b-gw-openshift-default-68489c4759-z9m8c" podUID="b8561883-c29f-403d-aaef-2bd9318a9eb3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.28:15021/healthz/ready\": dial tcp 10.134.0.28:15021: connect: connection refused" Jul 14 17:58:31.463463 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:58:31.463430 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-b28b559b-gw-openshift-default-68489c4759-z9m8c" Jul 14 17:58:31.463887 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:58:31.463866 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-b28b559b-gw-openshift-default-68489c4759-z9m8c" Jul 14 17:58:31.465114 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:58:31.465094 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-b28b559b-gw-openshift-default-68489c4759-z9m8c" Jul 14 17:59:13.040757 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:59:13.040721 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-b28b559b-gw-openshift-default-68489c4759-z9m8c"] Jul 14 17:59:13.041302 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:59:13.040945 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-b28b559b-gw-openshift-default-68489c4759-z9m8c" podUID="b8561883-c29f-403d-aaef-2bd9318a9eb3" containerName="istio-proxy" containerID="cri-o://a680240a930c9d0a3d1bcbedf05542ce02f28f69ddee7c93910c3780145e1e59" gracePeriod=30 Jul 14 17:59:16.466083 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:59:16.466049 2562 patch_prober.go:28] interesting pod/e2e-migrate-b28b559b-gw-openshift-default-68489c4759-z9m8c container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 14 17:59:16.466476 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:59:16.466108 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-b28b559b-gw-openshift-default-68489c4759-z9m8c" podUID="b8561883-c29f-403d-aaef-2bd9318a9eb3" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 14 17:59:18.293578 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:59:18.293552 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-b28b559b-gw-openshift-default-68489c4759-z9m8c" Jul 14 17:59:18.329640 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:59:18.329571 2562 generic.go:356] "Generic (PLEG): container finished" podID="b8561883-c29f-403d-aaef-2bd9318a9eb3" containerID="a680240a930c9d0a3d1bcbedf05542ce02f28f69ddee7c93910c3780145e1e59" exitCode=0 Jul 14 17:59:18.329640 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:59:18.329609 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-b28b559b-gw-openshift-default-68489c4759-z9m8c" Jul 14 17:59:18.329640 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:59:18.329635 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-b28b559b-gw-openshift-default-68489c4759-z9m8c" event={"ID":"b8561883-c29f-403d-aaef-2bd9318a9eb3","Type":"ContainerDied","Data":"a680240a930c9d0a3d1bcbedf05542ce02f28f69ddee7c93910c3780145e1e59"} Jul 14 17:59:18.329799 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:59:18.329683 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-b28b559b-gw-openshift-default-68489c4759-z9m8c" event={"ID":"b8561883-c29f-403d-aaef-2bd9318a9eb3","Type":"ContainerDied","Data":"297d278c5329a69869a9642fc727d37fb6c9788b910d0077840e8a3892e41fc1"} Jul 14 17:59:18.329799 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:59:18.329707 2562 scope.go:117] "RemoveContainer" containerID="a680240a930c9d0a3d1bcbedf05542ce02f28f69ddee7c93910c3780145e1e59" Jul 14 17:59:18.338928 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:59:18.338900 2562 scope.go:117] "RemoveContainer" containerID="84671d46dda8e16bb3462715018eca426b7a66f81053c016069b097899063a7d" Jul 14 17:59:18.347082 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:59:18.347055 2562 scope.go:117] "RemoveContainer" containerID="a680240a930c9d0a3d1bcbedf05542ce02f28f69ddee7c93910c3780145e1e59" Jul 14 17:59:18.347417 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:59:18.347395 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a680240a930c9d0a3d1bcbedf05542ce02f28f69ddee7c93910c3780145e1e59\": container with ID starting with a680240a930c9d0a3d1bcbedf05542ce02f28f69ddee7c93910c3780145e1e59 not found: ID does not exist" containerID="a680240a930c9d0a3d1bcbedf05542ce02f28f69ddee7c93910c3780145e1e59" Jul 14 17:59:18.347736 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:59:18.347424 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a680240a930c9d0a3d1bcbedf05542ce02f28f69ddee7c93910c3780145e1e59"} err="failed to get container status \"a680240a930c9d0a3d1bcbedf05542ce02f28f69ddee7c93910c3780145e1e59\": rpc error: code = NotFound desc = could not find container \"a680240a930c9d0a3d1bcbedf05542ce02f28f69ddee7c93910c3780145e1e59\": container with ID starting with a680240a930c9d0a3d1bcbedf05542ce02f28f69ddee7c93910c3780145e1e59 not found: ID does not exist" Jul 14 17:59:18.347736 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:59:18.347444 2562 scope.go:117] "RemoveContainer" containerID="84671d46dda8e16bb3462715018eca426b7a66f81053c016069b097899063a7d" Jul 14 17:59:18.348220 ip-10-0-129-209 kubenswrapper[2562]: E0714 17:59:18.348180 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"84671d46dda8e16bb3462715018eca426b7a66f81053c016069b097899063a7d\": container with ID starting with 84671d46dda8e16bb3462715018eca426b7a66f81053c016069b097899063a7d not found: ID does not exist" containerID="84671d46dda8e16bb3462715018eca426b7a66f81053c016069b097899063a7d" Jul 14 17:59:18.348303 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:59:18.348231 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"84671d46dda8e16bb3462715018eca426b7a66f81053c016069b097899063a7d"} err="failed to get container status \"84671d46dda8e16bb3462715018eca426b7a66f81053c016069b097899063a7d\": rpc error: code = NotFound desc = could not find container \"84671d46dda8e16bb3462715018eca426b7a66f81053c016069b097899063a7d\": container with ID starting with 84671d46dda8e16bb3462715018eca426b7a66f81053c016069b097899063a7d not found: ID does not exist" Jul 14 17:59:18.363146 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:59:18.363127 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b8561883-c29f-403d-aaef-2bd9318a9eb3-workload-socket\") pod \"b8561883-c29f-403d-aaef-2bd9318a9eb3\" (UID: \"b8561883-c29f-403d-aaef-2bd9318a9eb3\") " Jul 14 17:59:18.363273 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:59:18.363161 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b8561883-c29f-403d-aaef-2bd9318a9eb3-istio-envoy\") pod \"b8561883-c29f-403d-aaef-2bd9318a9eb3\" (UID: \"b8561883-c29f-403d-aaef-2bd9318a9eb3\") " Jul 14 17:59:18.363273 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:59:18.363227 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b8561883-c29f-403d-aaef-2bd9318a9eb3-istiod-ca-cert\") pod \"b8561883-c29f-403d-aaef-2bd9318a9eb3\" (UID: \"b8561883-c29f-403d-aaef-2bd9318a9eb3\") " Jul 14 17:59:18.363273 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:59:18.363269 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b8561883-c29f-403d-aaef-2bd9318a9eb3-credential-socket\") pod \"b8561883-c29f-403d-aaef-2bd9318a9eb3\" (UID: \"b8561883-c29f-403d-aaef-2bd9318a9eb3\") " Jul 14 17:59:18.363434 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:59:18.363298 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b8561883-c29f-403d-aaef-2bd9318a9eb3-istio-data\") pod \"b8561883-c29f-403d-aaef-2bd9318a9eb3\" (UID: \"b8561883-c29f-403d-aaef-2bd9318a9eb3\") " Jul 14 17:59:18.363434 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:59:18.363328 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b8561883-c29f-403d-aaef-2bd9318a9eb3-istio-podinfo\") pod \"b8561883-c29f-403d-aaef-2bd9318a9eb3\" (UID: \"b8561883-c29f-403d-aaef-2bd9318a9eb3\") " Jul 14 17:59:18.363534 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:59:18.363436 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b8561883-c29f-403d-aaef-2bd9318a9eb3-workload-certs\") pod \"b8561883-c29f-403d-aaef-2bd9318a9eb3\" (UID: \"b8561883-c29f-403d-aaef-2bd9318a9eb3\") " Jul 14 17:59:18.363534 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:59:18.363523 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6pxzk\" (UniqueName: \"kubernetes.io/projected/b8561883-c29f-403d-aaef-2bd9318a9eb3-kube-api-access-6pxzk\") pod \"b8561883-c29f-403d-aaef-2bd9318a9eb3\" (UID: \"b8561883-c29f-403d-aaef-2bd9318a9eb3\") " Jul 14 17:59:18.363633 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:59:18.363598 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b8561883-c29f-403d-aaef-2bd9318a9eb3-istio-token\") pod \"b8561883-c29f-403d-aaef-2bd9318a9eb3\" (UID: \"b8561883-c29f-403d-aaef-2bd9318a9eb3\") " Jul 14 17:59:18.363716 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:59:18.363430 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b8561883-c29f-403d-aaef-2bd9318a9eb3-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b8561883-c29f-403d-aaef-2bd9318a9eb3" (UID: "b8561883-c29f-403d-aaef-2bd9318a9eb3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 17:59:18.363774 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:59:18.363531 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b8561883-c29f-403d-aaef-2bd9318a9eb3-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b8561883-c29f-403d-aaef-2bd9318a9eb3" (UID: "b8561883-c29f-403d-aaef-2bd9318a9eb3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 17:59:18.363774 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:59:18.363680 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b8561883-c29f-403d-aaef-2bd9318a9eb3-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b8561883-c29f-403d-aaef-2bd9318a9eb3" (UID: "b8561883-c29f-403d-aaef-2bd9318a9eb3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 17:59:18.364242 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:59:18.364152 2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b8561883-c29f-403d-aaef-2bd9318a9eb3-workload-certs\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 17:59:18.364242 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:59:18.364179 2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b8561883-c29f-403d-aaef-2bd9318a9eb3-workload-socket\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 17:59:18.364242 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:59:18.364221 2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b8561883-c29f-403d-aaef-2bd9318a9eb3-credential-socket\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 17:59:18.364452 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:59:18.364400 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b8561883-c29f-403d-aaef-2bd9318a9eb3-istio-data" (OuterVolumeSpecName: "istio-data") pod "b8561883-c29f-403d-aaef-2bd9318a9eb3" (UID: "b8561883-c29f-403d-aaef-2bd9318a9eb3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 17:59:18.364676 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:59:18.364579 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b8561883-c29f-403d-aaef-2bd9318a9eb3-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b8561883-c29f-403d-aaef-2bd9318a9eb3" (UID: "b8561883-c29f-403d-aaef-2bd9318a9eb3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 17:59:18.365755 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:59:18.365722 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b8561883-c29f-403d-aaef-2bd9318a9eb3-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b8561883-c29f-403d-aaef-2bd9318a9eb3" (UID: "b8561883-c29f-403d-aaef-2bd9318a9eb3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 17:59:18.366316 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:59:18.366287 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b8561883-c29f-403d-aaef-2bd9318a9eb3-istio-token" (OuterVolumeSpecName: "istio-token") pod "b8561883-c29f-403d-aaef-2bd9318a9eb3" (UID: "b8561883-c29f-403d-aaef-2bd9318a9eb3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 17:59:18.366416 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:59:18.366326 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b8561883-c29f-403d-aaef-2bd9318a9eb3-kube-api-access-6pxzk" (OuterVolumeSpecName: "kube-api-access-6pxzk") pod "b8561883-c29f-403d-aaef-2bd9318a9eb3" (UID: "b8561883-c29f-403d-aaef-2bd9318a9eb3"). InnerVolumeSpecName "kube-api-access-6pxzk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 17:59:18.366416 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:59:18.366360 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b8561883-c29f-403d-aaef-2bd9318a9eb3-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b8561883-c29f-403d-aaef-2bd9318a9eb3" (UID: "b8561883-c29f-403d-aaef-2bd9318a9eb3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 14 17:59:18.464933 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:59:18.464908 2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b8561883-c29f-403d-aaef-2bd9318a9eb3-istio-token\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 17:59:18.464933 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:59:18.464934 2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b8561883-c29f-403d-aaef-2bd9318a9eb3-istio-envoy\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 17:59:18.465087 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:59:18.464943 2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b8561883-c29f-403d-aaef-2bd9318a9eb3-istiod-ca-cert\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 17:59:18.465087 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:59:18.464951 2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b8561883-c29f-403d-aaef-2bd9318a9eb3-istio-data\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 17:59:18.465087 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:59:18.464960 2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b8561883-c29f-403d-aaef-2bd9318a9eb3-istio-podinfo\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 17:59:18.465087 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:59:18.464967 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6pxzk\" (UniqueName: \"kubernetes.io/projected/b8561883-c29f-403d-aaef-2bd9318a9eb3-kube-api-access-6pxzk\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 17:59:18.660783 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:59:18.660750 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-b28b559b-gw-openshift-default-68489c4759-z9m8c"] Jul 14 17:59:18.665835 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:59:18.665812 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-b28b559b-gw-openshift-default-68489c4759-z9m8c"] Jul 14 17:59:19.519938 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:59:19.519904 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b8561883-c29f-403d-aaef-2bd9318a9eb3" path="/var/lib/kubelet/pods/b8561883-c29f-403d-aaef-2bd9318a9eb3/volumes" Jul 14 17:59:26.674379 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:59:26.674345 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-439ad9d2-openshift-default-6bf8ffddc-s96dw"] Jul 14 17:59:26.674700 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:59:26.674670 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b8561883-c29f-403d-aaef-2bd9318a9eb3" containerName="istio-proxy" Jul 14 17:59:26.674700 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:59:26.674680 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="b8561883-c29f-403d-aaef-2bd9318a9eb3" containerName="istio-proxy" Jul 14 17:59:26.674700 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:59:26.674690 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b147d938-359e-423d-a33b-b9b6fd5269fa" containerName="istio-proxy" Jul 14 17:59:26.674700 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:59:26.674696 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="b147d938-359e-423d-a33b-b9b6fd5269fa" containerName="istio-proxy" Jul 14 17:59:26.674830 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:59:26.674704 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b8561883-c29f-403d-aaef-2bd9318a9eb3" containerName="istio-proxy" Jul 14 17:59:26.674830 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:59:26.674710 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="b8561883-c29f-403d-aaef-2bd9318a9eb3" containerName="istio-proxy" Jul 14 17:59:26.674830 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:59:26.674753 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="b8561883-c29f-403d-aaef-2bd9318a9eb3" containerName="istio-proxy" Jul 14 17:59:26.674830 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:59:26.674761 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="b147d938-359e-423d-a33b-b9b6fd5269fa" containerName="istio-proxy" Jul 14 17:59:26.678929 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:59:26.678912 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-439ad9d2-openshift-default-6bf8ffddc-s96dw" Jul 14 17:59:26.681763 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:59:26.681744 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-439ad9d2-openshift-default-dockercfg-kmst5\"" Jul 14 17:59:26.697161 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:59:26.697135 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-439ad9d2-openshift-default-6bf8ffddc-s96dw"] Jul 14 17:59:26.720811 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:59:26.720789 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/79c1a410-d02e-467a-b4a4-784e9417d267-workload-socket\") pod \"e2e-ait-439ad9d2-openshift-default-6bf8ffddc-s96dw\" (UID: \"79c1a410-d02e-467a-b4a4-784e9417d267\") " pod="openshift-ingress/e2e-ait-439ad9d2-openshift-default-6bf8ffddc-s96dw" Jul 14 17:59:26.720928 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:59:26.720817 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/79c1a410-d02e-467a-b4a4-784e9417d267-istiod-ca-cert\") pod \"e2e-ait-439ad9d2-openshift-default-6bf8ffddc-s96dw\" (UID: \"79c1a410-d02e-467a-b4a4-784e9417d267\") " pod="openshift-ingress/e2e-ait-439ad9d2-openshift-default-6bf8ffddc-s96dw" Jul 14 17:59:26.720928 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:59:26.720841 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/79c1a410-d02e-467a-b4a4-784e9417d267-istio-envoy\") pod \"e2e-ait-439ad9d2-openshift-default-6bf8ffddc-s96dw\" (UID: \"79c1a410-d02e-467a-b4a4-784e9417d267\") " pod="openshift-ingress/e2e-ait-439ad9d2-openshift-default-6bf8ffddc-s96dw" Jul 14 17:59:26.720928 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:59:26.720876 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/79c1a410-d02e-467a-b4a4-784e9417d267-istio-data\") pod \"e2e-ait-439ad9d2-openshift-default-6bf8ffddc-s96dw\" (UID: \"79c1a410-d02e-467a-b4a4-784e9417d267\") " pod="openshift-ingress/e2e-ait-439ad9d2-openshift-default-6bf8ffddc-s96dw" Jul 14 17:59:26.721075 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:59:26.720928 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/79c1a410-d02e-467a-b4a4-784e9417d267-workload-certs\") pod \"e2e-ait-439ad9d2-openshift-default-6bf8ffddc-s96dw\" (UID: \"79c1a410-d02e-467a-b4a4-784e9417d267\") " pod="openshift-ingress/e2e-ait-439ad9d2-openshift-default-6bf8ffddc-s96dw" Jul 14 17:59:26.721075 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:59:26.720960 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/79c1a410-d02e-467a-b4a4-784e9417d267-istio-podinfo\") pod \"e2e-ait-439ad9d2-openshift-default-6bf8ffddc-s96dw\" (UID: \"79c1a410-d02e-467a-b4a4-784e9417d267\") " pod="openshift-ingress/e2e-ait-439ad9d2-openshift-default-6bf8ffddc-s96dw" Jul 14 17:59:26.721075 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:59:26.720985 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-twrd7\" (UniqueName: \"kubernetes.io/projected/79c1a410-d02e-467a-b4a4-784e9417d267-kube-api-access-twrd7\") pod \"e2e-ait-439ad9d2-openshift-default-6bf8ffddc-s96dw\" (UID: \"79c1a410-d02e-467a-b4a4-784e9417d267\") " pod="openshift-ingress/e2e-ait-439ad9d2-openshift-default-6bf8ffddc-s96dw" Jul 14 17:59:26.721075 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:59:26.721002 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/79c1a410-d02e-467a-b4a4-784e9417d267-credential-socket\") pod \"e2e-ait-439ad9d2-openshift-default-6bf8ffddc-s96dw\" (UID: \"79c1a410-d02e-467a-b4a4-784e9417d267\") " pod="openshift-ingress/e2e-ait-439ad9d2-openshift-default-6bf8ffddc-s96dw" Jul 14 17:59:26.721075 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:59:26.721021 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/79c1a410-d02e-467a-b4a4-784e9417d267-istio-token\") pod \"e2e-ait-439ad9d2-openshift-default-6bf8ffddc-s96dw\" (UID: \"79c1a410-d02e-467a-b4a4-784e9417d267\") " pod="openshift-ingress/e2e-ait-439ad9d2-openshift-default-6bf8ffddc-s96dw" Jul 14 17:59:26.821970 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:59:26.821941 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/79c1a410-d02e-467a-b4a4-784e9417d267-istio-data\") pod \"e2e-ait-439ad9d2-openshift-default-6bf8ffddc-s96dw\" (UID: \"79c1a410-d02e-467a-b4a4-784e9417d267\") " pod="openshift-ingress/e2e-ait-439ad9d2-openshift-default-6bf8ffddc-s96dw" Jul 14 17:59:26.822147 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:59:26.821989 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/79c1a410-d02e-467a-b4a4-784e9417d267-workload-certs\") pod \"e2e-ait-439ad9d2-openshift-default-6bf8ffddc-s96dw\" (UID: \"79c1a410-d02e-467a-b4a4-784e9417d267\") " pod="openshift-ingress/e2e-ait-439ad9d2-openshift-default-6bf8ffddc-s96dw" Jul 14 17:59:26.822147 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:59:26.822021 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/79c1a410-d02e-467a-b4a4-784e9417d267-istio-podinfo\") pod \"e2e-ait-439ad9d2-openshift-default-6bf8ffddc-s96dw\" (UID: \"79c1a410-d02e-467a-b4a4-784e9417d267\") " pod="openshift-ingress/e2e-ait-439ad9d2-openshift-default-6bf8ffddc-s96dw" Jul 14 17:59:26.822147 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:59:26.822040 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-twrd7\" (UniqueName: \"kubernetes.io/projected/79c1a410-d02e-467a-b4a4-784e9417d267-kube-api-access-twrd7\") pod \"e2e-ait-439ad9d2-openshift-default-6bf8ffddc-s96dw\" (UID: \"79c1a410-d02e-467a-b4a4-784e9417d267\") " pod="openshift-ingress/e2e-ait-439ad9d2-openshift-default-6bf8ffddc-s96dw" Jul 14 17:59:26.822147 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:59:26.822056 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/79c1a410-d02e-467a-b4a4-784e9417d267-credential-socket\") pod \"e2e-ait-439ad9d2-openshift-default-6bf8ffddc-s96dw\" (UID: \"79c1a410-d02e-467a-b4a4-784e9417d267\") " pod="openshift-ingress/e2e-ait-439ad9d2-openshift-default-6bf8ffddc-s96dw" Jul 14 17:59:26.822147 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:59:26.822081 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/79c1a410-d02e-467a-b4a4-784e9417d267-istio-token\") pod \"e2e-ait-439ad9d2-openshift-default-6bf8ffddc-s96dw\" (UID: \"79c1a410-d02e-467a-b4a4-784e9417d267\") " pod="openshift-ingress/e2e-ait-439ad9d2-openshift-default-6bf8ffddc-s96dw" Jul 14 17:59:26.822147 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:59:26.822119 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/79c1a410-d02e-467a-b4a4-784e9417d267-workload-socket\") pod \"e2e-ait-439ad9d2-openshift-default-6bf8ffddc-s96dw\" (UID: \"79c1a410-d02e-467a-b4a4-784e9417d267\") " pod="openshift-ingress/e2e-ait-439ad9d2-openshift-default-6bf8ffddc-s96dw" Jul 14 17:59:26.822147 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:59:26.822145 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/79c1a410-d02e-467a-b4a4-784e9417d267-istiod-ca-cert\") pod \"e2e-ait-439ad9d2-openshift-default-6bf8ffddc-s96dw\" (UID: \"79c1a410-d02e-467a-b4a4-784e9417d267\") " pod="openshift-ingress/e2e-ait-439ad9d2-openshift-default-6bf8ffddc-s96dw" Jul 14 17:59:26.822558 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:59:26.822180 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/79c1a410-d02e-467a-b4a4-784e9417d267-istio-envoy\") pod \"e2e-ait-439ad9d2-openshift-default-6bf8ffddc-s96dw\" (UID: \"79c1a410-d02e-467a-b4a4-784e9417d267\") " pod="openshift-ingress/e2e-ait-439ad9d2-openshift-default-6bf8ffddc-s96dw" Jul 14 17:59:26.822558 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:59:26.822419 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/79c1a410-d02e-467a-b4a4-784e9417d267-istio-data\") pod \"e2e-ait-439ad9d2-openshift-default-6bf8ffddc-s96dw\" (UID: \"79c1a410-d02e-467a-b4a4-784e9417d267\") " pod="openshift-ingress/e2e-ait-439ad9d2-openshift-default-6bf8ffddc-s96dw" Jul 14 17:59:26.822558 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:59:26.822488 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/79c1a410-d02e-467a-b4a4-784e9417d267-workload-certs\") pod \"e2e-ait-439ad9d2-openshift-default-6bf8ffddc-s96dw\" (UID: \"79c1a410-d02e-467a-b4a4-784e9417d267\") " pod="openshift-ingress/e2e-ait-439ad9d2-openshift-default-6bf8ffddc-s96dw" Jul 14 17:59:26.822558 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:59:26.822535 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/79c1a410-d02e-467a-b4a4-784e9417d267-workload-socket\") pod \"e2e-ait-439ad9d2-openshift-default-6bf8ffddc-s96dw\" (UID: \"79c1a410-d02e-467a-b4a4-784e9417d267\") " pod="openshift-ingress/e2e-ait-439ad9d2-openshift-default-6bf8ffddc-s96dw" Jul 14 17:59:26.822823 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:59:26.822787 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/79c1a410-d02e-467a-b4a4-784e9417d267-credential-socket\") pod \"e2e-ait-439ad9d2-openshift-default-6bf8ffddc-s96dw\" (UID: \"79c1a410-d02e-467a-b4a4-784e9417d267\") " pod="openshift-ingress/e2e-ait-439ad9d2-openshift-default-6bf8ffddc-s96dw" Jul 14 17:59:26.822895 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:59:26.822876 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/79c1a410-d02e-467a-b4a4-784e9417d267-istiod-ca-cert\") pod \"e2e-ait-439ad9d2-openshift-default-6bf8ffddc-s96dw\" (UID: \"79c1a410-d02e-467a-b4a4-784e9417d267\") " pod="openshift-ingress/e2e-ait-439ad9d2-openshift-default-6bf8ffddc-s96dw" Jul 14 17:59:26.824513 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:59:26.824497 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/79c1a410-d02e-467a-b4a4-784e9417d267-istio-envoy\") pod \"e2e-ait-439ad9d2-openshift-default-6bf8ffddc-s96dw\" (UID: \"79c1a410-d02e-467a-b4a4-784e9417d267\") " pod="openshift-ingress/e2e-ait-439ad9d2-openshift-default-6bf8ffddc-s96dw" Jul 14 17:59:26.824658 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:59:26.824642 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/79c1a410-d02e-467a-b4a4-784e9417d267-istio-podinfo\") pod \"e2e-ait-439ad9d2-openshift-default-6bf8ffddc-s96dw\" (UID: \"79c1a410-d02e-467a-b4a4-784e9417d267\") " pod="openshift-ingress/e2e-ait-439ad9d2-openshift-default-6bf8ffddc-s96dw" Jul 14 17:59:26.831653 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:59:26.831624 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/79c1a410-d02e-467a-b4a4-784e9417d267-istio-token\") pod \"e2e-ait-439ad9d2-openshift-default-6bf8ffddc-s96dw\" (UID: \"79c1a410-d02e-467a-b4a4-784e9417d267\") " pod="openshift-ingress/e2e-ait-439ad9d2-openshift-default-6bf8ffddc-s96dw" Jul 14 17:59:26.832258 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:59:26.832238 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-twrd7\" (UniqueName: \"kubernetes.io/projected/79c1a410-d02e-467a-b4a4-784e9417d267-kube-api-access-twrd7\") pod \"e2e-ait-439ad9d2-openshift-default-6bf8ffddc-s96dw\" (UID: \"79c1a410-d02e-467a-b4a4-784e9417d267\") " pod="openshift-ingress/e2e-ait-439ad9d2-openshift-default-6bf8ffddc-s96dw" Jul 14 17:59:26.988443 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:59:26.988380 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-439ad9d2-openshift-default-6bf8ffddc-s96dw" Jul 14 17:59:27.127239 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:59:27.127215 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-439ad9d2-openshift-default-6bf8ffddc-s96dw"] Jul 14 17:59:27.129677 ip-10-0-129-209 kubenswrapper[2562]: W0714 17:59:27.129645 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79c1a410_d02e_467a_b4a4_784e9417d267.slice/crio-89eee1d5102e3a771a8cf45ba430282dc798f538342885a010888ea27d112b4c WatchSource:0}: Error finding container 89eee1d5102e3a771a8cf45ba430282dc798f538342885a010888ea27d112b4c: Status 404 returned error can't find the container with id 89eee1d5102e3a771a8cf45ba430282dc798f538342885a010888ea27d112b4c Jul 14 17:59:27.131753 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:59:27.131724 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 14 17:59:27.131845 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:59:27.131784 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 14 17:59:27.131845 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:59:27.131815 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 14 17:59:27.369508 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:59:27.369472 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-439ad9d2-openshift-default-6bf8ffddc-s96dw" event={"ID":"79c1a410-d02e-467a-b4a4-784e9417d267","Type":"ContainerStarted","Data":"6d96efb4e692e558af7d916d04bf054b443b7d5bfa541d156e01066dd2ec459d"} Jul 14 17:59:27.369673 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:59:27.369534 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-439ad9d2-openshift-default-6bf8ffddc-s96dw" event={"ID":"79c1a410-d02e-467a-b4a4-784e9417d267","Type":"ContainerStarted","Data":"89eee1d5102e3a771a8cf45ba430282dc798f538342885a010888ea27d112b4c"} Jul 14 17:59:27.400137 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:59:27.400089 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-439ad9d2-openshift-default-6bf8ffddc-s96dw" podStartSLOduration=1.400068154 podStartE2EDuration="1.400068154s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 17:59:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 17:59:27.398788567 +0000 UTC m=+1616.475246133" watchObservedRunningTime="2026-07-14 17:59:27.400068154 +0000 UTC m=+1616.476525775" Jul 14 17:59:27.989176 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:59:27.989140 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-439ad9d2-openshift-default-6bf8ffddc-s96dw" Jul 14 17:59:28.993062 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:59:28.993027 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-439ad9d2-openshift-default-6bf8ffddc-s96dw" Jul 14 17:59:29.378344 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:59:29.378320 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-439ad9d2-openshift-default-6bf8ffddc-s96dw" Jul 14 17:59:29.379399 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:59:29.379382 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-439ad9d2-openshift-default-6bf8ffddc-s96dw" Jul 14 17:59:38.417276 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:59:38.417217 2562 generic.go:356] "Generic (PLEG): container finished" podID="79c1a410-d02e-467a-b4a4-784e9417d267" containerID="6d96efb4e692e558af7d916d04bf054b443b7d5bfa541d156e01066dd2ec459d" exitCode=0 Jul 14 17:59:38.417702 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:59:38.417299 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-439ad9d2-openshift-default-6bf8ffddc-s96dw" event={"ID":"79c1a410-d02e-467a-b4a4-784e9417d267","Type":"ContainerDied","Data":"6d96efb4e692e558af7d916d04bf054b443b7d5bfa541d156e01066dd2ec459d"} Jul 14 17:59:38.417794 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:59:38.417775 2562 scope.go:117] "RemoveContainer" containerID="6d96efb4e692e558af7d916d04bf054b443b7d5bfa541d156e01066dd2ec459d" Jul 14 17:59:38.418780 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:59:38.418756 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 14 17:59:38.418852 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:59:38.418823 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 14 17:59:38.418902 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:59:38.418856 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 14 17:59:39.425795 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:59:39.425762 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-439ad9d2-openshift-default-6bf8ffddc-s96dw" event={"ID":"79c1a410-d02e-467a-b4a4-784e9417d267","Type":"ContainerStarted","Data":"ffa7bc25debf80b7e1e4b09f861d6f18f5c28b27445c09c3b48f194abade7964"} Jul 14 17:59:39.988896 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:59:39.988860 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-439ad9d2-openshift-default-6bf8ffddc-s96dw" Jul 14 17:59:39.989975 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:59:39.989948 2562 patch_prober.go:28] interesting pod/e2e-ait-439ad9d2-openshift-default-6bf8ffddc-s96dw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused" start-of-body= Jul 14 17:59:39.990097 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:59:39.989999 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-439ad9d2-openshift-default-6bf8ffddc-s96dw" podUID="79c1a410-d02e-467a-b4a4-784e9417d267" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused" Jul 14 17:59:40.989443 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:59:40.989406 2562 patch_prober.go:28] interesting pod/e2e-ait-439ad9d2-openshift-default-6bf8ffddc-s96dw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused" start-of-body= Jul 14 17:59:40.989858 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:59:40.989484 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-439ad9d2-openshift-default-6bf8ffddc-s96dw" podUID="79c1a410-d02e-467a-b4a4-784e9417d267" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused" Jul 14 17:59:41.989475 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:59:41.989435 2562 patch_prober.go:28] interesting pod/e2e-ait-439ad9d2-openshift-default-6bf8ffddc-s96dw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused" start-of-body= Jul 14 17:59:41.989891 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:59:41.989497 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-439ad9d2-openshift-default-6bf8ffddc-s96dw" podUID="79c1a410-d02e-467a-b4a4-784e9417d267" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused" Jul 14 17:59:42.989650 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:59:42.989612 2562 patch_prober.go:28] interesting pod/e2e-ait-439ad9d2-openshift-default-6bf8ffddc-s96dw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused" start-of-body= Jul 14 17:59:42.990143 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:59:42.989677 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-439ad9d2-openshift-default-6bf8ffddc-s96dw" podUID="79c1a410-d02e-467a-b4a4-784e9417d267" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused" Jul 14 17:59:43.993231 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:59:43.993181 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-439ad9d2-openshift-default-6bf8ffddc-s96dw" Jul 14 17:59:43.993716 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:59:43.993581 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-439ad9d2-openshift-default-6bf8ffddc-s96dw" Jul 14 17:59:43.994284 ip-10-0-129-209 kubenswrapper[2562]: I0714 17:59:43.994266 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-439ad9d2-openshift-default-6bf8ffddc-s96dw" Jul 14 18:00:49.069548 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:00:49.069517 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-439ad9d2-openshift-default-6bf8ffddc-s96dw"] Jul 14 18:00:49.070056 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:00:49.069775 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-439ad9d2-openshift-default-6bf8ffddc-s96dw" podUID="79c1a410-d02e-467a-b4a4-784e9417d267" containerName="istio-proxy" containerID="cri-o://ffa7bc25debf80b7e1e4b09f861d6f18f5c28b27445c09c3b48f194abade7964" gracePeriod=30 Jul 14 18:00:49.352596 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:00:49.352516 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-68125063-openshift-default-574c9c8c67-ngglh"] Jul 14 18:00:49.352926 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:00:49.352907 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b147d938-359e-423d-a33b-b9b6fd5269fa" containerName="istio-proxy" Jul 14 18:00:49.352996 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:00:49.352932 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="b147d938-359e-423d-a33b-b9b6fd5269fa" containerName="istio-proxy" Jul 14 18:00:49.353057 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:00:49.353015 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="b8561883-c29f-403d-aaef-2bd9318a9eb3" containerName="istio-proxy" Jul 14 18:00:49.353057 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:00:49.353023 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="b147d938-359e-423d-a33b-b9b6fd5269fa" containerName="istio-proxy" Jul 14 18:00:49.355840 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:00:49.355819 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-68125063-openshift-default-574c9c8c67-ngglh" Jul 14 18:00:49.358249 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:00:49.358231 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-68125063-openshift-default-dockercfg-g7x9t\"" Jul 14 18:00:49.372013 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:00:49.371991 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-68125063-openshift-default-574c9c8c67-ngglh"] Jul 14 18:00:49.542300 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:00:49.542269 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/533a91ec-be53-4df9-a11b-fd45f75a0c66-credential-socket\") pod \"e2e-derive-68125063-openshift-default-574c9c8c67-ngglh\" (UID: \"533a91ec-be53-4df9-a11b-fd45f75a0c66\") " pod="openshift-ingress/e2e-derive-68125063-openshift-default-574c9c8c67-ngglh" Jul 14 18:00:49.542300 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:00:49.542301 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/533a91ec-be53-4df9-a11b-fd45f75a0c66-istiod-ca-cert\") pod \"e2e-derive-68125063-openshift-default-574c9c8c67-ngglh\" (UID: \"533a91ec-be53-4df9-a11b-fd45f75a0c66\") " pod="openshift-ingress/e2e-derive-68125063-openshift-default-574c9c8c67-ngglh" Jul 14 18:00:49.542508 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:00:49.542325 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/533a91ec-be53-4df9-a11b-fd45f75a0c66-istio-data\") pod \"e2e-derive-68125063-openshift-default-574c9c8c67-ngglh\" (UID: \"533a91ec-be53-4df9-a11b-fd45f75a0c66\") " pod="openshift-ingress/e2e-derive-68125063-openshift-default-574c9c8c67-ngglh" Jul 14 18:00:49.542508 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:00:49.542384 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/533a91ec-be53-4df9-a11b-fd45f75a0c66-istio-podinfo\") pod \"e2e-derive-68125063-openshift-default-574c9c8c67-ngglh\" (UID: \"533a91ec-be53-4df9-a11b-fd45f75a0c66\") " pod="openshift-ingress/e2e-derive-68125063-openshift-default-574c9c8c67-ngglh" Jul 14 18:00:49.542508 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:00:49.542408 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ndm4m\" (UniqueName: \"kubernetes.io/projected/533a91ec-be53-4df9-a11b-fd45f75a0c66-kube-api-access-ndm4m\") pod \"e2e-derive-68125063-openshift-default-574c9c8c67-ngglh\" (UID: \"533a91ec-be53-4df9-a11b-fd45f75a0c66\") " pod="openshift-ingress/e2e-derive-68125063-openshift-default-574c9c8c67-ngglh" Jul 14 18:00:49.542508 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:00:49.542465 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/533a91ec-be53-4df9-a11b-fd45f75a0c66-istio-token\") pod \"e2e-derive-68125063-openshift-default-574c9c8c67-ngglh\" (UID: \"533a91ec-be53-4df9-a11b-fd45f75a0c66\") " pod="openshift-ingress/e2e-derive-68125063-openshift-default-574c9c8c67-ngglh" Jul 14 18:00:49.542508 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:00:49.542486 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/533a91ec-be53-4df9-a11b-fd45f75a0c66-workload-socket\") pod \"e2e-derive-68125063-openshift-default-574c9c8c67-ngglh\" (UID: \"533a91ec-be53-4df9-a11b-fd45f75a0c66\") " pod="openshift-ingress/e2e-derive-68125063-openshift-default-574c9c8c67-ngglh" Jul 14 18:00:49.542692 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:00:49.542537 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/533a91ec-be53-4df9-a11b-fd45f75a0c66-workload-certs\") pod \"e2e-derive-68125063-openshift-default-574c9c8c67-ngglh\" (UID: \"533a91ec-be53-4df9-a11b-fd45f75a0c66\") " pod="openshift-ingress/e2e-derive-68125063-openshift-default-574c9c8c67-ngglh" Jul 14 18:00:49.542692 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:00:49.542564 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/533a91ec-be53-4df9-a11b-fd45f75a0c66-istio-envoy\") pod \"e2e-derive-68125063-openshift-default-574c9c8c67-ngglh\" (UID: \"533a91ec-be53-4df9-a11b-fd45f75a0c66\") " pod="openshift-ingress/e2e-derive-68125063-openshift-default-574c9c8c67-ngglh" Jul 14 18:00:49.643484 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:00:49.643459 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/533a91ec-be53-4df9-a11b-fd45f75a0c66-istio-data\") pod \"e2e-derive-68125063-openshift-default-574c9c8c67-ngglh\" (UID: \"533a91ec-be53-4df9-a11b-fd45f75a0c66\") " pod="openshift-ingress/e2e-derive-68125063-openshift-default-574c9c8c67-ngglh" Jul 14 18:00:49.643632 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:00:49.643493 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/533a91ec-be53-4df9-a11b-fd45f75a0c66-istio-podinfo\") pod \"e2e-derive-68125063-openshift-default-574c9c8c67-ngglh\" (UID: \"533a91ec-be53-4df9-a11b-fd45f75a0c66\") " pod="openshift-ingress/e2e-derive-68125063-openshift-default-574c9c8c67-ngglh" Jul 14 18:00:49.643632 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:00:49.643511 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ndm4m\" (UniqueName: \"kubernetes.io/projected/533a91ec-be53-4df9-a11b-fd45f75a0c66-kube-api-access-ndm4m\") pod \"e2e-derive-68125063-openshift-default-574c9c8c67-ngglh\" (UID: \"533a91ec-be53-4df9-a11b-fd45f75a0c66\") " pod="openshift-ingress/e2e-derive-68125063-openshift-default-574c9c8c67-ngglh" Jul 14 18:00:49.643632 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:00:49.643567 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/533a91ec-be53-4df9-a11b-fd45f75a0c66-istio-token\") pod \"e2e-derive-68125063-openshift-default-574c9c8c67-ngglh\" (UID: \"533a91ec-be53-4df9-a11b-fd45f75a0c66\") " pod="openshift-ingress/e2e-derive-68125063-openshift-default-574c9c8c67-ngglh" Jul 14 18:00:49.643632 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:00:49.643593 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/533a91ec-be53-4df9-a11b-fd45f75a0c66-workload-socket\") pod \"e2e-derive-68125063-openshift-default-574c9c8c67-ngglh\" (UID: \"533a91ec-be53-4df9-a11b-fd45f75a0c66\") " pod="openshift-ingress/e2e-derive-68125063-openshift-default-574c9c8c67-ngglh" Jul 14 18:00:49.643852 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:00:49.643647 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/533a91ec-be53-4df9-a11b-fd45f75a0c66-workload-certs\") pod \"e2e-derive-68125063-openshift-default-574c9c8c67-ngglh\" (UID: \"533a91ec-be53-4df9-a11b-fd45f75a0c66\") " pod="openshift-ingress/e2e-derive-68125063-openshift-default-574c9c8c67-ngglh" Jul 14 18:00:49.643852 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:00:49.643684 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/533a91ec-be53-4df9-a11b-fd45f75a0c66-istio-envoy\") pod \"e2e-derive-68125063-openshift-default-574c9c8c67-ngglh\" (UID: \"533a91ec-be53-4df9-a11b-fd45f75a0c66\") " pod="openshift-ingress/e2e-derive-68125063-openshift-default-574c9c8c67-ngglh" Jul 14 18:00:49.643852 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:00:49.643735 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/533a91ec-be53-4df9-a11b-fd45f75a0c66-credential-socket\") pod \"e2e-derive-68125063-openshift-default-574c9c8c67-ngglh\" (UID: \"533a91ec-be53-4df9-a11b-fd45f75a0c66\") " pod="openshift-ingress/e2e-derive-68125063-openshift-default-574c9c8c67-ngglh" Jul 14 18:00:49.643852 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:00:49.643758 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/533a91ec-be53-4df9-a11b-fd45f75a0c66-istiod-ca-cert\") pod \"e2e-derive-68125063-openshift-default-574c9c8c67-ngglh\" (UID: \"533a91ec-be53-4df9-a11b-fd45f75a0c66\") " pod="openshift-ingress/e2e-derive-68125063-openshift-default-574c9c8c67-ngglh" Jul 14 18:00:49.644055 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:00:49.643901 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/533a91ec-be53-4df9-a11b-fd45f75a0c66-istio-data\") pod \"e2e-derive-68125063-openshift-default-574c9c8c67-ngglh\" (UID: \"533a91ec-be53-4df9-a11b-fd45f75a0c66\") " pod="openshift-ingress/e2e-derive-68125063-openshift-default-574c9c8c67-ngglh" Jul 14 18:00:49.644055 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:00:49.643950 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/533a91ec-be53-4df9-a11b-fd45f75a0c66-workload-socket\") pod \"e2e-derive-68125063-openshift-default-574c9c8c67-ngglh\" (UID: \"533a91ec-be53-4df9-a11b-fd45f75a0c66\") " pod="openshift-ingress/e2e-derive-68125063-openshift-default-574c9c8c67-ngglh" Jul 14 18:00:49.644055 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:00:49.644038 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/533a91ec-be53-4df9-a11b-fd45f75a0c66-workload-certs\") pod \"e2e-derive-68125063-openshift-default-574c9c8c67-ngglh\" (UID: \"533a91ec-be53-4df9-a11b-fd45f75a0c66\") " pod="openshift-ingress/e2e-derive-68125063-openshift-default-574c9c8c67-ngglh" Jul 14 18:00:49.644262 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:00:49.644136 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/533a91ec-be53-4df9-a11b-fd45f75a0c66-credential-socket\") pod \"e2e-derive-68125063-openshift-default-574c9c8c67-ngglh\" (UID: \"533a91ec-be53-4df9-a11b-fd45f75a0c66\") " pod="openshift-ingress/e2e-derive-68125063-openshift-default-574c9c8c67-ngglh" Jul 14 18:00:49.644518 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:00:49.644497 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/533a91ec-be53-4df9-a11b-fd45f75a0c66-istiod-ca-cert\") pod \"e2e-derive-68125063-openshift-default-574c9c8c67-ngglh\" (UID: \"533a91ec-be53-4df9-a11b-fd45f75a0c66\") " pod="openshift-ingress/e2e-derive-68125063-openshift-default-574c9c8c67-ngglh" Jul 14 18:00:49.646170 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:00:49.646144 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/533a91ec-be53-4df9-a11b-fd45f75a0c66-istio-envoy\") pod \"e2e-derive-68125063-openshift-default-574c9c8c67-ngglh\" (UID: \"533a91ec-be53-4df9-a11b-fd45f75a0c66\") " pod="openshift-ingress/e2e-derive-68125063-openshift-default-574c9c8c67-ngglh" Jul 14 18:00:49.646275 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:00:49.646149 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/533a91ec-be53-4df9-a11b-fd45f75a0c66-istio-podinfo\") pod \"e2e-derive-68125063-openshift-default-574c9c8c67-ngglh\" (UID: \"533a91ec-be53-4df9-a11b-fd45f75a0c66\") " pod="openshift-ingress/e2e-derive-68125063-openshift-default-574c9c8c67-ngglh" Jul 14 18:00:49.655042 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:00:49.655018 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/533a91ec-be53-4df9-a11b-fd45f75a0c66-istio-token\") pod \"e2e-derive-68125063-openshift-default-574c9c8c67-ngglh\" (UID: \"533a91ec-be53-4df9-a11b-fd45f75a0c66\") " pod="openshift-ingress/e2e-derive-68125063-openshift-default-574c9c8c67-ngglh" Jul 14 18:00:49.655416 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:00:49.655397 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ndm4m\" (UniqueName: \"kubernetes.io/projected/533a91ec-be53-4df9-a11b-fd45f75a0c66-kube-api-access-ndm4m\") pod \"e2e-derive-68125063-openshift-default-574c9c8c67-ngglh\" (UID: \"533a91ec-be53-4df9-a11b-fd45f75a0c66\") " pod="openshift-ingress/e2e-derive-68125063-openshift-default-574c9c8c67-ngglh" Jul 14 18:00:49.666107 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:00:49.666087 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-68125063-openshift-default-574c9c8c67-ngglh" Jul 14 18:00:49.801787 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:00:49.801762 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-68125063-openshift-default-574c9c8c67-ngglh"] Jul 14 18:00:49.804291 ip-10-0-129-209 kubenswrapper[2562]: W0714 18:00:49.804262 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod533a91ec_be53_4df9_a11b_fd45f75a0c66.slice/crio-bdfb9d87829bde3e22f2926c5333c451c428ccbf6f5f71a505f6e3bbdea3dfbc WatchSource:0}: Error finding container bdfb9d87829bde3e22f2926c5333c451c428ccbf6f5f71a505f6e3bbdea3dfbc: Status 404 returned error can't find the container with id bdfb9d87829bde3e22f2926c5333c451c428ccbf6f5f71a505f6e3bbdea3dfbc Jul 14 18:00:49.806378 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:00:49.806348 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 14 18:00:49.806447 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:00:49.806409 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 14 18:00:49.806447 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:00:49.806435 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 14 18:00:50.740529 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:00:50.740499 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-68125063-openshift-default-574c9c8c67-ngglh" event={"ID":"533a91ec-be53-4df9-a11b-fd45f75a0c66","Type":"ContainerStarted","Data":"abac056307899e744c50e2a366a29703c43c9efa6921e1e4b7687c942ba9c05a"} Jul 14 18:00:50.740872 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:00:50.740548 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-68125063-openshift-default-574c9c8c67-ngglh" event={"ID":"533a91ec-be53-4df9-a11b-fd45f75a0c66","Type":"ContainerStarted","Data":"bdfb9d87829bde3e22f2926c5333c451c428ccbf6f5f71a505f6e3bbdea3dfbc"} Jul 14 18:00:50.773596 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:00:50.773531 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-68125063-openshift-default-574c9c8c67-ngglh" podStartSLOduration=1.7735087950000001 podStartE2EDuration="1.773508795s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 18:00:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 18:00:50.770527462 +0000 UTC m=+1699.846985020" watchObservedRunningTime="2026-07-14 18:00:50.773508795 +0000 UTC m=+1699.849966336" Jul 14 18:00:51.666741 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:00:51.666661 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-68125063-openshift-default-574c9c8c67-ngglh" Jul 14 18:00:51.671147 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:00:51.671124 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-68125063-openshift-default-574c9c8c67-ngglh" Jul 14 18:00:51.745594 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:00:51.745574 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-68125063-openshift-default-574c9c8c67-ngglh" Jul 14 18:00:51.746425 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:00:51.746409 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-68125063-openshift-default-574c9c8c67-ngglh" Jul 14 18:00:54.312875 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:00:54.312853 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-439ad9d2-openshift-default-6bf8ffddc-s96dw" Jul 14 18:00:54.383476 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:00:54.383454 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/79c1a410-d02e-467a-b4a4-784e9417d267-istio-data\") pod \"79c1a410-d02e-467a-b4a4-784e9417d267\" (UID: \"79c1a410-d02e-467a-b4a4-784e9417d267\") " Jul 14 18:00:54.383601 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:00:54.383488 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/79c1a410-d02e-467a-b4a4-784e9417d267-istio-envoy\") pod \"79c1a410-d02e-467a-b4a4-784e9417d267\" (UID: \"79c1a410-d02e-467a-b4a4-784e9417d267\") " Jul 14 18:00:54.383601 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:00:54.383540 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/79c1a410-d02e-467a-b4a4-784e9417d267-workload-socket\") pod \"79c1a410-d02e-467a-b4a4-784e9417d267\" (UID: \"79c1a410-d02e-467a-b4a4-784e9417d267\") " Jul 14 18:00:54.383601 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:00:54.383565 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/79c1a410-d02e-467a-b4a4-784e9417d267-istiod-ca-cert\") pod \"79c1a410-d02e-467a-b4a4-784e9417d267\" (UID: \"79c1a410-d02e-467a-b4a4-784e9417d267\") " Jul 14 18:00:54.383762 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:00:54.383602 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/79c1a410-d02e-467a-b4a4-784e9417d267-istio-podinfo\") pod \"79c1a410-d02e-467a-b4a4-784e9417d267\" (UID: \"79c1a410-d02e-467a-b4a4-784e9417d267\") " Jul 14 18:00:54.383762 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:00:54.383627 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/79c1a410-d02e-467a-b4a4-784e9417d267-istio-token\") pod \"79c1a410-d02e-467a-b4a4-784e9417d267\" (UID: \"79c1a410-d02e-467a-b4a4-784e9417d267\") " Jul 14 18:00:54.383762 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:00:54.383692 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-twrd7\" (UniqueName: \"kubernetes.io/projected/79c1a410-d02e-467a-b4a4-784e9417d267-kube-api-access-twrd7\") pod \"79c1a410-d02e-467a-b4a4-784e9417d267\" (UID: \"79c1a410-d02e-467a-b4a4-784e9417d267\") " Jul 14 18:00:54.383916 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:00:54.383780 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/79c1a410-d02e-467a-b4a4-784e9417d267-workload-certs\") pod \"79c1a410-d02e-467a-b4a4-784e9417d267\" (UID: \"79c1a410-d02e-467a-b4a4-784e9417d267\") " Jul 14 18:00:54.383916 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:00:54.383807 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/79c1a410-d02e-467a-b4a4-784e9417d267-credential-socket\") pod \"79c1a410-d02e-467a-b4a4-784e9417d267\" (UID: \"79c1a410-d02e-467a-b4a4-784e9417d267\") " Jul 14 18:00:54.383916 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:00:54.383889 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79c1a410-d02e-467a-b4a4-784e9417d267-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "79c1a410-d02e-467a-b4a4-784e9417d267" (UID: "79c1a410-d02e-467a-b4a4-784e9417d267"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 18:00:54.384078 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:00:54.383971 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/79c1a410-d02e-467a-b4a4-784e9417d267-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "79c1a410-d02e-467a-b4a4-784e9417d267" (UID: "79c1a410-d02e-467a-b4a4-784e9417d267"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 18:00:54.384299 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:00:54.384174 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79c1a410-d02e-467a-b4a4-784e9417d267-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "79c1a410-d02e-467a-b4a4-784e9417d267" (UID: "79c1a410-d02e-467a-b4a4-784e9417d267"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 18:00:54.384299 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:00:54.384262 2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/79c1a410-d02e-467a-b4a4-784e9417d267-workload-socket\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 18:00:54.384299 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:00:54.384285 2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/79c1a410-d02e-467a-b4a4-784e9417d267-istiod-ca-cert\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 18:00:54.384600 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:00:54.384341 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79c1a410-d02e-467a-b4a4-784e9417d267-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "79c1a410-d02e-467a-b4a4-784e9417d267" (UID: "79c1a410-d02e-467a-b4a4-784e9417d267"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 18:00:54.384600 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:00:54.384407 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79c1a410-d02e-467a-b4a4-784e9417d267-istio-data" (OuterVolumeSpecName: "istio-data") pod "79c1a410-d02e-467a-b4a4-784e9417d267" (UID: "79c1a410-d02e-467a-b4a4-784e9417d267"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 18:00:54.386433 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:00:54.386403 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79c1a410-d02e-467a-b4a4-784e9417d267-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "79c1a410-d02e-467a-b4a4-784e9417d267" (UID: "79c1a410-d02e-467a-b4a4-784e9417d267"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 18:00:54.386692 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:00:54.386666 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/79c1a410-d02e-467a-b4a4-784e9417d267-istio-token" (OuterVolumeSpecName: "istio-token") pod "79c1a410-d02e-467a-b4a4-784e9417d267" (UID: "79c1a410-d02e-467a-b4a4-784e9417d267"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 18:00:54.386894 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:00:54.386868 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/79c1a410-d02e-467a-b4a4-784e9417d267-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "79c1a410-d02e-467a-b4a4-784e9417d267" (UID: "79c1a410-d02e-467a-b4a4-784e9417d267"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 14 18:00:54.387164 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:00:54.387135 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/79c1a410-d02e-467a-b4a4-784e9417d267-kube-api-access-twrd7" (OuterVolumeSpecName: "kube-api-access-twrd7") pod "79c1a410-d02e-467a-b4a4-784e9417d267" (UID: "79c1a410-d02e-467a-b4a4-784e9417d267"). InnerVolumeSpecName "kube-api-access-twrd7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 18:00:54.485000 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:00:54.484970 2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/79c1a410-d02e-467a-b4a4-784e9417d267-istio-envoy\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 18:00:54.485000 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:00:54.484994 2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/79c1a410-d02e-467a-b4a4-784e9417d267-istio-podinfo\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 18:00:54.485158 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:00:54.485006 2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/79c1a410-d02e-467a-b4a4-784e9417d267-istio-token\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 18:00:54.485158 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:00:54.485019 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-twrd7\" (UniqueName: \"kubernetes.io/projected/79c1a410-d02e-467a-b4a4-784e9417d267-kube-api-access-twrd7\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 18:00:54.485158 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:00:54.485031 2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/79c1a410-d02e-467a-b4a4-784e9417d267-workload-certs\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 18:00:54.485158 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:00:54.485042 2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/79c1a410-d02e-467a-b4a4-784e9417d267-credential-socket\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 18:00:54.485158 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:00:54.485054 2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/79c1a410-d02e-467a-b4a4-784e9417d267-istio-data\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 18:00:54.765640 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:00:54.765545 2562 generic.go:356] "Generic (PLEG): container finished" podID="79c1a410-d02e-467a-b4a4-784e9417d267" containerID="ffa7bc25debf80b7e1e4b09f861d6f18f5c28b27445c09c3b48f194abade7964" exitCode=0 Jul 14 18:00:54.765795 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:00:54.765648 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-439ad9d2-openshift-default-6bf8ffddc-s96dw" event={"ID":"79c1a410-d02e-467a-b4a4-784e9417d267","Type":"ContainerDied","Data":"ffa7bc25debf80b7e1e4b09f861d6f18f5c28b27445c09c3b48f194abade7964"} Jul 14 18:00:54.765795 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:00:54.765660 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-439ad9d2-openshift-default-6bf8ffddc-s96dw" Jul 14 18:00:54.765795 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:00:54.765729 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-439ad9d2-openshift-default-6bf8ffddc-s96dw" event={"ID":"79c1a410-d02e-467a-b4a4-784e9417d267","Type":"ContainerDied","Data":"89eee1d5102e3a771a8cf45ba430282dc798f538342885a010888ea27d112b4c"} Jul 14 18:00:54.765795 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:00:54.765758 2562 scope.go:117] "RemoveContainer" containerID="ffa7bc25debf80b7e1e4b09f861d6f18f5c28b27445c09c3b48f194abade7964" Jul 14 18:00:54.774960 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:00:54.774945 2562 scope.go:117] "RemoveContainer" containerID="6d96efb4e692e558af7d916d04bf054b443b7d5bfa541d156e01066dd2ec459d" Jul 14 18:00:54.782568 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:00:54.782550 2562 scope.go:117] "RemoveContainer" containerID="ffa7bc25debf80b7e1e4b09f861d6f18f5c28b27445c09c3b48f194abade7964" Jul 14 18:00:54.782809 ip-10-0-129-209 kubenswrapper[2562]: E0714 18:00:54.782790 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ffa7bc25debf80b7e1e4b09f861d6f18f5c28b27445c09c3b48f194abade7964\": container with ID starting with ffa7bc25debf80b7e1e4b09f861d6f18f5c28b27445c09c3b48f194abade7964 not found: ID does not exist" containerID="ffa7bc25debf80b7e1e4b09f861d6f18f5c28b27445c09c3b48f194abade7964" Jul 14 18:00:54.782874 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:00:54.782816 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ffa7bc25debf80b7e1e4b09f861d6f18f5c28b27445c09c3b48f194abade7964"} err="failed to get container status \"ffa7bc25debf80b7e1e4b09f861d6f18f5c28b27445c09c3b48f194abade7964\": rpc error: code = NotFound desc = could not find container \"ffa7bc25debf80b7e1e4b09f861d6f18f5c28b27445c09c3b48f194abade7964\": container with ID starting with ffa7bc25debf80b7e1e4b09f861d6f18f5c28b27445c09c3b48f194abade7964 not found: ID does not exist" Jul 14 18:00:54.782874 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:00:54.782833 2562 scope.go:117] "RemoveContainer" containerID="6d96efb4e692e558af7d916d04bf054b443b7d5bfa541d156e01066dd2ec459d" Jul 14 18:00:54.783076 ip-10-0-129-209 kubenswrapper[2562]: E0714 18:00:54.783058 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6d96efb4e692e558af7d916d04bf054b443b7d5bfa541d156e01066dd2ec459d\": container with ID starting with 6d96efb4e692e558af7d916d04bf054b443b7d5bfa541d156e01066dd2ec459d not found: ID does not exist" containerID="6d96efb4e692e558af7d916d04bf054b443b7d5bfa541d156e01066dd2ec459d" Jul 14 18:00:54.783151 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:00:54.783082 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d96efb4e692e558af7d916d04bf054b443b7d5bfa541d156e01066dd2ec459d"} err="failed to get container status \"6d96efb4e692e558af7d916d04bf054b443b7d5bfa541d156e01066dd2ec459d\": rpc error: code = NotFound desc = could not find container \"6d96efb4e692e558af7d916d04bf054b443b7d5bfa541d156e01066dd2ec459d\": container with ID starting with 6d96efb4e692e558af7d916d04bf054b443b7d5bfa541d156e01066dd2ec459d not found: ID does not exist" Jul 14 18:00:54.787779 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:00:54.787758 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-439ad9d2-openshift-default-6bf8ffddc-s96dw"] Jul 14 18:00:54.792027 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:00:54.792005 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-439ad9d2-openshift-default-6bf8ffddc-s96dw"] Jul 14 18:00:55.519713 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:00:55.519681 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="79c1a410-d02e-467a-b4a4-784e9417d267" path="/var/lib/kubelet/pods/79c1a410-d02e-467a-b4a4-784e9417d267/volumes" Jul 14 18:01:05.814212 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:01:05.814096 2562 generic.go:356] "Generic (PLEG): container finished" podID="533a91ec-be53-4df9-a11b-fd45f75a0c66" containerID="abac056307899e744c50e2a366a29703c43c9efa6921e1e4b7687c942ba9c05a" exitCode=0 Jul 14 18:01:05.814212 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:01:05.814134 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-68125063-openshift-default-574c9c8c67-ngglh" event={"ID":"533a91ec-be53-4df9-a11b-fd45f75a0c66","Type":"ContainerDied","Data":"abac056307899e744c50e2a366a29703c43c9efa6921e1e4b7687c942ba9c05a"} Jul 14 18:01:05.814709 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:01:05.814487 2562 scope.go:117] "RemoveContainer" containerID="abac056307899e744c50e2a366a29703c43c9efa6921e1e4b7687c942ba9c05a" Jul 14 18:01:05.815458 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:01:05.815432 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 14 18:01:05.815532 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:01:05.815502 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 14 18:01:05.815581 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:01:05.815547 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 14 18:01:06.746037 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:01:06.746004 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-68125063-openshift-default-574c9c8c67-ngglh" Jul 14 18:01:06.820527 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:01:06.820488 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-68125063-openshift-default-574c9c8c67-ngglh" event={"ID":"533a91ec-be53-4df9-a11b-fd45f75a0c66","Type":"ContainerStarted","Data":"1bd1b59f453b37eb9e2784e13454b695ae2a97c40658f134feed4d99461f8b12"} Jul 14 18:01:07.666684 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:01:07.666649 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-68125063-openshift-default-574c9c8c67-ngglh" Jul 14 18:01:07.667794 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:01:07.667763 2562 patch_prober.go:28] interesting pod/e2e-derive-68125063-openshift-default-574c9c8c67-ngglh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" start-of-body= Jul 14 18:01:07.667925 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:01:07.667826 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-68125063-openshift-default-574c9c8c67-ngglh" podUID="533a91ec-be53-4df9-a11b-fd45f75a0c66" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" Jul 14 18:01:08.675215 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:01:08.672586 2562 patch_prober.go:28] interesting pod/e2e-derive-68125063-openshift-default-574c9c8c67-ngglh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" start-of-body= Jul 14 18:01:08.675215 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:01:08.672644 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-68125063-openshift-default-574c9c8c67-ngglh" podUID="533a91ec-be53-4df9-a11b-fd45f75a0c66" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" Jul 14 18:01:09.667298 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:01:09.667260 2562 patch_prober.go:28] interesting pod/e2e-derive-68125063-openshift-default-574c9c8c67-ngglh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" start-of-body= Jul 14 18:01:09.667611 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:01:09.667581 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-68125063-openshift-default-574c9c8c67-ngglh" podUID="533a91ec-be53-4df9-a11b-fd45f75a0c66" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" Jul 14 18:01:10.666600 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:01:10.666569 2562 patch_prober.go:28] interesting pod/e2e-derive-68125063-openshift-default-574c9c8c67-ngglh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" start-of-body= Jul 14 18:01:10.667164 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:01:10.667134 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-68125063-openshift-default-574c9c8c67-ngglh" podUID="533a91ec-be53-4df9-a11b-fd45f75a0c66" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.30:15021/healthz/ready\": dial tcp 10.134.0.30:15021: connect: connection refused" Jul 14 18:01:11.671138 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:01:11.671104 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-68125063-openshift-default-574c9c8c67-ngglh" Jul 14 18:01:11.671582 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:01:11.671414 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-68125063-openshift-default-574c9c8c67-ngglh" Jul 14 18:01:11.672131 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:01:11.672113 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-68125063-openshift-default-574c9c8c67-ngglh" Jul 14 18:02:20.256139 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:02:20.256099 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-68125063-openshift-default-574c9c8c67-ngglh"] Jul 14 18:02:20.256688 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:02:20.256313 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-68125063-openshift-default-574c9c8c67-ngglh" podUID="533a91ec-be53-4df9-a11b-fd45f75a0c66" containerName="istio-proxy" containerID="cri-o://1bd1b59f453b37eb9e2784e13454b695ae2a97c40658f134feed4d99461f8b12" gracePeriod=30 Jul 14 18:02:25.501800 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:02:25.501774 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-68125063-openshift-default-574c9c8c67-ngglh" Jul 14 18:02:25.660940 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:02:25.660908 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/533a91ec-be53-4df9-a11b-fd45f75a0c66-workload-certs\") pod \"533a91ec-be53-4df9-a11b-fd45f75a0c66\" (UID: \"533a91ec-be53-4df9-a11b-fd45f75a0c66\") " Jul 14 18:02:25.661108 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:02:25.660950 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/533a91ec-be53-4df9-a11b-fd45f75a0c66-credential-socket\") pod \"533a91ec-be53-4df9-a11b-fd45f75a0c66\" (UID: \"533a91ec-be53-4df9-a11b-fd45f75a0c66\") " Jul 14 18:02:25.661108 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:02:25.661009 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/533a91ec-be53-4df9-a11b-fd45f75a0c66-istiod-ca-cert\") pod \"533a91ec-be53-4df9-a11b-fd45f75a0c66\" (UID: \"533a91ec-be53-4df9-a11b-fd45f75a0c66\") " Jul 14 18:02:25.661108 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:02:25.661043 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ndm4m\" (UniqueName: \"kubernetes.io/projected/533a91ec-be53-4df9-a11b-fd45f75a0c66-kube-api-access-ndm4m\") pod \"533a91ec-be53-4df9-a11b-fd45f75a0c66\" (UID: \"533a91ec-be53-4df9-a11b-fd45f75a0c66\") " Jul 14 18:02:25.661108 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:02:25.661078 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/533a91ec-be53-4df9-a11b-fd45f75a0c66-workload-socket\") pod \"533a91ec-be53-4df9-a11b-fd45f75a0c66\" (UID: \"533a91ec-be53-4df9-a11b-fd45f75a0c66\") " Jul 14 18:02:25.661108 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:02:25.661104 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/533a91ec-be53-4df9-a11b-fd45f75a0c66-istio-data\") pod \"533a91ec-be53-4df9-a11b-fd45f75a0c66\" (UID: \"533a91ec-be53-4df9-a11b-fd45f75a0c66\") " Jul 14 18:02:25.661404 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:02:25.661128 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/533a91ec-be53-4df9-a11b-fd45f75a0c66-istio-envoy\") pod \"533a91ec-be53-4df9-a11b-fd45f75a0c66\" (UID: \"533a91ec-be53-4df9-a11b-fd45f75a0c66\") " Jul 14 18:02:25.661404 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:02:25.661225 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/533a91ec-be53-4df9-a11b-fd45f75a0c66-istio-token\") pod \"533a91ec-be53-4df9-a11b-fd45f75a0c66\" (UID: \"533a91ec-be53-4df9-a11b-fd45f75a0c66\") " Jul 14 18:02:25.661404 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:02:25.661229 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/533a91ec-be53-4df9-a11b-fd45f75a0c66-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "533a91ec-be53-4df9-a11b-fd45f75a0c66" (UID: "533a91ec-be53-4df9-a11b-fd45f75a0c66"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 18:02:25.661404 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:02:25.661263 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/533a91ec-be53-4df9-a11b-fd45f75a0c66-istio-podinfo\") pod \"533a91ec-be53-4df9-a11b-fd45f75a0c66\" (UID: \"533a91ec-be53-4df9-a11b-fd45f75a0c66\") " Jul 14 18:02:25.661404 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:02:25.661325 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/533a91ec-be53-4df9-a11b-fd45f75a0c66-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "533a91ec-be53-4df9-a11b-fd45f75a0c66" (UID: "533a91ec-be53-4df9-a11b-fd45f75a0c66"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 18:02:25.661676 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:02:25.661395 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/533a91ec-be53-4df9-a11b-fd45f75a0c66-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "533a91ec-be53-4df9-a11b-fd45f75a0c66" (UID: "533a91ec-be53-4df9-a11b-fd45f75a0c66"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 18:02:25.661676 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:02:25.661465 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/533a91ec-be53-4df9-a11b-fd45f75a0c66-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "533a91ec-be53-4df9-a11b-fd45f75a0c66" (UID: "533a91ec-be53-4df9-a11b-fd45f75a0c66"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 18:02:25.661676 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:02:25.661619 2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/533a91ec-be53-4df9-a11b-fd45f75a0c66-istiod-ca-cert\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 18:02:25.661676 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:02:25.661638 2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/533a91ec-be53-4df9-a11b-fd45f75a0c66-workload-socket\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 18:02:25.661676 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:02:25.661651 2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/533a91ec-be53-4df9-a11b-fd45f75a0c66-workload-certs\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 18:02:25.661676 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:02:25.661664 2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/533a91ec-be53-4df9-a11b-fd45f75a0c66-credential-socket\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 18:02:25.662228 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:02:25.662176 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/533a91ec-be53-4df9-a11b-fd45f75a0c66-istio-data" (OuterVolumeSpecName: "istio-data") pod "533a91ec-be53-4df9-a11b-fd45f75a0c66" (UID: "533a91ec-be53-4df9-a11b-fd45f75a0c66"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 18:02:25.663426 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:02:25.663397 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/533a91ec-be53-4df9-a11b-fd45f75a0c66-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "533a91ec-be53-4df9-a11b-fd45f75a0c66" (UID: "533a91ec-be53-4df9-a11b-fd45f75a0c66"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 14 18:02:25.663426 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:02:25.663408 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/533a91ec-be53-4df9-a11b-fd45f75a0c66-kube-api-access-ndm4m" (OuterVolumeSpecName: "kube-api-access-ndm4m") pod "533a91ec-be53-4df9-a11b-fd45f75a0c66" (UID: "533a91ec-be53-4df9-a11b-fd45f75a0c66"). InnerVolumeSpecName "kube-api-access-ndm4m". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 18:02:25.663573 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:02:25.663456 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/533a91ec-be53-4df9-a11b-fd45f75a0c66-istio-token" (OuterVolumeSpecName: "istio-token") pod "533a91ec-be53-4df9-a11b-fd45f75a0c66" (UID: "533a91ec-be53-4df9-a11b-fd45f75a0c66"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 18:02:25.663573 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:02:25.663525 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/533a91ec-be53-4df9-a11b-fd45f75a0c66-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "533a91ec-be53-4df9-a11b-fd45f75a0c66" (UID: "533a91ec-be53-4df9-a11b-fd45f75a0c66"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 18:02:25.762881 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:02:25.762852 2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/533a91ec-be53-4df9-a11b-fd45f75a0c66-istio-data\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 18:02:25.762881 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:02:25.762874 2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/533a91ec-be53-4df9-a11b-fd45f75a0c66-istio-envoy\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 18:02:25.762881 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:02:25.762883 2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/533a91ec-be53-4df9-a11b-fd45f75a0c66-istio-token\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 18:02:25.763076 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:02:25.762892 2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/533a91ec-be53-4df9-a11b-fd45f75a0c66-istio-podinfo\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 18:02:25.763076 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:02:25.762900 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ndm4m\" (UniqueName: \"kubernetes.io/projected/533a91ec-be53-4df9-a11b-fd45f75a0c66-kube-api-access-ndm4m\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 18:02:26.168887 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:02:26.168848 2562 generic.go:356] "Generic (PLEG): container finished" podID="533a91ec-be53-4df9-a11b-fd45f75a0c66" containerID="1bd1b59f453b37eb9e2784e13454b695ae2a97c40658f134feed4d99461f8b12" exitCode=0 Jul 14 18:02:26.169053 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:02:26.168899 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-68125063-openshift-default-574c9c8c67-ngglh" Jul 14 18:02:26.169053 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:02:26.168898 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-68125063-openshift-default-574c9c8c67-ngglh" event={"ID":"533a91ec-be53-4df9-a11b-fd45f75a0c66","Type":"ContainerDied","Data":"1bd1b59f453b37eb9e2784e13454b695ae2a97c40658f134feed4d99461f8b12"} Jul 14 18:02:26.169053 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:02:26.168955 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-68125063-openshift-default-574c9c8c67-ngglh" event={"ID":"533a91ec-be53-4df9-a11b-fd45f75a0c66","Type":"ContainerDied","Data":"bdfb9d87829bde3e22f2926c5333c451c428ccbf6f5f71a505f6e3bbdea3dfbc"} Jul 14 18:02:26.169053 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:02:26.168979 2562 scope.go:117] "RemoveContainer" containerID="1bd1b59f453b37eb9e2784e13454b695ae2a97c40658f134feed4d99461f8b12" Jul 14 18:02:26.177674 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:02:26.177657 2562 scope.go:117] "RemoveContainer" containerID="abac056307899e744c50e2a366a29703c43c9efa6921e1e4b7687c942ba9c05a" Jul 14 18:02:26.186991 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:02:26.186975 2562 scope.go:117] "RemoveContainer" containerID="1bd1b59f453b37eb9e2784e13454b695ae2a97c40658f134feed4d99461f8b12" Jul 14 18:02:26.187471 ip-10-0-129-209 kubenswrapper[2562]: E0714 18:02:26.187453 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1bd1b59f453b37eb9e2784e13454b695ae2a97c40658f134feed4d99461f8b12\": container with ID starting with 1bd1b59f453b37eb9e2784e13454b695ae2a97c40658f134feed4d99461f8b12 not found: ID does not exist" containerID="1bd1b59f453b37eb9e2784e13454b695ae2a97c40658f134feed4d99461f8b12" Jul 14 18:02:26.187552 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:02:26.187477 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1bd1b59f453b37eb9e2784e13454b695ae2a97c40658f134feed4d99461f8b12"} err="failed to get container status \"1bd1b59f453b37eb9e2784e13454b695ae2a97c40658f134feed4d99461f8b12\": rpc error: code = NotFound desc = could not find container \"1bd1b59f453b37eb9e2784e13454b695ae2a97c40658f134feed4d99461f8b12\": container with ID starting with 1bd1b59f453b37eb9e2784e13454b695ae2a97c40658f134feed4d99461f8b12 not found: ID does not exist" Jul 14 18:02:26.187552 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:02:26.187497 2562 scope.go:117] "RemoveContainer" containerID="abac056307899e744c50e2a366a29703c43c9efa6921e1e4b7687c942ba9c05a" Jul 14 18:02:26.187733 ip-10-0-129-209 kubenswrapper[2562]: E0714 18:02:26.187716 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"abac056307899e744c50e2a366a29703c43c9efa6921e1e4b7687c942ba9c05a\": container with ID starting with abac056307899e744c50e2a366a29703c43c9efa6921e1e4b7687c942ba9c05a not found: ID does not exist" containerID="abac056307899e744c50e2a366a29703c43c9efa6921e1e4b7687c942ba9c05a" Jul 14 18:02:26.187783 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:02:26.187740 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"abac056307899e744c50e2a366a29703c43c9efa6921e1e4b7687c942ba9c05a"} err="failed to get container status \"abac056307899e744c50e2a366a29703c43c9efa6921e1e4b7687c942ba9c05a\": rpc error: code = NotFound desc = could not find container \"abac056307899e744c50e2a366a29703c43c9efa6921e1e4b7687c942ba9c05a\": container with ID starting with abac056307899e744c50e2a366a29703c43c9efa6921e1e4b7687c942ba9c05a not found: ID does not exist" Jul 14 18:02:26.201808 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:02:26.201784 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-68125063-openshift-default-574c9c8c67-ngglh"] Jul 14 18:02:26.205365 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:02:26.205343 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-68125063-openshift-default-574c9c8c67-ngglh"] Jul 14 18:02:27.518825 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:02:27.518748 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="533a91ec-be53-4df9-a11b-fd45f75a0c66" path="/var/lib/kubelet/pods/533a91ec-be53-4df9-a11b-fd45f75a0c66/volumes" Jul 14 18:02:31.475503 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:02:31.475470 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2sbww_c3d00b23-d0d5-4d18-a722-464d7b114745/ovn-acl-logging/0.log" Jul 14 18:02:31.477675 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:02:31.477652 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2sbww_c3d00b23-d0d5-4d18-a722-464d7b114745/ovn-acl-logging/0.log" Jul 14 18:04:02.755401 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:04:02.755325 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-8cb2afcb-openshift-default-d47c5d887-tlvgp"] Jul 14 18:04:02.755818 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:04:02.755639 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="533a91ec-be53-4df9-a11b-fd45f75a0c66" containerName="istio-proxy" Jul 14 18:04:02.755818 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:04:02.755649 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="533a91ec-be53-4df9-a11b-fd45f75a0c66" containerName="istio-proxy" Jul 14 18:04:02.755818 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:04:02.755659 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="79c1a410-d02e-467a-b4a4-784e9417d267" containerName="istio-proxy" Jul 14 18:04:02.755818 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:04:02.755664 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="79c1a410-d02e-467a-b4a4-784e9417d267" containerName="istio-proxy" Jul 14 18:04:02.755818 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:04:02.755684 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="79c1a410-d02e-467a-b4a4-784e9417d267" containerName="istio-proxy" Jul 14 18:04:02.755818 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:04:02.755689 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="79c1a410-d02e-467a-b4a4-784e9417d267" containerName="istio-proxy" Jul 14 18:04:02.755818 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:04:02.755737 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="533a91ec-be53-4df9-a11b-fd45f75a0c66" containerName="istio-proxy" Jul 14 18:04:02.755818 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:04:02.755744 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="533a91ec-be53-4df9-a11b-fd45f75a0c66" containerName="istio-proxy" Jul 14 18:04:02.755818 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:04:02.755749 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="79c1a410-d02e-467a-b4a4-784e9417d267" containerName="istio-proxy" Jul 14 18:04:02.755818 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:04:02.755756 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="79c1a410-d02e-467a-b4a4-784e9417d267" containerName="istio-proxy" Jul 14 18:04:02.757869 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:04:02.757848 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-8cb2afcb-openshift-default-d47c5d887-tlvgp" Jul 14 18:04:02.761173 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:04:02.761156 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-8cb2afcb-openshift-default-dockercfg-fzr7k\"" Jul 14 18:04:02.864068 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:04:02.864039 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8f26454-670d-4db6-aac0-a195513f89a6-workload-socket\") pod \"e2e-mt-8cb2afcb-openshift-default-d47c5d887-tlvgp\" (UID: \"c8f26454-670d-4db6-aac0-a195513f89a6\") " pod="openshift-ingress/e2e-mt-8cb2afcb-openshift-default-d47c5d887-tlvgp" Jul 14 18:04:02.864269 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:04:02.864073 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c8f26454-670d-4db6-aac0-a195513f89a6-istio-envoy\") pod \"e2e-mt-8cb2afcb-openshift-default-d47c5d887-tlvgp\" (UID: \"c8f26454-670d-4db6-aac0-a195513f89a6\") " pod="openshift-ingress/e2e-mt-8cb2afcb-openshift-default-d47c5d887-tlvgp" Jul 14 18:04:02.864269 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:04:02.864108 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8f26454-670d-4db6-aac0-a195513f89a6-credential-socket\") pod \"e2e-mt-8cb2afcb-openshift-default-d47c5d887-tlvgp\" (UID: \"c8f26454-670d-4db6-aac0-a195513f89a6\") " pod="openshift-ingress/e2e-mt-8cb2afcb-openshift-default-d47c5d887-tlvgp" Jul 14 18:04:02.864269 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:04:02.864160 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c8f26454-670d-4db6-aac0-a195513f89a6-istio-data\") pod \"e2e-mt-8cb2afcb-openshift-default-d47c5d887-tlvgp\" (UID: \"c8f26454-670d-4db6-aac0-a195513f89a6\") " pod="openshift-ingress/e2e-mt-8cb2afcb-openshift-default-d47c5d887-tlvgp" Jul 14 18:04:02.864269 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:04:02.864177 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c8f26454-670d-4db6-aac0-a195513f89a6-istio-token\") pod \"e2e-mt-8cb2afcb-openshift-default-d47c5d887-tlvgp\" (UID: \"c8f26454-670d-4db6-aac0-a195513f89a6\") " pod="openshift-ingress/e2e-mt-8cb2afcb-openshift-default-d47c5d887-tlvgp" Jul 14 18:04:02.864269 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:04:02.864219 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vm6rn\" (UniqueName: \"kubernetes.io/projected/c8f26454-670d-4db6-aac0-a195513f89a6-kube-api-access-vm6rn\") pod \"e2e-mt-8cb2afcb-openshift-default-d47c5d887-tlvgp\" (UID: \"c8f26454-670d-4db6-aac0-a195513f89a6\") " pod="openshift-ingress/e2e-mt-8cb2afcb-openshift-default-d47c5d887-tlvgp" Jul 14 18:04:02.864269 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:04:02.864238 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c8f26454-670d-4db6-aac0-a195513f89a6-istiod-ca-cert\") pod \"e2e-mt-8cb2afcb-openshift-default-d47c5d887-tlvgp\" (UID: \"c8f26454-670d-4db6-aac0-a195513f89a6\") " pod="openshift-ingress/e2e-mt-8cb2afcb-openshift-default-d47c5d887-tlvgp" Jul 14 18:04:02.864491 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:04:02.864280 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c8f26454-670d-4db6-aac0-a195513f89a6-istio-podinfo\") pod \"e2e-mt-8cb2afcb-openshift-default-d47c5d887-tlvgp\" (UID: \"c8f26454-670d-4db6-aac0-a195513f89a6\") " pod="openshift-ingress/e2e-mt-8cb2afcb-openshift-default-d47c5d887-tlvgp" Jul 14 18:04:02.864491 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:04:02.864320 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c8f26454-670d-4db6-aac0-a195513f89a6-workload-certs\") pod \"e2e-mt-8cb2afcb-openshift-default-d47c5d887-tlvgp\" (UID: \"c8f26454-670d-4db6-aac0-a195513f89a6\") " pod="openshift-ingress/e2e-mt-8cb2afcb-openshift-default-d47c5d887-tlvgp" Jul 14 18:04:02.890229 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:04:02.890175 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-8cb2afcb-openshift-default-d47c5d887-tlvgp"] Jul 14 18:04:02.965711 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:04:02.965679 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c8f26454-670d-4db6-aac0-a195513f89a6-istio-podinfo\") pod \"e2e-mt-8cb2afcb-openshift-default-d47c5d887-tlvgp\" (UID: \"c8f26454-670d-4db6-aac0-a195513f89a6\") " pod="openshift-ingress/e2e-mt-8cb2afcb-openshift-default-d47c5d887-tlvgp" Jul 14 18:04:02.965881 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:04:02.965729 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c8f26454-670d-4db6-aac0-a195513f89a6-workload-certs\") pod \"e2e-mt-8cb2afcb-openshift-default-d47c5d887-tlvgp\" (UID: \"c8f26454-670d-4db6-aac0-a195513f89a6\") " pod="openshift-ingress/e2e-mt-8cb2afcb-openshift-default-d47c5d887-tlvgp" Jul 14 18:04:02.965881 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:04:02.965759 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8f26454-670d-4db6-aac0-a195513f89a6-workload-socket\") pod \"e2e-mt-8cb2afcb-openshift-default-d47c5d887-tlvgp\" (UID: \"c8f26454-670d-4db6-aac0-a195513f89a6\") " pod="openshift-ingress/e2e-mt-8cb2afcb-openshift-default-d47c5d887-tlvgp" Jul 14 18:04:02.965881 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:04:02.965785 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c8f26454-670d-4db6-aac0-a195513f89a6-istio-envoy\") pod \"e2e-mt-8cb2afcb-openshift-default-d47c5d887-tlvgp\" (UID: \"c8f26454-670d-4db6-aac0-a195513f89a6\") " pod="openshift-ingress/e2e-mt-8cb2afcb-openshift-default-d47c5d887-tlvgp" Jul 14 18:04:02.965881 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:04:02.965824 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8f26454-670d-4db6-aac0-a195513f89a6-credential-socket\") pod \"e2e-mt-8cb2afcb-openshift-default-d47c5d887-tlvgp\" (UID: \"c8f26454-670d-4db6-aac0-a195513f89a6\") " pod="openshift-ingress/e2e-mt-8cb2afcb-openshift-default-d47c5d887-tlvgp" Jul 14 18:04:02.965881 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:04:02.965855 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c8f26454-670d-4db6-aac0-a195513f89a6-istio-data\") pod \"e2e-mt-8cb2afcb-openshift-default-d47c5d887-tlvgp\" (UID: \"c8f26454-670d-4db6-aac0-a195513f89a6\") " pod="openshift-ingress/e2e-mt-8cb2afcb-openshift-default-d47c5d887-tlvgp" Jul 14 18:04:02.965881 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:04:02.965880 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c8f26454-670d-4db6-aac0-a195513f89a6-istio-token\") pod \"e2e-mt-8cb2afcb-openshift-default-d47c5d887-tlvgp\" (UID: \"c8f26454-670d-4db6-aac0-a195513f89a6\") " pod="openshift-ingress/e2e-mt-8cb2afcb-openshift-default-d47c5d887-tlvgp" Jul 14 18:04:02.966144 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:04:02.965917 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vm6rn\" (UniqueName: \"kubernetes.io/projected/c8f26454-670d-4db6-aac0-a195513f89a6-kube-api-access-vm6rn\") pod \"e2e-mt-8cb2afcb-openshift-default-d47c5d887-tlvgp\" (UID: \"c8f26454-670d-4db6-aac0-a195513f89a6\") " pod="openshift-ingress/e2e-mt-8cb2afcb-openshift-default-d47c5d887-tlvgp" Jul 14 18:04:02.966144 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:04:02.965943 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c8f26454-670d-4db6-aac0-a195513f89a6-istiod-ca-cert\") pod \"e2e-mt-8cb2afcb-openshift-default-d47c5d887-tlvgp\" (UID: \"c8f26454-670d-4db6-aac0-a195513f89a6\") " pod="openshift-ingress/e2e-mt-8cb2afcb-openshift-default-d47c5d887-tlvgp" Jul 14 18:04:02.966447 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:04:02.966396 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c8f26454-670d-4db6-aac0-a195513f89a6-workload-certs\") pod \"e2e-mt-8cb2afcb-openshift-default-d47c5d887-tlvgp\" (UID: \"c8f26454-670d-4db6-aac0-a195513f89a6\") " pod="openshift-ingress/e2e-mt-8cb2afcb-openshift-default-d47c5d887-tlvgp" Jul 14 18:04:02.966447 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:04:02.966431 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c8f26454-670d-4db6-aac0-a195513f89a6-istio-data\") pod \"e2e-mt-8cb2afcb-openshift-default-d47c5d887-tlvgp\" (UID: \"c8f26454-670d-4db6-aac0-a195513f89a6\") " pod="openshift-ingress/e2e-mt-8cb2afcb-openshift-default-d47c5d887-tlvgp" Jul 14 18:04:02.966625 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:04:02.966542 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8f26454-670d-4db6-aac0-a195513f89a6-workload-socket\") pod \"e2e-mt-8cb2afcb-openshift-default-d47c5d887-tlvgp\" (UID: \"c8f26454-670d-4db6-aac0-a195513f89a6\") " pod="openshift-ingress/e2e-mt-8cb2afcb-openshift-default-d47c5d887-tlvgp" Jul 14 18:04:02.966870 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:04:02.966850 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c8f26454-670d-4db6-aac0-a195513f89a6-istiod-ca-cert\") pod \"e2e-mt-8cb2afcb-openshift-default-d47c5d887-tlvgp\" (UID: \"c8f26454-670d-4db6-aac0-a195513f89a6\") " pod="openshift-ingress/e2e-mt-8cb2afcb-openshift-default-d47c5d887-tlvgp" Jul 14 18:04:02.966960 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:04:02.966875 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8f26454-670d-4db6-aac0-a195513f89a6-credential-socket\") pod \"e2e-mt-8cb2afcb-openshift-default-d47c5d887-tlvgp\" (UID: \"c8f26454-670d-4db6-aac0-a195513f89a6\") " pod="openshift-ingress/e2e-mt-8cb2afcb-openshift-default-d47c5d887-tlvgp" Jul 14 18:04:02.968410 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:04:02.968385 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c8f26454-670d-4db6-aac0-a195513f89a6-istio-envoy\") pod \"e2e-mt-8cb2afcb-openshift-default-d47c5d887-tlvgp\" (UID: \"c8f26454-670d-4db6-aac0-a195513f89a6\") " pod="openshift-ingress/e2e-mt-8cb2afcb-openshift-default-d47c5d887-tlvgp" Jul 14 18:04:02.968496 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:04:02.968451 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c8f26454-670d-4db6-aac0-a195513f89a6-istio-podinfo\") pod \"e2e-mt-8cb2afcb-openshift-default-d47c5d887-tlvgp\" (UID: \"c8f26454-670d-4db6-aac0-a195513f89a6\") " pod="openshift-ingress/e2e-mt-8cb2afcb-openshift-default-d47c5d887-tlvgp" Jul 14 18:04:03.005171 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:04:03.005143 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c8f26454-670d-4db6-aac0-a195513f89a6-istio-token\") pod \"e2e-mt-8cb2afcb-openshift-default-d47c5d887-tlvgp\" (UID: \"c8f26454-670d-4db6-aac0-a195513f89a6\") " pod="openshift-ingress/e2e-mt-8cb2afcb-openshift-default-d47c5d887-tlvgp" Jul 14 18:04:03.018385 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:04:03.018321 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vm6rn\" (UniqueName: \"kubernetes.io/projected/c8f26454-670d-4db6-aac0-a195513f89a6-kube-api-access-vm6rn\") pod \"e2e-mt-8cb2afcb-openshift-default-d47c5d887-tlvgp\" (UID: \"c8f26454-670d-4db6-aac0-a195513f89a6\") " pod="openshift-ingress/e2e-mt-8cb2afcb-openshift-default-d47c5d887-tlvgp" Jul 14 18:04:03.067315 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:04:03.067277 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-8cb2afcb-openshift-default-d47c5d887-tlvgp" Jul 14 18:04:03.241836 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:04:03.241813 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-8cb2afcb-openshift-default-d47c5d887-tlvgp"] Jul 14 18:04:03.242463 ip-10-0-129-209 kubenswrapper[2562]: W0714 18:04:03.242438 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8f26454_670d_4db6_aac0_a195513f89a6.slice/crio-1e86867a16a858686068920f7be45410c5b08659aa1a54195cc0e6f06ea83245 WatchSource:0}: Error finding container 1e86867a16a858686068920f7be45410c5b08659aa1a54195cc0e6f06ea83245: Status 404 returned error can't find the container with id 1e86867a16a858686068920f7be45410c5b08659aa1a54195cc0e6f06ea83245 Jul 14 18:04:03.244555 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:04:03.244522 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 14 18:04:03.244642 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:04:03.244583 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 14 18:04:03.244642 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:04:03.244610 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 14 18:04:03.578456 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:04:03.578375 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8cb2afcb-openshift-default-d47c5d887-tlvgp" event={"ID":"c8f26454-670d-4db6-aac0-a195513f89a6","Type":"ContainerStarted","Data":"3d957ed42d34556cb56faa3f6a5f2a4ff2557d4cb350defcb8984c64edff56f3"} Jul 14 18:04:03.578456 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:04:03.578422 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8cb2afcb-openshift-default-d47c5d887-tlvgp" event={"ID":"c8f26454-670d-4db6-aac0-a195513f89a6","Type":"ContainerStarted","Data":"1e86867a16a858686068920f7be45410c5b08659aa1a54195cc0e6f06ea83245"} Jul 14 18:04:03.625365 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:04:03.625310 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-8cb2afcb-openshift-default-d47c5d887-tlvgp" podStartSLOduration=1.625290557 podStartE2EDuration="1.625290557s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 18:04:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 18:04:03.624234594 +0000 UTC m=+1892.700692133" watchObservedRunningTime="2026-07-14 18:04:03.625290557 +0000 UTC m=+1892.701748098" Jul 14 18:04:04.067933 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:04:04.067896 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-8cb2afcb-openshift-default-d47c5d887-tlvgp" Jul 14 18:04:04.072955 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:04:04.072929 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-8cb2afcb-openshift-default-d47c5d887-tlvgp" Jul 14 18:04:04.585017 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:04:04.584984 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-8cb2afcb-openshift-default-d47c5d887-tlvgp" Jul 14 18:04:04.588703 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:04:04.588676 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-8cb2afcb-openshift-default-d47c5d887-tlvgp" Jul 14 18:04:20.211356 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:04:20.211105 2562 patch_prober.go:28] interesting pod/e2e-mt-8cb2afcb-openshift-default-d47c5d887-tlvgp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": read tcp 10.134.0.2:57718->10.134.0.31:15021: read: connection reset by peer" start-of-body= Jul 14 18:04:20.211356 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:04:20.211201 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-8cb2afcb-openshift-default-d47c5d887-tlvgp" podUID="c8f26454-670d-4db6-aac0-a195513f89a6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": read tcp 10.134.0.2:57718->10.134.0.31:15021: read: connection reset by peer" Jul 14 18:04:20.653699 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:04:20.653667 2562 generic.go:356] "Generic (PLEG): container finished" podID="c8f26454-670d-4db6-aac0-a195513f89a6" containerID="3d957ed42d34556cb56faa3f6a5f2a4ff2557d4cb350defcb8984c64edff56f3" exitCode=0 Jul 14 18:04:20.653857 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:04:20.653748 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8cb2afcb-openshift-default-d47c5d887-tlvgp" event={"ID":"c8f26454-670d-4db6-aac0-a195513f89a6","Type":"ContainerDied","Data":"3d957ed42d34556cb56faa3f6a5f2a4ff2557d4cb350defcb8984c64edff56f3"} Jul 14 18:04:20.654161 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:04:20.654146 2562 scope.go:117] "RemoveContainer" containerID="3d957ed42d34556cb56faa3f6a5f2a4ff2557d4cb350defcb8984c64edff56f3" Jul 14 18:04:20.655131 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:04:20.655107 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 14 18:04:20.655210 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:04:20.655170 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 14 18:04:20.655255 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:04:20.655218 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 14 18:04:21.659357 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:04:21.659320 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8cb2afcb-openshift-default-d47c5d887-tlvgp" event={"ID":"c8f26454-670d-4db6-aac0-a195513f89a6","Type":"ContainerStarted","Data":"2b9e7cfb778108c61b33a003db504339eef065c5995d7a35cfd55cee9a0986aa"} Jul 14 18:04:22.067798 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:04:22.067721 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-8cb2afcb-openshift-default-d47c5d887-tlvgp" Jul 14 18:04:22.067965 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:04:22.067890 2562 patch_prober.go:28] interesting pod/e2e-mt-8cb2afcb-openshift-default-d47c5d887-tlvgp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body= Jul 14 18:04:22.067965 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:04:22.067938 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-8cb2afcb-openshift-default-d47c5d887-tlvgp" podUID="c8f26454-670d-4db6-aac0-a195513f89a6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" Jul 14 18:04:23.067972 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:04:23.067938 2562 patch_prober.go:28] interesting pod/e2e-mt-8cb2afcb-openshift-default-d47c5d887-tlvgp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body= Jul 14 18:04:23.068373 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:04:23.067992 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-8cb2afcb-openshift-default-d47c5d887-tlvgp" podUID="c8f26454-670d-4db6-aac0-a195513f89a6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" Jul 14 18:04:24.068624 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:04:24.068579 2562 patch_prober.go:28] interesting pod/e2e-mt-8cb2afcb-openshift-default-d47c5d887-tlvgp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body= Jul 14 18:04:24.069141 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:04:24.068646 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-8cb2afcb-openshift-default-d47c5d887-tlvgp" podUID="c8f26454-670d-4db6-aac0-a195513f89a6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" Jul 14 18:04:25.068624 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:04:25.068579 2562 patch_prober.go:28] interesting pod/e2e-mt-8cb2afcb-openshift-default-d47c5d887-tlvgp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body= Jul 14 18:04:25.068826 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:04:25.068645 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-8cb2afcb-openshift-default-d47c5d887-tlvgp" podUID="c8f26454-670d-4db6-aac0-a195513f89a6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" Jul 14 18:04:26.068504 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:04:26.068465 2562 patch_prober.go:28] interesting pod/e2e-mt-8cb2afcb-openshift-default-d47c5d887-tlvgp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body= Jul 14 18:04:26.068794 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:04:26.068766 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-8cb2afcb-openshift-default-d47c5d887-tlvgp" podUID="c8f26454-670d-4db6-aac0-a195513f89a6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" Jul 14 18:04:27.068726 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:04:27.068687 2562 patch_prober.go:28] interesting pod/e2e-mt-8cb2afcb-openshift-default-d47c5d887-tlvgp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" start-of-body= Jul 14 18:04:27.069243 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:04:27.068759 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-8cb2afcb-openshift-default-d47c5d887-tlvgp" podUID="c8f26454-670d-4db6-aac0-a195513f89a6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.31:15021/healthz/ready\": dial tcp 10.134.0.31:15021: connect: connection refused" Jul 14 18:04:28.071122 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:04:28.071084 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-8cb2afcb-openshift-default-d47c5d887-tlvgp" Jul 14 18:04:28.071562 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:04:28.071451 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-8cb2afcb-openshift-default-d47c5d887-tlvgp" Jul 14 18:04:28.071977 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:04:28.071960 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-8cb2afcb-openshift-default-d47c5d887-tlvgp" Jul 14 18:04:44.874468 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:04:44.874435 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-8cb2afcb-openshift-default-d47c5d887-tlvgp"] Jul 14 18:04:44.874957 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:04:44.874656 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-8cb2afcb-openshift-default-d47c5d887-tlvgp" podUID="c8f26454-670d-4db6-aac0-a195513f89a6" containerName="istio-proxy" containerID="cri-o://2b9e7cfb778108c61b33a003db504339eef065c5995d7a35cfd55cee9a0986aa" gracePeriod=30 Jul 14 18:04:50.114452 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:04:50.114428 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-8cb2afcb-openshift-default-d47c5d887-tlvgp" Jul 14 18:04:50.169448 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:04:50.169374 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8f26454-670d-4db6-aac0-a195513f89a6-credential-socket\") pod \"c8f26454-670d-4db6-aac0-a195513f89a6\" (UID: \"c8f26454-670d-4db6-aac0-a195513f89a6\") " Jul 14 18:04:50.169448 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:04:50.169409 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c8f26454-670d-4db6-aac0-a195513f89a6-workload-certs\") pod \"c8f26454-670d-4db6-aac0-a195513f89a6\" (UID: \"c8f26454-670d-4db6-aac0-a195513f89a6\") " Jul 14 18:04:50.169613 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:04:50.169478 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vm6rn\" (UniqueName: \"kubernetes.io/projected/c8f26454-670d-4db6-aac0-a195513f89a6-kube-api-access-vm6rn\") pod \"c8f26454-670d-4db6-aac0-a195513f89a6\" (UID: \"c8f26454-670d-4db6-aac0-a195513f89a6\") " Jul 14 18:04:50.169613 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:04:50.169506 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8f26454-670d-4db6-aac0-a195513f89a6-workload-socket\") pod \"c8f26454-670d-4db6-aac0-a195513f89a6\" (UID: \"c8f26454-670d-4db6-aac0-a195513f89a6\") " Jul 14 18:04:50.169613 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:04:50.169542 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c8f26454-670d-4db6-aac0-a195513f89a6-istio-data\") pod \"c8f26454-670d-4db6-aac0-a195513f89a6\" (UID: \"c8f26454-670d-4db6-aac0-a195513f89a6\") " Jul 14 18:04:50.169769 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:04:50.169645 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c8f26454-670d-4db6-aac0-a195513f89a6-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c8f26454-670d-4db6-aac0-a195513f89a6" (UID: "c8f26454-670d-4db6-aac0-a195513f89a6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 18:04:50.169769 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:04:50.169677 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c8f26454-670d-4db6-aac0-a195513f89a6-istiod-ca-cert\") pod \"c8f26454-670d-4db6-aac0-a195513f89a6\" (UID: \"c8f26454-670d-4db6-aac0-a195513f89a6\") " Jul 14 18:04:50.169769 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:04:50.169724 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c8f26454-670d-4db6-aac0-a195513f89a6-istio-token\") pod \"c8f26454-670d-4db6-aac0-a195513f89a6\" (UID: \"c8f26454-670d-4db6-aac0-a195513f89a6\") " Jul 14 18:04:50.169769 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:04:50.169762 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c8f26454-670d-4db6-aac0-a195513f89a6-istio-envoy\") pod \"c8f26454-670d-4db6-aac0-a195513f89a6\" (UID: \"c8f26454-670d-4db6-aac0-a195513f89a6\") " Jul 14 18:04:50.169954 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:04:50.169762 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c8f26454-670d-4db6-aac0-a195513f89a6-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c8f26454-670d-4db6-aac0-a195513f89a6" (UID: "c8f26454-670d-4db6-aac0-a195513f89a6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 18:04:50.169954 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:04:50.169792 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c8f26454-670d-4db6-aac0-a195513f89a6-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c8f26454-670d-4db6-aac0-a195513f89a6" (UID: "c8f26454-670d-4db6-aac0-a195513f89a6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 18:04:50.169954 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:04:50.169812 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c8f26454-670d-4db6-aac0-a195513f89a6-istio-podinfo\") pod \"c8f26454-670d-4db6-aac0-a195513f89a6\" (UID: \"c8f26454-670d-4db6-aac0-a195513f89a6\") " Jul 14 18:04:50.170120 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:04:50.169969 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c8f26454-670d-4db6-aac0-a195513f89a6-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c8f26454-670d-4db6-aac0-a195513f89a6" (UID: "c8f26454-670d-4db6-aac0-a195513f89a6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 18:04:50.170174 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:04:50.170150 2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8f26454-670d-4db6-aac0-a195513f89a6-workload-socket\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 18:04:50.170174 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:04:50.170168 2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c8f26454-670d-4db6-aac0-a195513f89a6-istiod-ca-cert\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 18:04:50.170293 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:04:50.170180 2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8f26454-670d-4db6-aac0-a195513f89a6-credential-socket\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 18:04:50.170293 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:04:50.170213 2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c8f26454-670d-4db6-aac0-a195513f89a6-workload-certs\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 18:04:50.170685 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:04:50.170660 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c8f26454-670d-4db6-aac0-a195513f89a6-istio-data" (OuterVolumeSpecName: "istio-data") pod "c8f26454-670d-4db6-aac0-a195513f89a6" (UID: "c8f26454-670d-4db6-aac0-a195513f89a6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 18:04:50.171954 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:04:50.171936 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c8f26454-670d-4db6-aac0-a195513f89a6-istio-token" (OuterVolumeSpecName: "istio-token") pod "c8f26454-670d-4db6-aac0-a195513f89a6" (UID: "c8f26454-670d-4db6-aac0-a195513f89a6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 18:04:50.172063 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:04:50.172048 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c8f26454-670d-4db6-aac0-a195513f89a6-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c8f26454-670d-4db6-aac0-a195513f89a6" (UID: "c8f26454-670d-4db6-aac0-a195513f89a6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 18:04:50.172395 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:04:50.172372 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c8f26454-670d-4db6-aac0-a195513f89a6-kube-api-access-vm6rn" (OuterVolumeSpecName: "kube-api-access-vm6rn") pod "c8f26454-670d-4db6-aac0-a195513f89a6" (UID: "c8f26454-670d-4db6-aac0-a195513f89a6"). InnerVolumeSpecName "kube-api-access-vm6rn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 18:04:50.172486 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:04:50.172382 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c8f26454-670d-4db6-aac0-a195513f89a6-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c8f26454-670d-4db6-aac0-a195513f89a6" (UID: "c8f26454-670d-4db6-aac0-a195513f89a6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 14 18:04:50.270893 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:04:50.270868 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vm6rn\" (UniqueName: \"kubernetes.io/projected/c8f26454-670d-4db6-aac0-a195513f89a6-kube-api-access-vm6rn\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 18:04:50.270893 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:04:50.270888 2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c8f26454-670d-4db6-aac0-a195513f89a6-istio-data\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 18:04:50.271071 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:04:50.270899 2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c8f26454-670d-4db6-aac0-a195513f89a6-istio-token\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 18:04:50.271071 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:04:50.270907 2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c8f26454-670d-4db6-aac0-a195513f89a6-istio-envoy\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 18:04:50.271071 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:04:50.270915 2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c8f26454-670d-4db6-aac0-a195513f89a6-istio-podinfo\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 18:04:50.783838 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:04:50.783807 2562 generic.go:356] "Generic (PLEG): container finished" podID="c8f26454-670d-4db6-aac0-a195513f89a6" containerID="2b9e7cfb778108c61b33a003db504339eef065c5995d7a35cfd55cee9a0986aa" exitCode=0 Jul 14 18:04:50.784000 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:04:50.783854 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-8cb2afcb-openshift-default-d47c5d887-tlvgp" Jul 14 18:04:50.784000 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:04:50.783892 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8cb2afcb-openshift-default-d47c5d887-tlvgp" event={"ID":"c8f26454-670d-4db6-aac0-a195513f89a6","Type":"ContainerDied","Data":"2b9e7cfb778108c61b33a003db504339eef065c5995d7a35cfd55cee9a0986aa"} Jul 14 18:04:50.784000 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:04:50.783943 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8cb2afcb-openshift-default-d47c5d887-tlvgp" event={"ID":"c8f26454-670d-4db6-aac0-a195513f89a6","Type":"ContainerDied","Data":"1e86867a16a858686068920f7be45410c5b08659aa1a54195cc0e6f06ea83245"} Jul 14 18:04:50.784000 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:04:50.783966 2562 scope.go:117] "RemoveContainer" containerID="2b9e7cfb778108c61b33a003db504339eef065c5995d7a35cfd55cee9a0986aa" Jul 14 18:04:50.792984 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:04:50.792971 2562 scope.go:117] "RemoveContainer" containerID="3d957ed42d34556cb56faa3f6a5f2a4ff2557d4cb350defcb8984c64edff56f3" Jul 14 18:04:50.800269 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:04:50.800255 2562 scope.go:117] "RemoveContainer" containerID="2b9e7cfb778108c61b33a003db504339eef065c5995d7a35cfd55cee9a0986aa" Jul 14 18:04:50.800505 ip-10-0-129-209 kubenswrapper[2562]: E0714 18:04:50.800480 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2b9e7cfb778108c61b33a003db504339eef065c5995d7a35cfd55cee9a0986aa\": container with ID starting with 2b9e7cfb778108c61b33a003db504339eef065c5995d7a35cfd55cee9a0986aa not found: ID does not exist" containerID="2b9e7cfb778108c61b33a003db504339eef065c5995d7a35cfd55cee9a0986aa" Jul 14 18:04:50.800571 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:04:50.800514 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2b9e7cfb778108c61b33a003db504339eef065c5995d7a35cfd55cee9a0986aa"} err="failed to get container status \"2b9e7cfb778108c61b33a003db504339eef065c5995d7a35cfd55cee9a0986aa\": rpc error: code = NotFound desc = could not find container \"2b9e7cfb778108c61b33a003db504339eef065c5995d7a35cfd55cee9a0986aa\": container with ID starting with 2b9e7cfb778108c61b33a003db504339eef065c5995d7a35cfd55cee9a0986aa not found: ID does not exist" Jul 14 18:04:50.800571 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:04:50.800531 2562 scope.go:117] "RemoveContainer" containerID="3d957ed42d34556cb56faa3f6a5f2a4ff2557d4cb350defcb8984c64edff56f3" Jul 14 18:04:50.800748 ip-10-0-129-209 kubenswrapper[2562]: E0714 18:04:50.800733 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3d957ed42d34556cb56faa3f6a5f2a4ff2557d4cb350defcb8984c64edff56f3\": container with ID starting with 3d957ed42d34556cb56faa3f6a5f2a4ff2557d4cb350defcb8984c64edff56f3 not found: ID does not exist" containerID="3d957ed42d34556cb56faa3f6a5f2a4ff2557d4cb350defcb8984c64edff56f3" Jul 14 18:04:50.800796 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:04:50.800753 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d957ed42d34556cb56faa3f6a5f2a4ff2557d4cb350defcb8984c64edff56f3"} err="failed to get container status \"3d957ed42d34556cb56faa3f6a5f2a4ff2557d4cb350defcb8984c64edff56f3\": rpc error: code = NotFound desc = could not find container \"3d957ed42d34556cb56faa3f6a5f2a4ff2557d4cb350defcb8984c64edff56f3\": container with ID starting with 3d957ed42d34556cb56faa3f6a5f2a4ff2557d4cb350defcb8984c64edff56f3 not found: ID does not exist" Jul 14 18:04:50.806377 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:04:50.806358 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-8cb2afcb-openshift-default-d47c5d887-tlvgp"] Jul 14 18:04:50.812871 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:04:50.812852 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-8cb2afcb-openshift-default-d47c5d887-tlvgp"] Jul 14 18:04:51.519066 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:04:51.519037 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c8f26454-670d-4db6-aac0-a195513f89a6" path="/var/lib/kubelet/pods/c8f26454-670d-4db6-aac0-a195513f89a6/volumes" Jul 14 18:05:05.705453 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:05:05.705423 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-ef19c3e1-openshift-default-69977b88ff-lt5ws"] Jul 14 18:05:05.705854 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:05:05.705736 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="533a91ec-be53-4df9-a11b-fd45f75a0c66" containerName="istio-proxy" Jul 14 18:05:05.705854 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:05:05.705747 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="533a91ec-be53-4df9-a11b-fd45f75a0c66" containerName="istio-proxy" Jul 14 18:05:05.705854 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:05:05.705770 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c8f26454-670d-4db6-aac0-a195513f89a6" containerName="istio-proxy" Jul 14 18:05:05.705854 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:05:05.705775 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="c8f26454-670d-4db6-aac0-a195513f89a6" containerName="istio-proxy" Jul 14 18:05:05.705854 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:05:05.705784 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c8f26454-670d-4db6-aac0-a195513f89a6" containerName="istio-proxy" Jul 14 18:05:05.705854 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:05:05.705789 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="c8f26454-670d-4db6-aac0-a195513f89a6" containerName="istio-proxy" Jul 14 18:05:05.705854 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:05:05.705838 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="c8f26454-670d-4db6-aac0-a195513f89a6" containerName="istio-proxy" Jul 14 18:05:05.708807 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:05:05.708792 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ef19c3e1-openshift-default-69977b88ff-lt5ws" Jul 14 18:05:05.711373 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:05:05.711353 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-ef19c3e1-openshift-default-dockercfg-vlj7m\"" Jul 14 18:05:05.740385 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:05:05.740362 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-ef19c3e1-openshift-default-69977b88ff-lt5ws"] Jul 14 18:05:05.797663 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:05:05.797627 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8cf24882-6384-48b2-bb2e-ce9991c98f20-credential-socket\") pod \"e2e-mt-ef19c3e1-openshift-default-69977b88ff-lt5ws\" (UID: \"8cf24882-6384-48b2-bb2e-ce9991c98f20\") " pod="openshift-ingress/e2e-mt-ef19c3e1-openshift-default-69977b88ff-lt5ws" Jul 14 18:05:05.797826 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:05:05.797720 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8cf24882-6384-48b2-bb2e-ce9991c98f20-workload-socket\") pod \"e2e-mt-ef19c3e1-openshift-default-69977b88ff-lt5ws\" (UID: \"8cf24882-6384-48b2-bb2e-ce9991c98f20\") " pod="openshift-ingress/e2e-mt-ef19c3e1-openshift-default-69977b88ff-lt5ws" Jul 14 18:05:05.797826 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:05:05.797769 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8cf24882-6384-48b2-bb2e-ce9991c98f20-istio-data\") pod \"e2e-mt-ef19c3e1-openshift-default-69977b88ff-lt5ws\" (UID: \"8cf24882-6384-48b2-bb2e-ce9991c98f20\") " pod="openshift-ingress/e2e-mt-ef19c3e1-openshift-default-69977b88ff-lt5ws" Jul 14 18:05:05.797930 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:05:05.797846 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8cf24882-6384-48b2-bb2e-ce9991c98f20-istio-podinfo\") pod \"e2e-mt-ef19c3e1-openshift-default-69977b88ff-lt5ws\" (UID: \"8cf24882-6384-48b2-bb2e-ce9991c98f20\") " pod="openshift-ingress/e2e-mt-ef19c3e1-openshift-default-69977b88ff-lt5ws" Jul 14 18:05:05.797930 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:05:05.797888 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8cf24882-6384-48b2-bb2e-ce9991c98f20-workload-certs\") pod \"e2e-mt-ef19c3e1-openshift-default-69977b88ff-lt5ws\" (UID: \"8cf24882-6384-48b2-bb2e-ce9991c98f20\") " pod="openshift-ingress/e2e-mt-ef19c3e1-openshift-default-69977b88ff-lt5ws" Jul 14 18:05:05.797930 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:05:05.797917 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vwpdc\" (UniqueName: \"kubernetes.io/projected/8cf24882-6384-48b2-bb2e-ce9991c98f20-kube-api-access-vwpdc\") pod \"e2e-mt-ef19c3e1-openshift-default-69977b88ff-lt5ws\" (UID: \"8cf24882-6384-48b2-bb2e-ce9991c98f20\") " pod="openshift-ingress/e2e-mt-ef19c3e1-openshift-default-69977b88ff-lt5ws" Jul 14 18:05:05.798033 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:05:05.797987 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8cf24882-6384-48b2-bb2e-ce9991c98f20-istio-envoy\") pod \"e2e-mt-ef19c3e1-openshift-default-69977b88ff-lt5ws\" (UID: \"8cf24882-6384-48b2-bb2e-ce9991c98f20\") " pod="openshift-ingress/e2e-mt-ef19c3e1-openshift-default-69977b88ff-lt5ws" Jul 14 18:05:05.798077 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:05:05.798063 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8cf24882-6384-48b2-bb2e-ce9991c98f20-istio-token\") pod \"e2e-mt-ef19c3e1-openshift-default-69977b88ff-lt5ws\" (UID: \"8cf24882-6384-48b2-bb2e-ce9991c98f20\") " pod="openshift-ingress/e2e-mt-ef19c3e1-openshift-default-69977b88ff-lt5ws" Jul 14 18:05:05.798133 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:05:05.798102 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8cf24882-6384-48b2-bb2e-ce9991c98f20-istiod-ca-cert\") pod \"e2e-mt-ef19c3e1-openshift-default-69977b88ff-lt5ws\" (UID: \"8cf24882-6384-48b2-bb2e-ce9991c98f20\") " pod="openshift-ingress/e2e-mt-ef19c3e1-openshift-default-69977b88ff-lt5ws" Jul 14 18:05:05.898773 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:05:05.898743 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8cf24882-6384-48b2-bb2e-ce9991c98f20-istio-token\") pod \"e2e-mt-ef19c3e1-openshift-default-69977b88ff-lt5ws\" (UID: \"8cf24882-6384-48b2-bb2e-ce9991c98f20\") " pod="openshift-ingress/e2e-mt-ef19c3e1-openshift-default-69977b88ff-lt5ws" Jul 14 18:05:05.898773 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:05:05.898777 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8cf24882-6384-48b2-bb2e-ce9991c98f20-istiod-ca-cert\") pod \"e2e-mt-ef19c3e1-openshift-default-69977b88ff-lt5ws\" (UID: \"8cf24882-6384-48b2-bb2e-ce9991c98f20\") " pod="openshift-ingress/e2e-mt-ef19c3e1-openshift-default-69977b88ff-lt5ws" Jul 14 18:05:05.899026 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:05:05.898798 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8cf24882-6384-48b2-bb2e-ce9991c98f20-credential-socket\") pod \"e2e-mt-ef19c3e1-openshift-default-69977b88ff-lt5ws\" (UID: \"8cf24882-6384-48b2-bb2e-ce9991c98f20\") " pod="openshift-ingress/e2e-mt-ef19c3e1-openshift-default-69977b88ff-lt5ws" Jul 14 18:05:05.899026 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:05:05.898941 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8cf24882-6384-48b2-bb2e-ce9991c98f20-workload-socket\") pod \"e2e-mt-ef19c3e1-openshift-default-69977b88ff-lt5ws\" (UID: \"8cf24882-6384-48b2-bb2e-ce9991c98f20\") " pod="openshift-ingress/e2e-mt-ef19c3e1-openshift-default-69977b88ff-lt5ws" Jul 14 18:05:05.899164 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:05:05.899021 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8cf24882-6384-48b2-bb2e-ce9991c98f20-istio-data\") pod \"e2e-mt-ef19c3e1-openshift-default-69977b88ff-lt5ws\" (UID: \"8cf24882-6384-48b2-bb2e-ce9991c98f20\") " pod="openshift-ingress/e2e-mt-ef19c3e1-openshift-default-69977b88ff-lt5ws" Jul 14 18:05:05.899164 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:05:05.899063 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8cf24882-6384-48b2-bb2e-ce9991c98f20-istio-podinfo\") pod \"e2e-mt-ef19c3e1-openshift-default-69977b88ff-lt5ws\" (UID: \"8cf24882-6384-48b2-bb2e-ce9991c98f20\") " pod="openshift-ingress/e2e-mt-ef19c3e1-openshift-default-69977b88ff-lt5ws" Jul 14 18:05:05.899312 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:05:05.899180 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8cf24882-6384-48b2-bb2e-ce9991c98f20-credential-socket\") pod \"e2e-mt-ef19c3e1-openshift-default-69977b88ff-lt5ws\" (UID: \"8cf24882-6384-48b2-bb2e-ce9991c98f20\") " pod="openshift-ingress/e2e-mt-ef19c3e1-openshift-default-69977b88ff-lt5ws" Jul 14 18:05:05.899312 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:05:05.899228 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8cf24882-6384-48b2-bb2e-ce9991c98f20-workload-certs\") pod \"e2e-mt-ef19c3e1-openshift-default-69977b88ff-lt5ws\" (UID: \"8cf24882-6384-48b2-bb2e-ce9991c98f20\") " pod="openshift-ingress/e2e-mt-ef19c3e1-openshift-default-69977b88ff-lt5ws" Jul 14 18:05:05.899312 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:05:05.899257 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vwpdc\" (UniqueName: \"kubernetes.io/projected/8cf24882-6384-48b2-bb2e-ce9991c98f20-kube-api-access-vwpdc\") pod \"e2e-mt-ef19c3e1-openshift-default-69977b88ff-lt5ws\" (UID: \"8cf24882-6384-48b2-bb2e-ce9991c98f20\") " pod="openshift-ingress/e2e-mt-ef19c3e1-openshift-default-69977b88ff-lt5ws" Jul 14 18:05:05.899312 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:05:05.899277 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8cf24882-6384-48b2-bb2e-ce9991c98f20-workload-socket\") pod \"e2e-mt-ef19c3e1-openshift-default-69977b88ff-lt5ws\" (UID: \"8cf24882-6384-48b2-bb2e-ce9991c98f20\") " pod="openshift-ingress/e2e-mt-ef19c3e1-openshift-default-69977b88ff-lt5ws" Jul 14 18:05:05.899312 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:05:05.899296 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8cf24882-6384-48b2-bb2e-ce9991c98f20-istio-envoy\") pod \"e2e-mt-ef19c3e1-openshift-default-69977b88ff-lt5ws\" (UID: \"8cf24882-6384-48b2-bb2e-ce9991c98f20\") " pod="openshift-ingress/e2e-mt-ef19c3e1-openshift-default-69977b88ff-lt5ws" Jul 14 18:05:05.899578 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:05:05.899399 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8cf24882-6384-48b2-bb2e-ce9991c98f20-istio-data\") pod \"e2e-mt-ef19c3e1-openshift-default-69977b88ff-lt5ws\" (UID: \"8cf24882-6384-48b2-bb2e-ce9991c98f20\") " pod="openshift-ingress/e2e-mt-ef19c3e1-openshift-default-69977b88ff-lt5ws" Jul 14 18:05:05.899578 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:05:05.899467 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8cf24882-6384-48b2-bb2e-ce9991c98f20-workload-certs\") pod \"e2e-mt-ef19c3e1-openshift-default-69977b88ff-lt5ws\" (UID: \"8cf24882-6384-48b2-bb2e-ce9991c98f20\") " pod="openshift-ingress/e2e-mt-ef19c3e1-openshift-default-69977b88ff-lt5ws" Jul 14 18:05:05.899578 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:05:05.899555 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8cf24882-6384-48b2-bb2e-ce9991c98f20-istiod-ca-cert\") pod \"e2e-mt-ef19c3e1-openshift-default-69977b88ff-lt5ws\" (UID: \"8cf24882-6384-48b2-bb2e-ce9991c98f20\") " pod="openshift-ingress/e2e-mt-ef19c3e1-openshift-default-69977b88ff-lt5ws" Jul 14 18:05:05.901492 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:05:05.901467 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8cf24882-6384-48b2-bb2e-ce9991c98f20-istio-envoy\") pod \"e2e-mt-ef19c3e1-openshift-default-69977b88ff-lt5ws\" (UID: \"8cf24882-6384-48b2-bb2e-ce9991c98f20\") " pod="openshift-ingress/e2e-mt-ef19c3e1-openshift-default-69977b88ff-lt5ws" Jul 14 18:05:05.901761 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:05:05.901746 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8cf24882-6384-48b2-bb2e-ce9991c98f20-istio-podinfo\") pod \"e2e-mt-ef19c3e1-openshift-default-69977b88ff-lt5ws\" (UID: \"8cf24882-6384-48b2-bb2e-ce9991c98f20\") " pod="openshift-ingress/e2e-mt-ef19c3e1-openshift-default-69977b88ff-lt5ws" Jul 14 18:05:05.907933 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:05:05.907904 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8cf24882-6384-48b2-bb2e-ce9991c98f20-istio-token\") pod \"e2e-mt-ef19c3e1-openshift-default-69977b88ff-lt5ws\" (UID: \"8cf24882-6384-48b2-bb2e-ce9991c98f20\") " pod="openshift-ingress/e2e-mt-ef19c3e1-openshift-default-69977b88ff-lt5ws" Jul 14 18:05:05.908268 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:05:05.908249 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vwpdc\" (UniqueName: \"kubernetes.io/projected/8cf24882-6384-48b2-bb2e-ce9991c98f20-kube-api-access-vwpdc\") pod \"e2e-mt-ef19c3e1-openshift-default-69977b88ff-lt5ws\" (UID: \"8cf24882-6384-48b2-bb2e-ce9991c98f20\") " pod="openshift-ingress/e2e-mt-ef19c3e1-openshift-default-69977b88ff-lt5ws" Jul 14 18:05:06.019105 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:05:06.019027 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ef19c3e1-openshift-default-69977b88ff-lt5ws" Jul 14 18:05:06.159979 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:05:06.159947 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-ef19c3e1-openshift-default-69977b88ff-lt5ws"] Jul 14 18:05:06.162975 ip-10-0-129-209 kubenswrapper[2562]: W0714 18:05:06.162935 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8cf24882_6384_48b2_bb2e_ce9991c98f20.slice/crio-9f699268b189310e0666683ef440bce0476fb49d198288be784200334a1460e0 WatchSource:0}: Error finding container 9f699268b189310e0666683ef440bce0476fb49d198288be784200334a1460e0: Status 404 returned error can't find the container with id 9f699268b189310e0666683ef440bce0476fb49d198288be784200334a1460e0 Jul 14 18:05:06.164909 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:05:06.164880 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 14 18:05:06.164981 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:05:06.164941 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 14 18:05:06.164981 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:05:06.164970 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 14 18:05:06.854836 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:05:06.854803 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ef19c3e1-openshift-default-69977b88ff-lt5ws" event={"ID":"8cf24882-6384-48b2-bb2e-ce9991c98f20","Type":"ContainerStarted","Data":"4fb55e98ab1e17ab28327ca3537a0313e49c1a9135739e229c819fe42231a46e"} Jul 14 18:05:06.855237 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:05:06.854876 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ef19c3e1-openshift-default-69977b88ff-lt5ws" event={"ID":"8cf24882-6384-48b2-bb2e-ce9991c98f20","Type":"ContainerStarted","Data":"9f699268b189310e0666683ef440bce0476fb49d198288be784200334a1460e0"} Jul 14 18:05:06.886018 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:05:06.885968 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-ef19c3e1-openshift-default-69977b88ff-lt5ws" podStartSLOduration=1.885952143 podStartE2EDuration="1.885952143s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 18:05:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 18:05:06.881642945 +0000 UTC m=+1955.958100495" watchObservedRunningTime="2026-07-14 18:05:06.885952143 +0000 UTC m=+1955.962409682" Jul 14 18:05:07.019367 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:05:07.019336 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-ef19c3e1-openshift-default-69977b88ff-lt5ws" Jul 14 18:05:07.023928 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:05:07.023907 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-ef19c3e1-openshift-default-69977b88ff-lt5ws" Jul 14 18:05:07.859669 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:05:07.859638 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-ef19c3e1-openshift-default-69977b88ff-lt5ws" Jul 14 18:05:07.860675 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:05:07.860651 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-ef19c3e1-openshift-default-69977b88ff-lt5ws" Jul 14 18:05:25.942097 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:05:25.942068 2562 generic.go:356] "Generic (PLEG): container finished" podID="8cf24882-6384-48b2-bb2e-ce9991c98f20" containerID="4fb55e98ab1e17ab28327ca3537a0313e49c1a9135739e229c819fe42231a46e" exitCode=0 Jul 14 18:05:25.942494 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:05:25.942146 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ef19c3e1-openshift-default-69977b88ff-lt5ws" event={"ID":"8cf24882-6384-48b2-bb2e-ce9991c98f20","Type":"ContainerDied","Data":"4fb55e98ab1e17ab28327ca3537a0313e49c1a9135739e229c819fe42231a46e"} Jul 14 18:05:25.942560 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:05:25.942531 2562 scope.go:117] "RemoveContainer" containerID="4fb55e98ab1e17ab28327ca3537a0313e49c1a9135739e229c819fe42231a46e" Jul 14 18:05:25.943501 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:05:25.943476 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 14 18:05:25.943558 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:05:25.943536 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 14 18:05:25.943594 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:05:25.943565 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 14 18:05:26.948350 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:05:26.948317 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ef19c3e1-openshift-default-69977b88ff-lt5ws" event={"ID":"8cf24882-6384-48b2-bb2e-ce9991c98f20","Type":"ContainerStarted","Data":"280319b106c097652d2a4a597fdc8daa47392f463c087106c0460dbc722cee1e"} Jul 14 18:05:27.019307 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:05:27.019273 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-ef19c3e1-openshift-default-69977b88ff-lt5ws" Jul 14 18:05:27.019554 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:05:27.019533 2562 patch_prober.go:28] interesting pod/e2e-mt-ef19c3e1-openshift-default-69977b88ff-lt5ws container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body= Jul 14 18:05:27.019650 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:05:27.019615 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ef19c3e1-openshift-default-69977b88ff-lt5ws" podUID="8cf24882-6384-48b2-bb2e-ce9991c98f20" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" Jul 14 18:05:28.020030 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:05:28.019997 2562 patch_prober.go:28] interesting pod/e2e-mt-ef19c3e1-openshift-default-69977b88ff-lt5ws container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body= Jul 14 18:05:28.020427 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:05:28.020046 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ef19c3e1-openshift-default-69977b88ff-lt5ws" podUID="8cf24882-6384-48b2-bb2e-ce9991c98f20" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" Jul 14 18:05:29.020248 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:05:29.020214 2562 patch_prober.go:28] interesting pod/e2e-mt-ef19c3e1-openshift-default-69977b88ff-lt5ws container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body= Jul 14 18:05:29.020626 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:05:29.020270 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ef19c3e1-openshift-default-69977b88ff-lt5ws" podUID="8cf24882-6384-48b2-bb2e-ce9991c98f20" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" Jul 14 18:05:30.020112 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:05:30.020073 2562 patch_prober.go:28] interesting pod/e2e-mt-ef19c3e1-openshift-default-69977b88ff-lt5ws container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" start-of-body= Jul 14 18:05:30.020312 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:05:30.020128 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ef19c3e1-openshift-default-69977b88ff-lt5ws" podUID="8cf24882-6384-48b2-bb2e-ce9991c98f20" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.32:15021/healthz/ready\": dial tcp 10.134.0.32:15021: connect: connection refused" Jul 14 18:05:31.022701 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:05:31.022674 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-ef19c3e1-openshift-default-69977b88ff-lt5ws" Jul 14 18:05:31.023123 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:05:31.022933 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-ef19c3e1-openshift-default-69977b88ff-lt5ws" Jul 14 18:05:31.023539 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:05:31.023522 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-ef19c3e1-openshift-default-69977b88ff-lt5ws" Jul 14 18:06:59.058975 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:06:59.058899 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-ef19c3e1-openshift-default-69977b88ff-lt5ws"] Jul 14 18:06:59.059489 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:06:59.059121 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-ef19c3e1-openshift-default-69977b88ff-lt5ws" podUID="8cf24882-6384-48b2-bb2e-ce9991c98f20" containerName="istio-proxy" containerID="cri-o://280319b106c097652d2a4a597fdc8daa47392f463c087106c0460dbc722cee1e" gracePeriod=30 Jul 14 18:07:01.025631 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:07:01.025599 2562 patch_prober.go:28] interesting pod/e2e-mt-ef19c3e1-openshift-default-69977b88ff-lt5ws container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 14 18:07:01.026061 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:07:01.025663 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-ef19c3e1-openshift-default-69977b88ff-lt5ws" podUID="8cf24882-6384-48b2-bb2e-ce9991c98f20" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 14 18:07:04.315300 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:07:04.315276 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ef19c3e1-openshift-default-69977b88ff-lt5ws" Jul 14 18:07:04.374266 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:07:04.374236 2562 generic.go:356] "Generic (PLEG): container finished" podID="8cf24882-6384-48b2-bb2e-ce9991c98f20" containerID="280319b106c097652d2a4a597fdc8daa47392f463c087106c0460dbc722cee1e" exitCode=0 Jul 14 18:07:04.374405 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:07:04.374285 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ef19c3e1-openshift-default-69977b88ff-lt5ws" Jul 14 18:07:04.374405 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:07:04.374320 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ef19c3e1-openshift-default-69977b88ff-lt5ws" event={"ID":"8cf24882-6384-48b2-bb2e-ce9991c98f20","Type":"ContainerDied","Data":"280319b106c097652d2a4a597fdc8daa47392f463c087106c0460dbc722cee1e"} Jul 14 18:07:04.374405 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:07:04.374376 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ef19c3e1-openshift-default-69977b88ff-lt5ws" event={"ID":"8cf24882-6384-48b2-bb2e-ce9991c98f20","Type":"ContainerDied","Data":"9f699268b189310e0666683ef440bce0476fb49d198288be784200334a1460e0"} Jul 14 18:07:04.374527 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:07:04.374406 2562 scope.go:117] "RemoveContainer" containerID="280319b106c097652d2a4a597fdc8daa47392f463c087106c0460dbc722cee1e" Jul 14 18:07:04.382518 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:07:04.382498 2562 scope.go:117] "RemoveContainer" containerID="4fb55e98ab1e17ab28327ca3537a0313e49c1a9135739e229c819fe42231a46e" Jul 14 18:07:04.390091 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:07:04.390074 2562 scope.go:117] "RemoveContainer" containerID="280319b106c097652d2a4a597fdc8daa47392f463c087106c0460dbc722cee1e" Jul 14 18:07:04.390452 ip-10-0-129-209 kubenswrapper[2562]: E0714 18:07:04.390435 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"280319b106c097652d2a4a597fdc8daa47392f463c087106c0460dbc722cee1e\": container with ID starting with 280319b106c097652d2a4a597fdc8daa47392f463c087106c0460dbc722cee1e not found: ID does not exist" containerID="280319b106c097652d2a4a597fdc8daa47392f463c087106c0460dbc722cee1e" Jul 14 18:07:04.390523 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:07:04.390460 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"280319b106c097652d2a4a597fdc8daa47392f463c087106c0460dbc722cee1e"} err="failed to get container status \"280319b106c097652d2a4a597fdc8daa47392f463c087106c0460dbc722cee1e\": rpc error: code = NotFound desc = could not find container \"280319b106c097652d2a4a597fdc8daa47392f463c087106c0460dbc722cee1e\": container with ID starting with 280319b106c097652d2a4a597fdc8daa47392f463c087106c0460dbc722cee1e not found: ID does not exist" Jul 14 18:07:04.390523 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:07:04.390476 2562 scope.go:117] "RemoveContainer" containerID="4fb55e98ab1e17ab28327ca3537a0313e49c1a9135739e229c819fe42231a46e" Jul 14 18:07:04.390720 ip-10-0-129-209 kubenswrapper[2562]: E0714 18:07:04.390702 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4fb55e98ab1e17ab28327ca3537a0313e49c1a9135739e229c819fe42231a46e\": container with ID starting with 4fb55e98ab1e17ab28327ca3537a0313e49c1a9135739e229c819fe42231a46e not found: ID does not exist" containerID="4fb55e98ab1e17ab28327ca3537a0313e49c1a9135739e229c819fe42231a46e" Jul 14 18:07:04.390781 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:07:04.390729 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4fb55e98ab1e17ab28327ca3537a0313e49c1a9135739e229c819fe42231a46e"} err="failed to get container status \"4fb55e98ab1e17ab28327ca3537a0313e49c1a9135739e229c819fe42231a46e\": rpc error: code = NotFound desc = could not find container \"4fb55e98ab1e17ab28327ca3537a0313e49c1a9135739e229c819fe42231a46e\": container with ID starting with 4fb55e98ab1e17ab28327ca3537a0313e49c1a9135739e229c819fe42231a46e not found: ID does not exist" Jul 14 18:07:04.403031 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:07:04.403012 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8cf24882-6384-48b2-bb2e-ce9991c98f20-istio-podinfo\") pod \"8cf24882-6384-48b2-bb2e-ce9991c98f20\" (UID: \"8cf24882-6384-48b2-bb2e-ce9991c98f20\") " Jul 14 18:07:04.403127 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:07:04.403058 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8cf24882-6384-48b2-bb2e-ce9991c98f20-istio-data\") pod \"8cf24882-6384-48b2-bb2e-ce9991c98f20\" (UID: \"8cf24882-6384-48b2-bb2e-ce9991c98f20\") " Jul 14 18:07:04.403127 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:07:04.403095 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8cf24882-6384-48b2-bb2e-ce9991c98f20-istio-envoy\") pod \"8cf24882-6384-48b2-bb2e-ce9991c98f20\" (UID: \"8cf24882-6384-48b2-bb2e-ce9991c98f20\") " Jul 14 18:07:04.403127 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:07:04.403119 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8cf24882-6384-48b2-bb2e-ce9991c98f20-istio-token\") pod \"8cf24882-6384-48b2-bb2e-ce9991c98f20\" (UID: \"8cf24882-6384-48b2-bb2e-ce9991c98f20\") " Jul 14 18:07:04.403319 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:07:04.403156 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vwpdc\" (UniqueName: \"kubernetes.io/projected/8cf24882-6384-48b2-bb2e-ce9991c98f20-kube-api-access-vwpdc\") pod \"8cf24882-6384-48b2-bb2e-ce9991c98f20\" (UID: \"8cf24882-6384-48b2-bb2e-ce9991c98f20\") " Jul 14 18:07:04.403319 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:07:04.403215 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8cf24882-6384-48b2-bb2e-ce9991c98f20-istiod-ca-cert\") pod \"8cf24882-6384-48b2-bb2e-ce9991c98f20\" (UID: \"8cf24882-6384-48b2-bb2e-ce9991c98f20\") " Jul 14 18:07:04.403319 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:07:04.403263 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8cf24882-6384-48b2-bb2e-ce9991c98f20-credential-socket\") pod \"8cf24882-6384-48b2-bb2e-ce9991c98f20\" (UID: \"8cf24882-6384-48b2-bb2e-ce9991c98f20\") " Jul 14 18:07:04.403319 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:07:04.403293 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8cf24882-6384-48b2-bb2e-ce9991c98f20-workload-certs\") pod \"8cf24882-6384-48b2-bb2e-ce9991c98f20\" (UID: \"8cf24882-6384-48b2-bb2e-ce9991c98f20\") " Jul 14 18:07:04.403510 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:07:04.403339 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8cf24882-6384-48b2-bb2e-ce9991c98f20-workload-socket\") pod \"8cf24882-6384-48b2-bb2e-ce9991c98f20\" (UID: \"8cf24882-6384-48b2-bb2e-ce9991c98f20\") " Jul 14 18:07:04.404399 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:07:04.404300 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8cf24882-6384-48b2-bb2e-ce9991c98f20-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8cf24882-6384-48b2-bb2e-ce9991c98f20" (UID: "8cf24882-6384-48b2-bb2e-ce9991c98f20"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 18:07:04.404510 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:07:04.404418 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8cf24882-6384-48b2-bb2e-ce9991c98f20-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8cf24882-6384-48b2-bb2e-ce9991c98f20" (UID: "8cf24882-6384-48b2-bb2e-ce9991c98f20"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 18:07:04.404510 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:07:04.404446 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8cf24882-6384-48b2-bb2e-ce9991c98f20-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8cf24882-6384-48b2-bb2e-ce9991c98f20" (UID: "8cf24882-6384-48b2-bb2e-ce9991c98f20"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 18:07:04.406257 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:07:04.404731 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8cf24882-6384-48b2-bb2e-ce9991c98f20-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8cf24882-6384-48b2-bb2e-ce9991c98f20" (UID: "8cf24882-6384-48b2-bb2e-ce9991c98f20"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 18:07:04.406257 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:07:04.404752 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8cf24882-6384-48b2-bb2e-ce9991c98f20-istio-data" (OuterVolumeSpecName: "istio-data") pod "8cf24882-6384-48b2-bb2e-ce9991c98f20" (UID: "8cf24882-6384-48b2-bb2e-ce9991c98f20"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 18:07:04.406257 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:07:04.405966 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8cf24882-6384-48b2-bb2e-ce9991c98f20-istio-token" (OuterVolumeSpecName: "istio-token") pod "8cf24882-6384-48b2-bb2e-ce9991c98f20" (UID: "8cf24882-6384-48b2-bb2e-ce9991c98f20"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 18:07:04.410886 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:07:04.406355 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8cf24882-6384-48b2-bb2e-ce9991c98f20-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8cf24882-6384-48b2-bb2e-ce9991c98f20" (UID: "8cf24882-6384-48b2-bb2e-ce9991c98f20"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 14 18:07:04.411340 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:07:04.411317 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8cf24882-6384-48b2-bb2e-ce9991c98f20-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8cf24882-6384-48b2-bb2e-ce9991c98f20" (UID: "8cf24882-6384-48b2-bb2e-ce9991c98f20"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 18:07:04.411516 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:07:04.411493 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8cf24882-6384-48b2-bb2e-ce9991c98f20-kube-api-access-vwpdc" (OuterVolumeSpecName: "kube-api-access-vwpdc") pod "8cf24882-6384-48b2-bb2e-ce9991c98f20" (UID: "8cf24882-6384-48b2-bb2e-ce9991c98f20"). InnerVolumeSpecName "kube-api-access-vwpdc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 18:07:04.505274 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:07:04.505249 2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8cf24882-6384-48b2-bb2e-ce9991c98f20-istio-podinfo\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 18:07:04.505274 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:07:04.505271 2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8cf24882-6384-48b2-bb2e-ce9991c98f20-istio-data\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 18:07:04.505419 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:07:04.505286 2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8cf24882-6384-48b2-bb2e-ce9991c98f20-istio-envoy\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 18:07:04.505419 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:07:04.505297 2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8cf24882-6384-48b2-bb2e-ce9991c98f20-istio-token\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 18:07:04.505419 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:07:04.505309 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vwpdc\" (UniqueName: \"kubernetes.io/projected/8cf24882-6384-48b2-bb2e-ce9991c98f20-kube-api-access-vwpdc\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 18:07:04.505419 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:07:04.505321 2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8cf24882-6384-48b2-bb2e-ce9991c98f20-istiod-ca-cert\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 18:07:04.505419 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:07:04.505333 2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8cf24882-6384-48b2-bb2e-ce9991c98f20-credential-socket\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 18:07:04.505419 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:07:04.505345 2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8cf24882-6384-48b2-bb2e-ce9991c98f20-workload-certs\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 18:07:04.505419 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:07:04.505357 2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8cf24882-6384-48b2-bb2e-ce9991c98f20-workload-socket\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 18:07:04.702509 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:07:04.702485 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-ef19c3e1-openshift-default-69977b88ff-lt5ws"] Jul 14 18:07:04.706718 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:07:04.706689 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-ef19c3e1-openshift-default-69977b88ff-lt5ws"] Jul 14 18:07:05.518441 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:07:05.518402 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8cf24882-6384-48b2-bb2e-ce9991c98f20" path="/var/lib/kubelet/pods/8cf24882-6384-48b2-bb2e-ce9991c98f20/volumes" Jul 14 18:07:31.501948 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:07:31.501913 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2sbww_c3d00b23-d0d5-4d18-a722-464d7b114745/ovn-acl-logging/0.log" Jul 14 18:07:31.505076 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:07:31.505054 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2sbww_c3d00b23-d0d5-4d18-a722-464d7b114745/ovn-acl-logging/0.log" Jul 14 18:07:58.082585 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:07:58.082551 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-ae8cb8-openshift-default-7798899975-fzxd8"] Jul 14 18:07:58.082967 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:07:58.082881 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8cf24882-6384-48b2-bb2e-ce9991c98f20" containerName="istio-proxy" Jul 14 18:07:58.082967 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:07:58.082892 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="8cf24882-6384-48b2-bb2e-ce9991c98f20" containerName="istio-proxy" Jul 14 18:07:58.082967 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:07:58.082964 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="8cf24882-6384-48b2-bb2e-ce9991c98f20" containerName="istio-proxy" Jul 14 18:07:58.083075 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:07:58.082973 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="c8f26454-670d-4db6-aac0-a195513f89a6" containerName="istio-proxy" Jul 14 18:07:58.085346 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:07:58.085330 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-ae8cb8-openshift-default-7798899975-fzxd8" Jul 14 18:07:58.089525 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:07:58.089507 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-ae8cb8-openshift-default-dockercfg-bqh7c\"" Jul 14 18:07:58.104097 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:07:58.104076 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-ae8cb8-openshift-default-7798899975-fzxd8"] Jul 14 18:07:58.164907 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:07:58.164877 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21-istio-token\") pod \"e2e-inf-a-ae8cb8-openshift-default-7798899975-fzxd8\" (UID: \"663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21\") " pod="openshift-ingress/e2e-inf-a-ae8cb8-openshift-default-7798899975-fzxd8" Jul 14 18:07:58.165064 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:07:58.164943 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21-istiod-ca-cert\") pod \"e2e-inf-a-ae8cb8-openshift-default-7798899975-fzxd8\" (UID: \"663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21\") " pod="openshift-ingress/e2e-inf-a-ae8cb8-openshift-default-7798899975-fzxd8" Jul 14 18:07:58.165064 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:07:58.164972 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21-istio-data\") pod \"e2e-inf-a-ae8cb8-openshift-default-7798899975-fzxd8\" (UID: \"663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21\") " pod="openshift-ingress/e2e-inf-a-ae8cb8-openshift-default-7798899975-fzxd8" Jul 14 18:07:58.165064 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:07:58.164992 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21-istio-envoy\") pod \"e2e-inf-a-ae8cb8-openshift-default-7798899975-fzxd8\" (UID: \"663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21\") " pod="openshift-ingress/e2e-inf-a-ae8cb8-openshift-default-7798899975-fzxd8" Jul 14 18:07:58.165064 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:07:58.165014 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21-workload-socket\") pod \"e2e-inf-a-ae8cb8-openshift-default-7798899975-fzxd8\" (UID: \"663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21\") " pod="openshift-ingress/e2e-inf-a-ae8cb8-openshift-default-7798899975-fzxd8" Jul 14 18:07:58.165064 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:07:58.165038 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21-istio-podinfo\") pod \"e2e-inf-a-ae8cb8-openshift-default-7798899975-fzxd8\" (UID: \"663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21\") " pod="openshift-ingress/e2e-inf-a-ae8cb8-openshift-default-7798899975-fzxd8" Jul 14 18:07:58.165365 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:07:58.165074 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21-credential-socket\") pod \"e2e-inf-a-ae8cb8-openshift-default-7798899975-fzxd8\" (UID: \"663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21\") " pod="openshift-ingress/e2e-inf-a-ae8cb8-openshift-default-7798899975-fzxd8" Jul 14 18:07:58.165365 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:07:58.165102 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2cpz6\" (UniqueName: \"kubernetes.io/projected/663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21-kube-api-access-2cpz6\") pod \"e2e-inf-a-ae8cb8-openshift-default-7798899975-fzxd8\" (UID: \"663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21\") " pod="openshift-ingress/e2e-inf-a-ae8cb8-openshift-default-7798899975-fzxd8" Jul 14 18:07:58.165365 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:07:58.165151 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21-workload-certs\") pod \"e2e-inf-a-ae8cb8-openshift-default-7798899975-fzxd8\" (UID: \"663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21\") " pod="openshift-ingress/e2e-inf-a-ae8cb8-openshift-default-7798899975-fzxd8" Jul 14 18:07:58.266375 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:07:58.266348 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21-istio-token\") pod \"e2e-inf-a-ae8cb8-openshift-default-7798899975-fzxd8\" (UID: \"663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21\") " pod="openshift-ingress/e2e-inf-a-ae8cb8-openshift-default-7798899975-fzxd8" Jul 14 18:07:58.266536 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:07:58.266392 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21-istiod-ca-cert\") pod \"e2e-inf-a-ae8cb8-openshift-default-7798899975-fzxd8\" (UID: \"663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21\") " pod="openshift-ingress/e2e-inf-a-ae8cb8-openshift-default-7798899975-fzxd8" Jul 14 18:07:58.266536 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:07:58.266416 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21-istio-data\") pod \"e2e-inf-a-ae8cb8-openshift-default-7798899975-fzxd8\" (UID: \"663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21\") " pod="openshift-ingress/e2e-inf-a-ae8cb8-openshift-default-7798899975-fzxd8" Jul 14 18:07:58.266536 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:07:58.266441 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21-istio-envoy\") pod \"e2e-inf-a-ae8cb8-openshift-default-7798899975-fzxd8\" (UID: \"663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21\") " pod="openshift-ingress/e2e-inf-a-ae8cb8-openshift-default-7798899975-fzxd8" Jul 14 18:07:58.266536 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:07:58.266467 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21-workload-socket\") pod \"e2e-inf-a-ae8cb8-openshift-default-7798899975-fzxd8\" (UID: \"663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21\") " pod="openshift-ingress/e2e-inf-a-ae8cb8-openshift-default-7798899975-fzxd8" Jul 14 18:07:58.266536 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:07:58.266487 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21-istio-podinfo\") pod \"e2e-inf-a-ae8cb8-openshift-default-7798899975-fzxd8\" (UID: \"663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21\") " pod="openshift-ingress/e2e-inf-a-ae8cb8-openshift-default-7798899975-fzxd8" Jul 14 18:07:58.266536 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:07:58.266511 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21-credential-socket\") pod \"e2e-inf-a-ae8cb8-openshift-default-7798899975-fzxd8\" (UID: \"663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21\") " pod="openshift-ingress/e2e-inf-a-ae8cb8-openshift-default-7798899975-fzxd8" Jul 14 18:07:58.266869 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:07:58.266730 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2cpz6\" (UniqueName: \"kubernetes.io/projected/663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21-kube-api-access-2cpz6\") pod \"e2e-inf-a-ae8cb8-openshift-default-7798899975-fzxd8\" (UID: \"663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21\") " pod="openshift-ingress/e2e-inf-a-ae8cb8-openshift-default-7798899975-fzxd8" Jul 14 18:07:58.266869 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:07:58.266791 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21-workload-certs\") pod \"e2e-inf-a-ae8cb8-openshift-default-7798899975-fzxd8\" (UID: \"663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21\") " pod="openshift-ingress/e2e-inf-a-ae8cb8-openshift-default-7798899975-fzxd8" Jul 14 18:07:58.266977 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:07:58.266935 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21-istio-data\") pod \"e2e-inf-a-ae8cb8-openshift-default-7798899975-fzxd8\" (UID: \"663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21\") " pod="openshift-ingress/e2e-inf-a-ae8cb8-openshift-default-7798899975-fzxd8" Jul 14 18:07:58.267032 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:07:58.267001 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21-workload-socket\") pod \"e2e-inf-a-ae8cb8-openshift-default-7798899975-fzxd8\" (UID: \"663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21\") " pod="openshift-ingress/e2e-inf-a-ae8cb8-openshift-default-7798899975-fzxd8" Jul 14 18:07:58.267396 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:07:58.267374 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21-workload-certs\") pod \"e2e-inf-a-ae8cb8-openshift-default-7798899975-fzxd8\" (UID: \"663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21\") " pod="openshift-ingress/e2e-inf-a-ae8cb8-openshift-default-7798899975-fzxd8" Jul 14 18:07:58.267498 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:07:58.267380 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21-istiod-ca-cert\") pod \"e2e-inf-a-ae8cb8-openshift-default-7798899975-fzxd8\" (UID: \"663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21\") " pod="openshift-ingress/e2e-inf-a-ae8cb8-openshift-default-7798899975-fzxd8" Jul 14 18:07:58.267498 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:07:58.267410 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21-credential-socket\") pod \"e2e-inf-a-ae8cb8-openshift-default-7798899975-fzxd8\" (UID: \"663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21\") " pod="openshift-ingress/e2e-inf-a-ae8cb8-openshift-default-7798899975-fzxd8" Jul 14 18:07:58.268857 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:07:58.268838 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21-istio-envoy\") pod \"e2e-inf-a-ae8cb8-openshift-default-7798899975-fzxd8\" (UID: \"663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21\") " pod="openshift-ingress/e2e-inf-a-ae8cb8-openshift-default-7798899975-fzxd8" Jul 14 18:07:58.269099 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:07:58.269083 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21-istio-podinfo\") pod \"e2e-inf-a-ae8cb8-openshift-default-7798899975-fzxd8\" (UID: \"663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21\") " pod="openshift-ingress/e2e-inf-a-ae8cb8-openshift-default-7798899975-fzxd8" Jul 14 18:07:58.278097 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:07:58.278077 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2cpz6\" (UniqueName: \"kubernetes.io/projected/663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21-kube-api-access-2cpz6\") pod \"e2e-inf-a-ae8cb8-openshift-default-7798899975-fzxd8\" (UID: \"663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21\") " pod="openshift-ingress/e2e-inf-a-ae8cb8-openshift-default-7798899975-fzxd8" Jul 14 18:07:58.280137 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:07:58.280116 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21-istio-token\") pod \"e2e-inf-a-ae8cb8-openshift-default-7798899975-fzxd8\" (UID: \"663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21\") " pod="openshift-ingress/e2e-inf-a-ae8cb8-openshift-default-7798899975-fzxd8" Jul 14 18:07:58.395743 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:07:58.395715 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-ae8cb8-openshift-default-7798899975-fzxd8" Jul 14 18:07:58.547819 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:07:58.547788 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-ae8cb8-openshift-default-7798899975-fzxd8"] Jul 14 18:07:58.556434 ip-10-0-129-209 kubenswrapper[2562]: W0714 18:07:58.556405 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod663a00f8_e0a9_45ab_b8c2_cc9e2fdb0b21.slice/crio-c7c46c785b71f3e7fc0bcb09f53a750dc546bfdbfab7c51799225a0cd5cfb498 WatchSource:0}: Error finding container c7c46c785b71f3e7fc0bcb09f53a750dc546bfdbfab7c51799225a0cd5cfb498: Status 404 returned error can't find the container with id c7c46c785b71f3e7fc0bcb09f53a750dc546bfdbfab7c51799225a0cd5cfb498 Jul 14 18:07:58.558493 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:07:58.558467 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 14 18:07:58.558551 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:07:58.558528 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 14 18:07:58.558589 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:07:58.558555 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 14 18:07:58.609346 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:07:58.609319 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-ae8cb8-openshift-default-7798899975-fzxd8" event={"ID":"663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21","Type":"ContainerStarted","Data":"c7c46c785b71f3e7fc0bcb09f53a750dc546bfdbfab7c51799225a0cd5cfb498"} Jul 14 18:07:59.614338 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:07:59.614303 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-ae8cb8-openshift-default-7798899975-fzxd8" event={"ID":"663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21","Type":"ContainerStarted","Data":"dbffae9b6bb1b1dc2e6f93eabc01d16cf11586c4a5ba1ecfbedffe0f3c5bb6e8"} Jul 14 18:08:00.396781 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:08:00.396744 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-ae8cb8-openshift-default-7798899975-fzxd8" Jul 14 18:08:00.401280 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:08:00.401257 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-ae8cb8-openshift-default-7798899975-fzxd8" Jul 14 18:08:00.434221 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:08:00.434149 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-ae8cb8-openshift-default-7798899975-fzxd8" podStartSLOduration=2.4341312410000002 podStartE2EDuration="2.434131241s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 18:07:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 18:07:59.645761616 +0000 UTC m=+2128.722219156" watchObservedRunningTime="2026-07-14 18:08:00.434131241 +0000 UTC m=+2129.510588785" Jul 14 18:08:00.619147 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:08:00.619123 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-ae8cb8-openshift-default-7798899975-fzxd8" Jul 14 18:08:00.619891 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:08:00.619873 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-ae8cb8-openshift-default-7798899975-fzxd8" Jul 14 18:08:06.792465 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:08:06.792430 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-9c28f2-openshift-default-7b6cd775f5-h95pj"] Jul 14 18:08:06.792915 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:08:06.792758 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8cf24882-6384-48b2-bb2e-ce9991c98f20" containerName="istio-proxy" Jul 14 18:08:06.792915 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:08:06.792769 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="8cf24882-6384-48b2-bb2e-ce9991c98f20" containerName="istio-proxy" Jul 14 18:08:06.792915 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:08:06.792820 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="8cf24882-6384-48b2-bb2e-ce9991c98f20" containerName="istio-proxy" Jul 14 18:08:06.795608 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:08:06.795579 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-9c28f2-openshift-default-7b6cd775f5-h95pj" Jul 14 18:08:06.800051 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:08:06.800031 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-9c28f2-openshift-default-dockercfg-8n7hh\"" Jul 14 18:08:06.815601 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:08:06.815579 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-9c28f2-openshift-default-7b6cd775f5-h95pj"] Jul 14 18:08:06.947937 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:08:06.947905 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d396a328-bf35-4376-a40f-3a09d9d9dd18-istio-data\") pod \"e2e-inf-b-9c28f2-openshift-default-7b6cd775f5-h95pj\" (UID: \"d396a328-bf35-4376-a40f-3a09d9d9dd18\") " pod="openshift-ingress/e2e-inf-b-9c28f2-openshift-default-7b6cd775f5-h95pj" Jul 14 18:08:06.948139 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:08:06.947948 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d396a328-bf35-4376-a40f-3a09d9d9dd18-credential-socket\") pod \"e2e-inf-b-9c28f2-openshift-default-7b6cd775f5-h95pj\" (UID: \"d396a328-bf35-4376-a40f-3a09d9d9dd18\") " pod="openshift-ingress/e2e-inf-b-9c28f2-openshift-default-7b6cd775f5-h95pj" Jul 14 18:08:06.948139 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:08:06.947966 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d396a328-bf35-4376-a40f-3a09d9d9dd18-istio-token\") pod \"e2e-inf-b-9c28f2-openshift-default-7b6cd775f5-h95pj\" (UID: \"d396a328-bf35-4376-a40f-3a09d9d9dd18\") " pod="openshift-ingress/e2e-inf-b-9c28f2-openshift-default-7b6cd775f5-h95pj" Jul 14 18:08:06.948139 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:08:06.947989 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d396a328-bf35-4376-a40f-3a09d9d9dd18-istio-envoy\") pod \"e2e-inf-b-9c28f2-openshift-default-7b6cd775f5-h95pj\" (UID: \"d396a328-bf35-4376-a40f-3a09d9d9dd18\") " pod="openshift-ingress/e2e-inf-b-9c28f2-openshift-default-7b6cd775f5-h95pj" Jul 14 18:08:06.948139 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:08:06.948069 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d396a328-bf35-4376-a40f-3a09d9d9dd18-workload-certs\") pod \"e2e-inf-b-9c28f2-openshift-default-7b6cd775f5-h95pj\" (UID: \"d396a328-bf35-4376-a40f-3a09d9d9dd18\") " pod="openshift-ingress/e2e-inf-b-9c28f2-openshift-default-7b6cd775f5-h95pj" Jul 14 18:08:06.948139 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:08:06.948116 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-plxcv\" (UniqueName: \"kubernetes.io/projected/d396a328-bf35-4376-a40f-3a09d9d9dd18-kube-api-access-plxcv\") pod \"e2e-inf-b-9c28f2-openshift-default-7b6cd775f5-h95pj\" (UID: \"d396a328-bf35-4376-a40f-3a09d9d9dd18\") " pod="openshift-ingress/e2e-inf-b-9c28f2-openshift-default-7b6cd775f5-h95pj" Jul 14 18:08:06.948434 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:08:06.948215 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d396a328-bf35-4376-a40f-3a09d9d9dd18-workload-socket\") pod \"e2e-inf-b-9c28f2-openshift-default-7b6cd775f5-h95pj\" (UID: \"d396a328-bf35-4376-a40f-3a09d9d9dd18\") " pod="openshift-ingress/e2e-inf-b-9c28f2-openshift-default-7b6cd775f5-h95pj" Jul 14 18:08:06.948434 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:08:06.948277 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d396a328-bf35-4376-a40f-3a09d9d9dd18-istio-podinfo\") pod \"e2e-inf-b-9c28f2-openshift-default-7b6cd775f5-h95pj\" (UID: \"d396a328-bf35-4376-a40f-3a09d9d9dd18\") " pod="openshift-ingress/e2e-inf-b-9c28f2-openshift-default-7b6cd775f5-h95pj" Jul 14 18:08:06.948434 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:08:06.948303 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d396a328-bf35-4376-a40f-3a09d9d9dd18-istiod-ca-cert\") pod \"e2e-inf-b-9c28f2-openshift-default-7b6cd775f5-h95pj\" (UID: \"d396a328-bf35-4376-a40f-3a09d9d9dd18\") " pod="openshift-ingress/e2e-inf-b-9c28f2-openshift-default-7b6cd775f5-h95pj" Jul 14 18:08:07.049579 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:08:07.049498 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d396a328-bf35-4376-a40f-3a09d9d9dd18-istio-podinfo\") pod \"e2e-inf-b-9c28f2-openshift-default-7b6cd775f5-h95pj\" (UID: \"d396a328-bf35-4376-a40f-3a09d9d9dd18\") " pod="openshift-ingress/e2e-inf-b-9c28f2-openshift-default-7b6cd775f5-h95pj" Jul 14 18:08:07.049579 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:08:07.049529 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d396a328-bf35-4376-a40f-3a09d9d9dd18-istiod-ca-cert\") pod \"e2e-inf-b-9c28f2-openshift-default-7b6cd775f5-h95pj\" (UID: \"d396a328-bf35-4376-a40f-3a09d9d9dd18\") " pod="openshift-ingress/e2e-inf-b-9c28f2-openshift-default-7b6cd775f5-h95pj" Jul 14 18:08:07.049579 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:08:07.049572 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d396a328-bf35-4376-a40f-3a09d9d9dd18-istio-data\") pod \"e2e-inf-b-9c28f2-openshift-default-7b6cd775f5-h95pj\" (UID: \"d396a328-bf35-4376-a40f-3a09d9d9dd18\") " pod="openshift-ingress/e2e-inf-b-9c28f2-openshift-default-7b6cd775f5-h95pj" Jul 14 18:08:07.049847 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:08:07.049605 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d396a328-bf35-4376-a40f-3a09d9d9dd18-credential-socket\") pod \"e2e-inf-b-9c28f2-openshift-default-7b6cd775f5-h95pj\" (UID: \"d396a328-bf35-4376-a40f-3a09d9d9dd18\") " pod="openshift-ingress/e2e-inf-b-9c28f2-openshift-default-7b6cd775f5-h95pj" Jul 14 18:08:07.049847 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:08:07.049620 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d396a328-bf35-4376-a40f-3a09d9d9dd18-istio-token\") pod \"e2e-inf-b-9c28f2-openshift-default-7b6cd775f5-h95pj\" (UID: \"d396a328-bf35-4376-a40f-3a09d9d9dd18\") " pod="openshift-ingress/e2e-inf-b-9c28f2-openshift-default-7b6cd775f5-h95pj" Jul 14 18:08:07.049847 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:08:07.049640 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d396a328-bf35-4376-a40f-3a09d9d9dd18-istio-envoy\") pod \"e2e-inf-b-9c28f2-openshift-default-7b6cd775f5-h95pj\" (UID: \"d396a328-bf35-4376-a40f-3a09d9d9dd18\") " pod="openshift-ingress/e2e-inf-b-9c28f2-openshift-default-7b6cd775f5-h95pj" Jul 14 18:08:07.049847 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:08:07.049675 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d396a328-bf35-4376-a40f-3a09d9d9dd18-workload-certs\") pod \"e2e-inf-b-9c28f2-openshift-default-7b6cd775f5-h95pj\" (UID: \"d396a328-bf35-4376-a40f-3a09d9d9dd18\") " pod="openshift-ingress/e2e-inf-b-9c28f2-openshift-default-7b6cd775f5-h95pj" Jul 14 18:08:07.049847 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:08:07.049708 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-plxcv\" (UniqueName: \"kubernetes.io/projected/d396a328-bf35-4376-a40f-3a09d9d9dd18-kube-api-access-plxcv\") pod \"e2e-inf-b-9c28f2-openshift-default-7b6cd775f5-h95pj\" (UID: \"d396a328-bf35-4376-a40f-3a09d9d9dd18\") " pod="openshift-ingress/e2e-inf-b-9c28f2-openshift-default-7b6cd775f5-h95pj" Jul 14 18:08:07.049847 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:08:07.049737 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d396a328-bf35-4376-a40f-3a09d9d9dd18-workload-socket\") pod \"e2e-inf-b-9c28f2-openshift-default-7b6cd775f5-h95pj\" (UID: \"d396a328-bf35-4376-a40f-3a09d9d9dd18\") " pod="openshift-ingress/e2e-inf-b-9c28f2-openshift-default-7b6cd775f5-h95pj" Jul 14 18:08:07.050114 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:08:07.050096 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d396a328-bf35-4376-a40f-3a09d9d9dd18-istio-data\") pod \"e2e-inf-b-9c28f2-openshift-default-7b6cd775f5-h95pj\" (UID: \"d396a328-bf35-4376-a40f-3a09d9d9dd18\") " pod="openshift-ingress/e2e-inf-b-9c28f2-openshift-default-7b6cd775f5-h95pj" Jul 14 18:08:07.050180 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:08:07.050150 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d396a328-bf35-4376-a40f-3a09d9d9dd18-workload-socket\") pod \"e2e-inf-b-9c28f2-openshift-default-7b6cd775f5-h95pj\" (UID: \"d396a328-bf35-4376-a40f-3a09d9d9dd18\") " pod="openshift-ingress/e2e-inf-b-9c28f2-openshift-default-7b6cd775f5-h95pj" Jul 14 18:08:07.050276 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:08:07.050254 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d396a328-bf35-4376-a40f-3a09d9d9dd18-credential-socket\") pod \"e2e-inf-b-9c28f2-openshift-default-7b6cd775f5-h95pj\" (UID: \"d396a328-bf35-4376-a40f-3a09d9d9dd18\") " pod="openshift-ingress/e2e-inf-b-9c28f2-openshift-default-7b6cd775f5-h95pj" Jul 14 18:08:07.050336 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:08:07.050315 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d396a328-bf35-4376-a40f-3a09d9d9dd18-istiod-ca-cert\") pod \"e2e-inf-b-9c28f2-openshift-default-7b6cd775f5-h95pj\" (UID: \"d396a328-bf35-4376-a40f-3a09d9d9dd18\") " pod="openshift-ingress/e2e-inf-b-9c28f2-openshift-default-7b6cd775f5-h95pj" Jul 14 18:08:07.050380 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:08:07.050370 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d396a328-bf35-4376-a40f-3a09d9d9dd18-workload-certs\") pod \"e2e-inf-b-9c28f2-openshift-default-7b6cd775f5-h95pj\" (UID: \"d396a328-bf35-4376-a40f-3a09d9d9dd18\") " pod="openshift-ingress/e2e-inf-b-9c28f2-openshift-default-7b6cd775f5-h95pj" Jul 14 18:08:07.052117 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:08:07.052095 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d396a328-bf35-4376-a40f-3a09d9d9dd18-istio-envoy\") pod \"e2e-inf-b-9c28f2-openshift-default-7b6cd775f5-h95pj\" (UID: \"d396a328-bf35-4376-a40f-3a09d9d9dd18\") " pod="openshift-ingress/e2e-inf-b-9c28f2-openshift-default-7b6cd775f5-h95pj" Jul 14 18:08:07.052356 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:08:07.052337 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d396a328-bf35-4376-a40f-3a09d9d9dd18-istio-podinfo\") pod \"e2e-inf-b-9c28f2-openshift-default-7b6cd775f5-h95pj\" (UID: \"d396a328-bf35-4376-a40f-3a09d9d9dd18\") " pod="openshift-ingress/e2e-inf-b-9c28f2-openshift-default-7b6cd775f5-h95pj" Jul 14 18:08:07.063628 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:08:07.063601 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-plxcv\" (UniqueName: \"kubernetes.io/projected/d396a328-bf35-4376-a40f-3a09d9d9dd18-kube-api-access-plxcv\") pod \"e2e-inf-b-9c28f2-openshift-default-7b6cd775f5-h95pj\" (UID: \"d396a328-bf35-4376-a40f-3a09d9d9dd18\") " pod="openshift-ingress/e2e-inf-b-9c28f2-openshift-default-7b6cd775f5-h95pj" Jul 14 18:08:07.063897 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:08:07.063881 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d396a328-bf35-4376-a40f-3a09d9d9dd18-istio-token\") pod \"e2e-inf-b-9c28f2-openshift-default-7b6cd775f5-h95pj\" (UID: \"d396a328-bf35-4376-a40f-3a09d9d9dd18\") " pod="openshift-ingress/e2e-inf-b-9c28f2-openshift-default-7b6cd775f5-h95pj" Jul 14 18:08:07.106000 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:08:07.105978 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-9c28f2-openshift-default-7b6cd775f5-h95pj" Jul 14 18:08:07.250212 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:08:07.250166 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-9c28f2-openshift-default-7b6cd775f5-h95pj"] Jul 14 18:08:07.251373 ip-10-0-129-209 kubenswrapper[2562]: W0714 18:08:07.251346 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd396a328_bf35_4376_a40f_3a09d9d9dd18.slice/crio-a20a715628d778ce4889d22c1de680bd886ea3aa6c954d6507fb20b99b89c597 WatchSource:0}: Error finding container a20a715628d778ce4889d22c1de680bd886ea3aa6c954d6507fb20b99b89c597: Status 404 returned error can't find the container with id a20a715628d778ce4889d22c1de680bd886ea3aa6c954d6507fb20b99b89c597 Jul 14 18:08:07.253343 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:08:07.253315 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 14 18:08:07.253407 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:08:07.253373 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 14 18:08:07.253407 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:08:07.253400 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 14 18:08:07.652519 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:08:07.652484 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-9c28f2-openshift-default-7b6cd775f5-h95pj" event={"ID":"d396a328-bf35-4376-a40f-3a09d9d9dd18","Type":"ContainerStarted","Data":"8048a5b8a5231f11c448ea7c8d2edc9ff9cd8341cca8e863f96765b84eab96d0"} Jul 14 18:08:07.652715 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:08:07.652548 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-9c28f2-openshift-default-7b6cd775f5-h95pj" event={"ID":"d396a328-bf35-4376-a40f-3a09d9d9dd18","Type":"ContainerStarted","Data":"a20a715628d778ce4889d22c1de680bd886ea3aa6c954d6507fb20b99b89c597"} Jul 14 18:08:07.689546 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:08:07.689499 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-9c28f2-openshift-default-7b6cd775f5-h95pj" podStartSLOduration=1.689482923 podStartE2EDuration="1.689482923s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 18:08:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 18:08:07.688342217 +0000 UTC m=+2136.764799773" watchObservedRunningTime="2026-07-14 18:08:07.689482923 +0000 UTC m=+2136.765940462" Jul 14 18:08:08.106379 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:08:08.106302 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-9c28f2-openshift-default-7b6cd775f5-h95pj" Jul 14 18:08:08.110887 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:08:08.110864 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-9c28f2-openshift-default-7b6cd775f5-h95pj" Jul 14 18:08:08.656876 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:08:08.656845 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-9c28f2-openshift-default-7b6cd775f5-h95pj" Jul 14 18:08:08.657851 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:08:08.657826 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-9c28f2-openshift-default-7b6cd775f5-h95pj" Jul 14 18:08:16.184566 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:08:16.184517 2562 patch_prober.go:28] interesting pod/e2e-inf-a-ae8cb8-openshift-default-7798899975-fzxd8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.33:15021/healthz/ready\": read tcp 10.134.0.2:35230->10.134.0.33:15021: read: connection reset by peer" start-of-body= Jul 14 18:08:16.185017 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:08:16.184629 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-ae8cb8-openshift-default-7798899975-fzxd8" podUID="663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.33:15021/healthz/ready\": read tcp 10.134.0.2:35230->10.134.0.33:15021: read: connection reset by peer" Jul 14 18:08:16.704275 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:08:16.704239 2562 generic.go:356] "Generic (PLEG): container finished" podID="663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21" containerID="dbffae9b6bb1b1dc2e6f93eabc01d16cf11586c4a5ba1ecfbedffe0f3c5bb6e8" exitCode=0 Jul 14 18:08:16.704445 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:08:16.704317 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-ae8cb8-openshift-default-7798899975-fzxd8" event={"ID":"663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21","Type":"ContainerDied","Data":"dbffae9b6bb1b1dc2e6f93eabc01d16cf11586c4a5ba1ecfbedffe0f3c5bb6e8"} Jul 14 18:08:16.704762 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:08:16.704746 2562 scope.go:117] "RemoveContainer" containerID="dbffae9b6bb1b1dc2e6f93eabc01d16cf11586c4a5ba1ecfbedffe0f3c5bb6e8" Jul 14 18:08:16.705754 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:08:16.705728 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 14 18:08:16.705875 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:08:16.705807 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 14 18:08:16.705875 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:08:16.705850 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 14 18:08:17.711447 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:08:17.711399 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-ae8cb8-openshift-default-7798899975-fzxd8" event={"ID":"663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21","Type":"ContainerStarted","Data":"3768140ba599b0b4facbc4055cb1e3f4eecc893e04dfd774343909ca9618156c"} Jul 14 18:08:18.396515 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:08:18.396470 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-ae8cb8-openshift-default-7798899975-fzxd8" Jul 14 18:08:18.396763 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:08:18.396739 2562 patch_prober.go:28] interesting pod/e2e-inf-a-ae8cb8-openshift-default-7798899975-fzxd8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" start-of-body= Jul 14 18:08:18.396833 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:08:18.396798 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-ae8cb8-openshift-default-7798899975-fzxd8" podUID="663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" Jul 14 18:08:19.396264 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:08:19.396224 2562 patch_prober.go:28] interesting pod/e2e-inf-a-ae8cb8-openshift-default-7798899975-fzxd8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" start-of-body= Jul 14 18:08:19.396786 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:08:19.396286 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-ae8cb8-openshift-default-7798899975-fzxd8" podUID="663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" Jul 14 18:08:20.397064 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:08:20.397029 2562 patch_prober.go:28] interesting pod/e2e-inf-a-ae8cb8-openshift-default-7798899975-fzxd8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" start-of-body= Jul 14 18:08:20.397469 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:08:20.397088 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-ae8cb8-openshift-default-7798899975-fzxd8" podUID="663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" Jul 14 18:08:21.396896 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:08:21.396860 2562 patch_prober.go:28] interesting pod/e2e-inf-a-ae8cb8-openshift-default-7798899975-fzxd8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" start-of-body= Jul 14 18:08:21.397119 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:08:21.396939 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-ae8cb8-openshift-default-7798899975-fzxd8" podUID="663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.33:15021/healthz/ready\": dial tcp 10.134.0.33:15021: connect: connection refused" Jul 14 18:08:21.733122 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:08:21.733092 2562 generic.go:356] "Generic (PLEG): container finished" podID="d396a328-bf35-4376-a40f-3a09d9d9dd18" containerID="8048a5b8a5231f11c448ea7c8d2edc9ff9cd8341cca8e863f96765b84eab96d0" exitCode=0 Jul 14 18:08:21.733122 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:08:21.733127 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-9c28f2-openshift-default-7b6cd775f5-h95pj" event={"ID":"d396a328-bf35-4376-a40f-3a09d9d9dd18","Type":"ContainerDied","Data":"8048a5b8a5231f11c448ea7c8d2edc9ff9cd8341cca8e863f96765b84eab96d0"} Jul 14 18:08:21.733537 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:08:21.733523 2562 scope.go:117] "RemoveContainer" containerID="8048a5b8a5231f11c448ea7c8d2edc9ff9cd8341cca8e863f96765b84eab96d0" Jul 14 18:08:21.734538 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:08:21.734514 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 14 18:08:21.734645 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:08:21.734572 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 14 18:08:21.734645 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:08:21.734613 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 14 18:08:22.399147 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:08:22.399117 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-ae8cb8-openshift-default-7798899975-fzxd8" Jul 14 18:08:22.399619 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:08:22.399410 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-ae8cb8-openshift-default-7798899975-fzxd8" Jul 14 18:08:22.400283 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:08:22.400266 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-ae8cb8-openshift-default-7798899975-fzxd8" Jul 14 18:08:22.739618 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:08:22.739528 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-9c28f2-openshift-default-7b6cd775f5-h95pj" event={"ID":"d396a328-bf35-4376-a40f-3a09d9d9dd18","Type":"ContainerStarted","Data":"8f6fb6ae464da352727c69f3c09159e692fee5123f30f6b708e5f02a9f29c144"} Jul 14 18:08:23.106791 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:08:23.106689 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-9c28f2-openshift-default-7b6cd775f5-h95pj" Jul 14 18:08:23.107901 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:08:23.107871 2562 patch_prober.go:28] interesting pod/e2e-inf-b-9c28f2-openshift-default-7b6cd775f5-h95pj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body= Jul 14 18:08:23.108027 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:08:23.107950 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-9c28f2-openshift-default-7b6cd775f5-h95pj" podUID="d396a328-bf35-4376-a40f-3a09d9d9dd18" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" Jul 14 18:08:23.656930 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:08:23.656894 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-9c28f2-openshift-default-7b6cd775f5-h95pj" Jul 14 18:08:24.106497 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:08:24.106413 2562 patch_prober.go:28] interesting pod/e2e-inf-b-9c28f2-openshift-default-7b6cd775f5-h95pj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body= Jul 14 18:08:24.106497 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:08:24.106484 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-9c28f2-openshift-default-7b6cd775f5-h95pj" podUID="d396a328-bf35-4376-a40f-3a09d9d9dd18" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" Jul 14 18:08:25.107097 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:08:25.107052 2562 patch_prober.go:28] interesting pod/e2e-inf-b-9c28f2-openshift-default-7b6cd775f5-h95pj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body= Jul 14 18:08:25.107528 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:08:25.107131 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-9c28f2-openshift-default-7b6cd775f5-h95pj" podUID="d396a328-bf35-4376-a40f-3a09d9d9dd18" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" Jul 14 18:08:26.106633 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:08:26.106598 2562 patch_prober.go:28] interesting pod/e2e-inf-b-9c28f2-openshift-default-7b6cd775f5-h95pj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body= Jul 14 18:08:26.106811 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:08:26.106665 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-9c28f2-openshift-default-7b6cd775f5-h95pj" podUID="d396a328-bf35-4376-a40f-3a09d9d9dd18" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" Jul 14 18:08:27.110043 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:08:27.110014 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-9c28f2-openshift-default-7b6cd775f5-h95pj" Jul 14 18:08:27.110923 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:08:27.110905 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-9c28f2-openshift-default-7b6cd775f5-h95pj" Jul 14 18:10:20.684881 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:20.684851 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-ae8cb8-openshift-default-7798899975-fzxd8"] Jul 14 18:10:20.685453 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:20.685056 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-ae8cb8-openshift-default-7798899975-fzxd8" podUID="663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21" containerName="istio-proxy" containerID="cri-o://3768140ba599b0b4facbc4055cb1e3f4eecc893e04dfd774343909ca9618156c" gracePeriod=30 Jul 14 18:10:22.401397 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:22.401356 2562 patch_prober.go:28] interesting pod/e2e-inf-a-ae8cb8-openshift-default-7798899975-fzxd8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 14 18:10:22.401773 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:22.401410 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-ae8cb8-openshift-default-7798899975-fzxd8" podUID="663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 14 18:10:25.925861 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:25.925838 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-ae8cb8-openshift-default-7798899975-fzxd8" Jul 14 18:10:25.950465 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:25.950406 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21-istio-podinfo\") pod \"663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21\" (UID: \"663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21\") " Jul 14 18:10:25.950584 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:25.950465 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21-istio-data\") pod \"663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21\" (UID: \"663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21\") " Jul 14 18:10:25.950647 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:25.950586 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21-workload-socket\") pod \"663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21\" (UID: \"663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21\") " Jul 14 18:10:25.950703 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:25.950677 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21-istio-token\") pod \"663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21\" (UID: \"663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21\") " Jul 14 18:10:25.950765 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:25.950704 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21-istiod-ca-cert\") pod \"663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21\" (UID: \"663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21\") " Jul 14 18:10:25.950765 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:25.950740 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21-istio-envoy\") pod \"663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21\" (UID: \"663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21\") " Jul 14 18:10:25.950871 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:25.950764 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21-credential-socket\") pod \"663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21\" (UID: \"663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21\") " Jul 14 18:10:25.950871 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:25.950823 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21" (UID: "663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 18:10:25.950974 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:25.950878 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2cpz6\" (UniqueName: \"kubernetes.io/projected/663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21-kube-api-access-2cpz6\") pod \"663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21\" (UID: \"663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21\") " Jul 14 18:10:25.950974 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:25.950943 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21-workload-certs\") pod \"663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21\" (UID: \"663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21\") " Jul 14 18:10:25.951078 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:25.950979 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21" (UID: "663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 18:10:25.951078 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:25.951048 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21" (UID: "663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 18:10:25.951273 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:25.951251 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21" (UID: "663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 18:10:25.951356 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:25.951334 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21-istio-data" (OuterVolumeSpecName: "istio-data") pod "663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21" (UID: "663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 18:10:25.951423 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:25.951345 2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21-workload-socket\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 18:10:25.951423 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:25.951382 2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21-istiod-ca-cert\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 18:10:25.951423 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:25.951398 2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21-credential-socket\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 18:10:25.951423 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:25.951412 2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21-workload-certs\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 18:10:25.952913 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:25.952892 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21-istio-token" (OuterVolumeSpecName: "istio-token") pod "663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21" (UID: "663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 18:10:25.953055 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:25.953039 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21-kube-api-access-2cpz6" (OuterVolumeSpecName: "kube-api-access-2cpz6") pod "663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21" (UID: "663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21"). InnerVolumeSpecName "kube-api-access-2cpz6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 18:10:25.953525 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:25.953499 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21" (UID: "663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 18:10:25.953596 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:25.953577 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21" (UID: "663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 14 18:10:26.052022 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:26.051996 2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21-istio-token\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 18:10:26.052022 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:26.052020 2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21-istio-envoy\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 18:10:26.052214 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:26.052029 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2cpz6\" (UniqueName: \"kubernetes.io/projected/663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21-kube-api-access-2cpz6\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 18:10:26.052214 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:26.052039 2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21-istio-podinfo\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 18:10:26.052214 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:26.052047 2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21-istio-data\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 18:10:26.278836 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:26.278760 2562 generic.go:356] "Generic (PLEG): container finished" podID="663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21" containerID="3768140ba599b0b4facbc4055cb1e3f4eecc893e04dfd774343909ca9618156c" exitCode=0 Jul 14 18:10:26.278836 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:26.278813 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-ae8cb8-openshift-default-7798899975-fzxd8" Jul 14 18:10:26.279091 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:26.278847 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-ae8cb8-openshift-default-7798899975-fzxd8" event={"ID":"663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21","Type":"ContainerDied","Data":"3768140ba599b0b4facbc4055cb1e3f4eecc893e04dfd774343909ca9618156c"} Jul 14 18:10:26.279091 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:26.278905 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-ae8cb8-openshift-default-7798899975-fzxd8" event={"ID":"663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21","Type":"ContainerDied","Data":"c7c46c785b71f3e7fc0bcb09f53a750dc546bfdbfab7c51799225a0cd5cfb498"} Jul 14 18:10:26.279091 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:26.278926 2562 scope.go:117] "RemoveContainer" containerID="3768140ba599b0b4facbc4055cb1e3f4eecc893e04dfd774343909ca9618156c" Jul 14 18:10:26.287963 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:26.287947 2562 scope.go:117] "RemoveContainer" containerID="dbffae9b6bb1b1dc2e6f93eabc01d16cf11586c4a5ba1ecfbedffe0f3c5bb6e8" Jul 14 18:10:26.295362 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:26.295346 2562 scope.go:117] "RemoveContainer" containerID="3768140ba599b0b4facbc4055cb1e3f4eecc893e04dfd774343909ca9618156c" Jul 14 18:10:26.295608 ip-10-0-129-209 kubenswrapper[2562]: E0714 18:10:26.295591 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3768140ba599b0b4facbc4055cb1e3f4eecc893e04dfd774343909ca9618156c\": container with ID starting with 3768140ba599b0b4facbc4055cb1e3f4eecc893e04dfd774343909ca9618156c not found: ID does not exist" containerID="3768140ba599b0b4facbc4055cb1e3f4eecc893e04dfd774343909ca9618156c" Jul 14 18:10:26.295657 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:26.295616 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3768140ba599b0b4facbc4055cb1e3f4eecc893e04dfd774343909ca9618156c"} err="failed to get container status \"3768140ba599b0b4facbc4055cb1e3f4eecc893e04dfd774343909ca9618156c\": rpc error: code = NotFound desc = could not find container \"3768140ba599b0b4facbc4055cb1e3f4eecc893e04dfd774343909ca9618156c\": container with ID starting with 3768140ba599b0b4facbc4055cb1e3f4eecc893e04dfd774343909ca9618156c not found: ID does not exist" Jul 14 18:10:26.295657 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:26.295633 2562 scope.go:117] "RemoveContainer" containerID="dbffae9b6bb1b1dc2e6f93eabc01d16cf11586c4a5ba1ecfbedffe0f3c5bb6e8" Jul 14 18:10:26.295869 ip-10-0-129-209 kubenswrapper[2562]: E0714 18:10:26.295852 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dbffae9b6bb1b1dc2e6f93eabc01d16cf11586c4a5ba1ecfbedffe0f3c5bb6e8\": container with ID starting with dbffae9b6bb1b1dc2e6f93eabc01d16cf11586c4a5ba1ecfbedffe0f3c5bb6e8 not found: ID does not exist" containerID="dbffae9b6bb1b1dc2e6f93eabc01d16cf11586c4a5ba1ecfbedffe0f3c5bb6e8" Jul 14 18:10:26.295915 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:26.295874 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dbffae9b6bb1b1dc2e6f93eabc01d16cf11586c4a5ba1ecfbedffe0f3c5bb6e8"} err="failed to get container status \"dbffae9b6bb1b1dc2e6f93eabc01d16cf11586c4a5ba1ecfbedffe0f3c5bb6e8\": rpc error: code = NotFound desc = could not find container \"dbffae9b6bb1b1dc2e6f93eabc01d16cf11586c4a5ba1ecfbedffe0f3c5bb6e8\": container with ID starting with dbffae9b6bb1b1dc2e6f93eabc01d16cf11586c4a5ba1ecfbedffe0f3c5bb6e8 not found: ID does not exist" Jul 14 18:10:26.301762 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:26.301731 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-ae8cb8-openshift-default-7798899975-fzxd8"] Jul 14 18:10:26.307658 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:26.307635 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-ae8cb8-openshift-default-7798899975-fzxd8"] Jul 14 18:10:27.519482 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:27.519449 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21" path="/var/lib/kubelet/pods/663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21/volumes" Jul 14 18:10:46.996040 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:46.996008 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-9c28f2-openshift-default-7b6cd775f5-h95pj"] Jul 14 18:10:46.996440 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:46.996230 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-9c28f2-openshift-default-7b6cd775f5-h95pj" podUID="d396a328-bf35-4376-a40f-3a09d9d9dd18" containerName="istio-proxy" containerID="cri-o://8f6fb6ae464da352727c69f3c09159e692fee5123f30f6b708e5f02a9f29c144" gracePeriod=30 Jul 14 18:10:47.958775 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:47.958742 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-5a43f9-openshift-default-549d8686f9-ztvt5"] Jul 14 18:10:47.959175 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:47.959157 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21" containerName="istio-proxy" Jul 14 18:10:47.959302 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:47.959177 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21" containerName="istio-proxy" Jul 14 18:10:47.959302 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:47.959208 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21" containerName="istio-proxy" Jul 14 18:10:47.959302 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:47.959217 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21" containerName="istio-proxy" Jul 14 18:10:47.959453 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:47.959304 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21" containerName="istio-proxy" Jul 14 18:10:47.959453 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:47.959318 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="663a00f8-e0a9-45ab-b8c2-cc9e2fdb0b21" containerName="istio-proxy" Jul 14 18:10:47.961236 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:47.961215 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-5a43f9-openshift-default-549d8686f9-ztvt5" Jul 14 18:10:47.971881 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:47.971864 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-5a43f9-openshift-default-dockercfg-cxsdw\"" Jul 14 18:10:47.989525 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:47.989502 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-5a43f9-openshift-default-549d8686f9-ztvt5"] Jul 14 18:10:48.045141 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:48.045104 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-phzqc\" (UniqueName: \"kubernetes.io/projected/36c0e46a-2f62-479e-a42f-c910c0d5edaf-kube-api-access-phzqc\") pod \"e2e-api-a-5a43f9-openshift-default-549d8686f9-ztvt5\" (UID: \"36c0e46a-2f62-479e-a42f-c910c0d5edaf\") " pod="openshift-ingress/e2e-api-a-5a43f9-openshift-default-549d8686f9-ztvt5" Jul 14 18:10:48.045565 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:48.045173 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/36c0e46a-2f62-479e-a42f-c910c0d5edaf-workload-socket\") pod \"e2e-api-a-5a43f9-openshift-default-549d8686f9-ztvt5\" (UID: \"36c0e46a-2f62-479e-a42f-c910c0d5edaf\") " pod="openshift-ingress/e2e-api-a-5a43f9-openshift-default-549d8686f9-ztvt5" Jul 14 18:10:48.045565 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:48.045255 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/36c0e46a-2f62-479e-a42f-c910c0d5edaf-istio-token\") pod \"e2e-api-a-5a43f9-openshift-default-549d8686f9-ztvt5\" (UID: \"36c0e46a-2f62-479e-a42f-c910c0d5edaf\") " pod="openshift-ingress/e2e-api-a-5a43f9-openshift-default-549d8686f9-ztvt5" Jul 14 18:10:48.045565 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:48.045344 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/36c0e46a-2f62-479e-a42f-c910c0d5edaf-workload-certs\") pod \"e2e-api-a-5a43f9-openshift-default-549d8686f9-ztvt5\" (UID: \"36c0e46a-2f62-479e-a42f-c910c0d5edaf\") " pod="openshift-ingress/e2e-api-a-5a43f9-openshift-default-549d8686f9-ztvt5" Jul 14 18:10:48.045565 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:48.045381 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/36c0e46a-2f62-479e-a42f-c910c0d5edaf-credential-socket\") pod \"e2e-api-a-5a43f9-openshift-default-549d8686f9-ztvt5\" (UID: \"36c0e46a-2f62-479e-a42f-c910c0d5edaf\") " pod="openshift-ingress/e2e-api-a-5a43f9-openshift-default-549d8686f9-ztvt5" Jul 14 18:10:48.045565 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:48.045414 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/36c0e46a-2f62-479e-a42f-c910c0d5edaf-istio-envoy\") pod \"e2e-api-a-5a43f9-openshift-default-549d8686f9-ztvt5\" (UID: \"36c0e46a-2f62-479e-a42f-c910c0d5edaf\") " pod="openshift-ingress/e2e-api-a-5a43f9-openshift-default-549d8686f9-ztvt5" Jul 14 18:10:48.045565 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:48.045472 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/36c0e46a-2f62-479e-a42f-c910c0d5edaf-istio-podinfo\") pod \"e2e-api-a-5a43f9-openshift-default-549d8686f9-ztvt5\" (UID: \"36c0e46a-2f62-479e-a42f-c910c0d5edaf\") " pod="openshift-ingress/e2e-api-a-5a43f9-openshift-default-549d8686f9-ztvt5" Jul 14 18:10:48.045565 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:48.045500 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/36c0e46a-2f62-479e-a42f-c910c0d5edaf-istiod-ca-cert\") pod \"e2e-api-a-5a43f9-openshift-default-549d8686f9-ztvt5\" (UID: \"36c0e46a-2f62-479e-a42f-c910c0d5edaf\") " pod="openshift-ingress/e2e-api-a-5a43f9-openshift-default-549d8686f9-ztvt5" Jul 14 18:10:48.045565 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:48.045548 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/36c0e46a-2f62-479e-a42f-c910c0d5edaf-istio-data\") pod \"e2e-api-a-5a43f9-openshift-default-549d8686f9-ztvt5\" (UID: \"36c0e46a-2f62-479e-a42f-c910c0d5edaf\") " pod="openshift-ingress/e2e-api-a-5a43f9-openshift-default-549d8686f9-ztvt5" Jul 14 18:10:48.146552 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:48.146508 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/36c0e46a-2f62-479e-a42f-c910c0d5edaf-istio-podinfo\") pod \"e2e-api-a-5a43f9-openshift-default-549d8686f9-ztvt5\" (UID: \"36c0e46a-2f62-479e-a42f-c910c0d5edaf\") " pod="openshift-ingress/e2e-api-a-5a43f9-openshift-default-549d8686f9-ztvt5" Jul 14 18:10:48.146552 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:48.146556 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/36c0e46a-2f62-479e-a42f-c910c0d5edaf-istiod-ca-cert\") pod \"e2e-api-a-5a43f9-openshift-default-549d8686f9-ztvt5\" (UID: \"36c0e46a-2f62-479e-a42f-c910c0d5edaf\") " pod="openshift-ingress/e2e-api-a-5a43f9-openshift-default-549d8686f9-ztvt5" Jul 14 18:10:48.146801 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:48.146690 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/36c0e46a-2f62-479e-a42f-c910c0d5edaf-istio-data\") pod \"e2e-api-a-5a43f9-openshift-default-549d8686f9-ztvt5\" (UID: \"36c0e46a-2f62-479e-a42f-c910c0d5edaf\") " pod="openshift-ingress/e2e-api-a-5a43f9-openshift-default-549d8686f9-ztvt5" Jul 14 18:10:48.146801 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:48.146750 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-phzqc\" (UniqueName: \"kubernetes.io/projected/36c0e46a-2f62-479e-a42f-c910c0d5edaf-kube-api-access-phzqc\") pod \"e2e-api-a-5a43f9-openshift-default-549d8686f9-ztvt5\" (UID: \"36c0e46a-2f62-479e-a42f-c910c0d5edaf\") " pod="openshift-ingress/e2e-api-a-5a43f9-openshift-default-549d8686f9-ztvt5" Jul 14 18:10:48.146801 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:48.146797 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/36c0e46a-2f62-479e-a42f-c910c0d5edaf-workload-socket\") pod \"e2e-api-a-5a43f9-openshift-default-549d8686f9-ztvt5\" (UID: \"36c0e46a-2f62-479e-a42f-c910c0d5edaf\") " pod="openshift-ingress/e2e-api-a-5a43f9-openshift-default-549d8686f9-ztvt5" Jul 14 18:10:48.146972 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:48.146816 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/36c0e46a-2f62-479e-a42f-c910c0d5edaf-istio-token\") pod \"e2e-api-a-5a43f9-openshift-default-549d8686f9-ztvt5\" (UID: \"36c0e46a-2f62-479e-a42f-c910c0d5edaf\") " pod="openshift-ingress/e2e-api-a-5a43f9-openshift-default-549d8686f9-ztvt5" Jul 14 18:10:48.146972 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:48.146893 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/36c0e46a-2f62-479e-a42f-c910c0d5edaf-workload-certs\") pod \"e2e-api-a-5a43f9-openshift-default-549d8686f9-ztvt5\" (UID: \"36c0e46a-2f62-479e-a42f-c910c0d5edaf\") " pod="openshift-ingress/e2e-api-a-5a43f9-openshift-default-549d8686f9-ztvt5" Jul 14 18:10:48.146972 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:48.146927 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/36c0e46a-2f62-479e-a42f-c910c0d5edaf-credential-socket\") pod \"e2e-api-a-5a43f9-openshift-default-549d8686f9-ztvt5\" (UID: \"36c0e46a-2f62-479e-a42f-c910c0d5edaf\") " pod="openshift-ingress/e2e-api-a-5a43f9-openshift-default-549d8686f9-ztvt5" Jul 14 18:10:48.146972 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:48.146961 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/36c0e46a-2f62-479e-a42f-c910c0d5edaf-istio-envoy\") pod \"e2e-api-a-5a43f9-openshift-default-549d8686f9-ztvt5\" (UID: \"36c0e46a-2f62-479e-a42f-c910c0d5edaf\") " pod="openshift-ingress/e2e-api-a-5a43f9-openshift-default-549d8686f9-ztvt5" Jul 14 18:10:48.147203 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:48.147064 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/36c0e46a-2f62-479e-a42f-c910c0d5edaf-istio-data\") pod \"e2e-api-a-5a43f9-openshift-default-549d8686f9-ztvt5\" (UID: \"36c0e46a-2f62-479e-a42f-c910c0d5edaf\") " pod="openshift-ingress/e2e-api-a-5a43f9-openshift-default-549d8686f9-ztvt5" Jul 14 18:10:48.147203 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:48.147121 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/36c0e46a-2f62-479e-a42f-c910c0d5edaf-workload-socket\") pod \"e2e-api-a-5a43f9-openshift-default-549d8686f9-ztvt5\" (UID: \"36c0e46a-2f62-479e-a42f-c910c0d5edaf\") " pod="openshift-ingress/e2e-api-a-5a43f9-openshift-default-549d8686f9-ztvt5" Jul 14 18:10:48.147320 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:48.147285 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/36c0e46a-2f62-479e-a42f-c910c0d5edaf-istiod-ca-cert\") pod \"e2e-api-a-5a43f9-openshift-default-549d8686f9-ztvt5\" (UID: \"36c0e46a-2f62-479e-a42f-c910c0d5edaf\") " pod="openshift-ingress/e2e-api-a-5a43f9-openshift-default-549d8686f9-ztvt5" Jul 14 18:10:48.147320 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:48.147303 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/36c0e46a-2f62-479e-a42f-c910c0d5edaf-workload-certs\") pod \"e2e-api-a-5a43f9-openshift-default-549d8686f9-ztvt5\" (UID: \"36c0e46a-2f62-479e-a42f-c910c0d5edaf\") " pod="openshift-ingress/e2e-api-a-5a43f9-openshift-default-549d8686f9-ztvt5" Jul 14 18:10:48.147588 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:48.147368 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/36c0e46a-2f62-479e-a42f-c910c0d5edaf-credential-socket\") pod \"e2e-api-a-5a43f9-openshift-default-549d8686f9-ztvt5\" (UID: \"36c0e46a-2f62-479e-a42f-c910c0d5edaf\") " pod="openshift-ingress/e2e-api-a-5a43f9-openshift-default-549d8686f9-ztvt5" Jul 14 18:10:48.149050 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:48.149016 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/36c0e46a-2f62-479e-a42f-c910c0d5edaf-istio-envoy\") pod \"e2e-api-a-5a43f9-openshift-default-549d8686f9-ztvt5\" (UID: \"36c0e46a-2f62-479e-a42f-c910c0d5edaf\") " pod="openshift-ingress/e2e-api-a-5a43f9-openshift-default-549d8686f9-ztvt5" Jul 14 18:10:48.149214 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:48.149153 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/36c0e46a-2f62-479e-a42f-c910c0d5edaf-istio-podinfo\") pod \"e2e-api-a-5a43f9-openshift-default-549d8686f9-ztvt5\" (UID: \"36c0e46a-2f62-479e-a42f-c910c0d5edaf\") " pod="openshift-ingress/e2e-api-a-5a43f9-openshift-default-549d8686f9-ztvt5" Jul 14 18:10:48.156551 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:48.156530 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-phzqc\" (UniqueName: \"kubernetes.io/projected/36c0e46a-2f62-479e-a42f-c910c0d5edaf-kube-api-access-phzqc\") pod \"e2e-api-a-5a43f9-openshift-default-549d8686f9-ztvt5\" (UID: \"36c0e46a-2f62-479e-a42f-c910c0d5edaf\") " pod="openshift-ingress/e2e-api-a-5a43f9-openshift-default-549d8686f9-ztvt5" Jul 14 18:10:48.157817 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:48.157795 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/36c0e46a-2f62-479e-a42f-c910c0d5edaf-istio-token\") pod \"e2e-api-a-5a43f9-openshift-default-549d8686f9-ztvt5\" (UID: \"36c0e46a-2f62-479e-a42f-c910c0d5edaf\") " pod="openshift-ingress/e2e-api-a-5a43f9-openshift-default-549d8686f9-ztvt5" Jul 14 18:10:48.271448 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:48.271363 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-5a43f9-openshift-default-549d8686f9-ztvt5" Jul 14 18:10:48.427896 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:48.427862 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-5a43f9-openshift-default-549d8686f9-ztvt5"] Jul 14 18:10:48.437809 ip-10-0-129-209 kubenswrapper[2562]: W0714 18:10:48.437784 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36c0e46a_2f62_479e_a42f_c910c0d5edaf.slice/crio-26f01ffe8dd62c997a27dacf6a4538a79b8b77dbe18dce30efffbc6e37572aa5 WatchSource:0}: Error finding container 26f01ffe8dd62c997a27dacf6a4538a79b8b77dbe18dce30efffbc6e37572aa5: Status 404 returned error can't find the container with id 26f01ffe8dd62c997a27dacf6a4538a79b8b77dbe18dce30efffbc6e37572aa5 Jul 14 18:10:48.439828 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:48.439797 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 14 18:10:48.439900 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:48.439866 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 14 18:10:48.439900 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:48.439896 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 14 18:10:49.385438 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:49.385405 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-5a43f9-openshift-default-549d8686f9-ztvt5" event={"ID":"36c0e46a-2f62-479e-a42f-c910c0d5edaf","Type":"ContainerStarted","Data":"824a0fd38b12b1737b80fff3e595b8299cdc7409920b5f901bddc1295e6a56f6"} Jul 14 18:10:49.385819 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:49.385457 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-5a43f9-openshift-default-549d8686f9-ztvt5" event={"ID":"36c0e46a-2f62-479e-a42f-c910c0d5edaf","Type":"ContainerStarted","Data":"26f01ffe8dd62c997a27dacf6a4538a79b8b77dbe18dce30efffbc6e37572aa5"} Jul 14 18:10:49.426575 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:49.426521 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-5a43f9-openshift-default-549d8686f9-ztvt5" podStartSLOduration=2.426501579 podStartE2EDuration="2.426501579s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 18:10:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 18:10:49.425801938 +0000 UTC m=+2298.502259497" watchObservedRunningTime="2026-07-14 18:10:49.426501579 +0000 UTC m=+2298.502959118" Jul 14 18:10:50.271822 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:50.271786 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-5a43f9-openshift-default-549d8686f9-ztvt5" Jul 14 18:10:50.276454 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:50.276429 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-5a43f9-openshift-default-549d8686f9-ztvt5" Jul 14 18:10:50.389818 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:50.389789 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-5a43f9-openshift-default-549d8686f9-ztvt5" Jul 14 18:10:50.390606 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:50.390586 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-5a43f9-openshift-default-549d8686f9-ztvt5" Jul 14 18:10:52.240089 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:52.240061 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-9c28f2-openshift-default-7b6cd775f5-h95pj" Jul 14 18:10:52.384066 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:52.384044 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d396a328-bf35-4376-a40f-3a09d9d9dd18-istio-token\") pod \"d396a328-bf35-4376-a40f-3a09d9d9dd18\" (UID: \"d396a328-bf35-4376-a40f-3a09d9d9dd18\") " Jul 14 18:10:52.384245 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:52.384123 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d396a328-bf35-4376-a40f-3a09d9d9dd18-istio-podinfo\") pod \"d396a328-bf35-4376-a40f-3a09d9d9dd18\" (UID: \"d396a328-bf35-4376-a40f-3a09d9d9dd18\") " Jul 14 18:10:52.384245 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:52.384152 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d396a328-bf35-4376-a40f-3a09d9d9dd18-istio-data\") pod \"d396a328-bf35-4376-a40f-3a09d9d9dd18\" (UID: \"d396a328-bf35-4376-a40f-3a09d9d9dd18\") " Jul 14 18:10:52.384245 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:52.384176 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d396a328-bf35-4376-a40f-3a09d9d9dd18-credential-socket\") pod \"d396a328-bf35-4376-a40f-3a09d9d9dd18\" (UID: \"d396a328-bf35-4376-a40f-3a09d9d9dd18\") " Jul 14 18:10:52.384245 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:52.384233 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d396a328-bf35-4376-a40f-3a09d9d9dd18-istiod-ca-cert\") pod \"d396a328-bf35-4376-a40f-3a09d9d9dd18\" (UID: \"d396a328-bf35-4376-a40f-3a09d9d9dd18\") " Jul 14 18:10:52.384440 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:52.384275 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d396a328-bf35-4376-a40f-3a09d9d9dd18-istio-envoy\") pod \"d396a328-bf35-4376-a40f-3a09d9d9dd18\" (UID: \"d396a328-bf35-4376-a40f-3a09d9d9dd18\") " Jul 14 18:10:52.384440 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:52.384303 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d396a328-bf35-4376-a40f-3a09d9d9dd18-workload-certs\") pod \"d396a328-bf35-4376-a40f-3a09d9d9dd18\" (UID: \"d396a328-bf35-4376-a40f-3a09d9d9dd18\") " Jul 14 18:10:52.384440 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:52.384332 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d396a328-bf35-4376-a40f-3a09d9d9dd18-workload-socket\") pod \"d396a328-bf35-4376-a40f-3a09d9d9dd18\" (UID: \"d396a328-bf35-4376-a40f-3a09d9d9dd18\") " Jul 14 18:10:52.384440 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:52.384376 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-plxcv\" (UniqueName: \"kubernetes.io/projected/d396a328-bf35-4376-a40f-3a09d9d9dd18-kube-api-access-plxcv\") pod \"d396a328-bf35-4376-a40f-3a09d9d9dd18\" (UID: \"d396a328-bf35-4376-a40f-3a09d9d9dd18\") " Jul 14 18:10:52.384635 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:52.384473 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d396a328-bf35-4376-a40f-3a09d9d9dd18-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d396a328-bf35-4376-a40f-3a09d9d9dd18" (UID: "d396a328-bf35-4376-a40f-3a09d9d9dd18"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 18:10:52.384635 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:52.384604 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d396a328-bf35-4376-a40f-3a09d9d9dd18-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d396a328-bf35-4376-a40f-3a09d9d9dd18" (UID: "d396a328-bf35-4376-a40f-3a09d9d9dd18"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 18:10:52.384886 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:52.384758 2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d396a328-bf35-4376-a40f-3a09d9d9dd18-credential-socket\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 18:10:52.384886 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:52.384782 2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d396a328-bf35-4376-a40f-3a09d9d9dd18-workload-certs\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 18:10:52.384886 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:52.384764 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d396a328-bf35-4376-a40f-3a09d9d9dd18-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d396a328-bf35-4376-a40f-3a09d9d9dd18" (UID: "d396a328-bf35-4376-a40f-3a09d9d9dd18"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 18:10:52.385070 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:52.384912 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d396a328-bf35-4376-a40f-3a09d9d9dd18-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d396a328-bf35-4376-a40f-3a09d9d9dd18" (UID: "d396a328-bf35-4376-a40f-3a09d9d9dd18"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 18:10:52.385266 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:52.385243 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d396a328-bf35-4376-a40f-3a09d9d9dd18-istio-data" (OuterVolumeSpecName: "istio-data") pod "d396a328-bf35-4376-a40f-3a09d9d9dd18" (UID: "d396a328-bf35-4376-a40f-3a09d9d9dd18"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 18:10:52.386551 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:52.386528 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d396a328-bf35-4376-a40f-3a09d9d9dd18-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d396a328-bf35-4376-a40f-3a09d9d9dd18" (UID: "d396a328-bf35-4376-a40f-3a09d9d9dd18"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 14 18:10:52.386648 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:52.386631 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d396a328-bf35-4376-a40f-3a09d9d9dd18-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d396a328-bf35-4376-a40f-3a09d9d9dd18" (UID: "d396a328-bf35-4376-a40f-3a09d9d9dd18"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 18:10:52.386974 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:52.386956 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d396a328-bf35-4376-a40f-3a09d9d9dd18-kube-api-access-plxcv" (OuterVolumeSpecName: "kube-api-access-plxcv") pod "d396a328-bf35-4376-a40f-3a09d9d9dd18" (UID: "d396a328-bf35-4376-a40f-3a09d9d9dd18"). InnerVolumeSpecName "kube-api-access-plxcv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 18:10:52.387027 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:52.386970 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d396a328-bf35-4376-a40f-3a09d9d9dd18-istio-token" (OuterVolumeSpecName: "istio-token") pod "d396a328-bf35-4376-a40f-3a09d9d9dd18" (UID: "d396a328-bf35-4376-a40f-3a09d9d9dd18"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 18:10:52.399909 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:52.399886 2562 generic.go:356] "Generic (PLEG): container finished" podID="d396a328-bf35-4376-a40f-3a09d9d9dd18" containerID="8f6fb6ae464da352727c69f3c09159e692fee5123f30f6b708e5f02a9f29c144" exitCode=0 Jul 14 18:10:52.400014 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:52.399989 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-9c28f2-openshift-default-7b6cd775f5-h95pj" Jul 14 18:10:52.400075 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:52.400024 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-9c28f2-openshift-default-7b6cd775f5-h95pj" event={"ID":"d396a328-bf35-4376-a40f-3a09d9d9dd18","Type":"ContainerDied","Data":"8f6fb6ae464da352727c69f3c09159e692fee5123f30f6b708e5f02a9f29c144"} Jul 14 18:10:52.400133 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:52.400088 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-9c28f2-openshift-default-7b6cd775f5-h95pj" event={"ID":"d396a328-bf35-4376-a40f-3a09d9d9dd18","Type":"ContainerDied","Data":"a20a715628d778ce4889d22c1de680bd886ea3aa6c954d6507fb20b99b89c597"} Jul 14 18:10:52.400133 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:52.400123 2562 scope.go:117] "RemoveContainer" containerID="8f6fb6ae464da352727c69f3c09159e692fee5123f30f6b708e5f02a9f29c144" Jul 14 18:10:52.408792 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:52.408778 2562 scope.go:117] "RemoveContainer" containerID="8048a5b8a5231f11c448ea7c8d2edc9ff9cd8341cca8e863f96765b84eab96d0" Jul 14 18:10:52.416056 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:52.416038 2562 scope.go:117] "RemoveContainer" containerID="8f6fb6ae464da352727c69f3c09159e692fee5123f30f6b708e5f02a9f29c144" Jul 14 18:10:52.416299 ip-10-0-129-209 kubenswrapper[2562]: E0714 18:10:52.416282 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8f6fb6ae464da352727c69f3c09159e692fee5123f30f6b708e5f02a9f29c144\": container with ID starting with 8f6fb6ae464da352727c69f3c09159e692fee5123f30f6b708e5f02a9f29c144 not found: ID does not exist" containerID="8f6fb6ae464da352727c69f3c09159e692fee5123f30f6b708e5f02a9f29c144" Jul 14 18:10:52.416348 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:52.416306 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f6fb6ae464da352727c69f3c09159e692fee5123f30f6b708e5f02a9f29c144"} err="failed to get container status \"8f6fb6ae464da352727c69f3c09159e692fee5123f30f6b708e5f02a9f29c144\": rpc error: code = NotFound desc = could not find container \"8f6fb6ae464da352727c69f3c09159e692fee5123f30f6b708e5f02a9f29c144\": container with ID starting with 8f6fb6ae464da352727c69f3c09159e692fee5123f30f6b708e5f02a9f29c144 not found: ID does not exist" Jul 14 18:10:52.416348 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:52.416322 2562 scope.go:117] "RemoveContainer" containerID="8048a5b8a5231f11c448ea7c8d2edc9ff9cd8341cca8e863f96765b84eab96d0" Jul 14 18:10:52.416557 ip-10-0-129-209 kubenswrapper[2562]: E0714 18:10:52.416541 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8048a5b8a5231f11c448ea7c8d2edc9ff9cd8341cca8e863f96765b84eab96d0\": container with ID starting with 8048a5b8a5231f11c448ea7c8d2edc9ff9cd8341cca8e863f96765b84eab96d0 not found: ID does not exist" containerID="8048a5b8a5231f11c448ea7c8d2edc9ff9cd8341cca8e863f96765b84eab96d0" Jul 14 18:10:52.416593 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:52.416563 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8048a5b8a5231f11c448ea7c8d2edc9ff9cd8341cca8e863f96765b84eab96d0"} err="failed to get container status \"8048a5b8a5231f11c448ea7c8d2edc9ff9cd8341cca8e863f96765b84eab96d0\": rpc error: code = NotFound desc = could not find container \"8048a5b8a5231f11c448ea7c8d2edc9ff9cd8341cca8e863f96765b84eab96d0\": container with ID starting with 8048a5b8a5231f11c448ea7c8d2edc9ff9cd8341cca8e863f96765b84eab96d0 not found: ID does not exist" Jul 14 18:10:52.445320 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:52.445295 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-9c28f2-openshift-default-7b6cd775f5-h95pj"] Jul 14 18:10:52.455884 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:52.455864 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-9c28f2-openshift-default-7b6cd775f5-h95pj"] Jul 14 18:10:52.485632 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:52.485608 2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d396a328-bf35-4376-a40f-3a09d9d9dd18-istio-token\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 18:10:52.485632 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:52.485633 2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d396a328-bf35-4376-a40f-3a09d9d9dd18-istio-podinfo\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 18:10:52.485780 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:52.485646 2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d396a328-bf35-4376-a40f-3a09d9d9dd18-istio-data\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 18:10:52.485780 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:52.485661 2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d396a328-bf35-4376-a40f-3a09d9d9dd18-istiod-ca-cert\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 18:10:52.485780 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:52.485674 2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d396a328-bf35-4376-a40f-3a09d9d9dd18-istio-envoy\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 18:10:52.485780 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:52.485686 2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d396a328-bf35-4376-a40f-3a09d9d9dd18-workload-socket\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 18:10:52.485780 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:52.485698 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-plxcv\" (UniqueName: \"kubernetes.io/projected/d396a328-bf35-4376-a40f-3a09d9d9dd18-kube-api-access-plxcv\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 18:10:53.519357 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:53.519324 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d396a328-bf35-4376-a40f-3a09d9d9dd18" path="/var/lib/kubelet/pods/d396a328-bf35-4376-a40f-3a09d9d9dd18/volumes" Jul 14 18:10:56.739369 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:56.739338 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-e9d840-openshift-default-5677d74ffc-twd6f"] Jul 14 18:10:56.739822 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:56.739810 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d396a328-bf35-4376-a40f-3a09d9d9dd18" containerName="istio-proxy" Jul 14 18:10:56.739890 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:56.739828 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="d396a328-bf35-4376-a40f-3a09d9d9dd18" containerName="istio-proxy" Jul 14 18:10:56.739890 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:56.739845 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d396a328-bf35-4376-a40f-3a09d9d9dd18" containerName="istio-proxy" Jul 14 18:10:56.739890 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:56.739854 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="d396a328-bf35-4376-a40f-3a09d9d9dd18" containerName="istio-proxy" Jul 14 18:10:56.740056 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:56.739969 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="d396a328-bf35-4376-a40f-3a09d9d9dd18" containerName="istio-proxy" Jul 14 18:10:56.740056 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:56.739980 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="d396a328-bf35-4376-a40f-3a09d9d9dd18" containerName="istio-proxy" Jul 14 18:10:56.742811 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:56.742790 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-e9d840-openshift-default-5677d74ffc-twd6f" Jul 14 18:10:56.745505 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:56.745482 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-e9d840-openshift-default-dockercfg-j8w6g\"" Jul 14 18:10:56.770945 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:56.770919 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-e9d840-openshift-default-5677d74ffc-twd6f"] Jul 14 18:10:56.924720 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:56.924680 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6ad039bb-dff9-4919-83bb-0a5813fa01b4-istio-podinfo\") pod \"e2e-api-b-e9d840-openshift-default-5677d74ffc-twd6f\" (UID: \"6ad039bb-dff9-4919-83bb-0a5813fa01b4\") " pod="openshift-ingress/e2e-api-b-e9d840-openshift-default-5677d74ffc-twd6f" Jul 14 18:10:56.924720 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:56.924720 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ad039bb-dff9-4919-83bb-0a5813fa01b4-workload-socket\") pod \"e2e-api-b-e9d840-openshift-default-5677d74ffc-twd6f\" (UID: \"6ad039bb-dff9-4919-83bb-0a5813fa01b4\") " pod="openshift-ingress/e2e-api-b-e9d840-openshift-default-5677d74ffc-twd6f" Jul 14 18:10:56.924928 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:56.924749 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rgshc\" (UniqueName: \"kubernetes.io/projected/6ad039bb-dff9-4919-83bb-0a5813fa01b4-kube-api-access-rgshc\") pod \"e2e-api-b-e9d840-openshift-default-5677d74ffc-twd6f\" (UID: \"6ad039bb-dff9-4919-83bb-0a5813fa01b4\") " pod="openshift-ingress/e2e-api-b-e9d840-openshift-default-5677d74ffc-twd6f" Jul 14 18:10:56.924928 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:56.924838 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6ad039bb-dff9-4919-83bb-0a5813fa01b4-workload-certs\") pod \"e2e-api-b-e9d840-openshift-default-5677d74ffc-twd6f\" (UID: \"6ad039bb-dff9-4919-83bb-0a5813fa01b4\") " pod="openshift-ingress/e2e-api-b-e9d840-openshift-default-5677d74ffc-twd6f" Jul 14 18:10:56.924928 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:56.924872 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6ad039bb-dff9-4919-83bb-0a5813fa01b4-istio-token\") pod \"e2e-api-b-e9d840-openshift-default-5677d74ffc-twd6f\" (UID: \"6ad039bb-dff9-4919-83bb-0a5813fa01b4\") " pod="openshift-ingress/e2e-api-b-e9d840-openshift-default-5677d74ffc-twd6f" Jul 14 18:10:56.924928 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:56.924919 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ad039bb-dff9-4919-83bb-0a5813fa01b4-credential-socket\") pod \"e2e-api-b-e9d840-openshift-default-5677d74ffc-twd6f\" (UID: \"6ad039bb-dff9-4919-83bb-0a5813fa01b4\") " pod="openshift-ingress/e2e-api-b-e9d840-openshift-default-5677d74ffc-twd6f" Jul 14 18:10:56.925055 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:56.924949 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6ad039bb-dff9-4919-83bb-0a5813fa01b4-istio-envoy\") pod \"e2e-api-b-e9d840-openshift-default-5677d74ffc-twd6f\" (UID: \"6ad039bb-dff9-4919-83bb-0a5813fa01b4\") " pod="openshift-ingress/e2e-api-b-e9d840-openshift-default-5677d74ffc-twd6f" Jul 14 18:10:56.925055 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:56.924984 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6ad039bb-dff9-4919-83bb-0a5813fa01b4-istiod-ca-cert\") pod \"e2e-api-b-e9d840-openshift-default-5677d74ffc-twd6f\" (UID: \"6ad039bb-dff9-4919-83bb-0a5813fa01b4\") " pod="openshift-ingress/e2e-api-b-e9d840-openshift-default-5677d74ffc-twd6f" Jul 14 18:10:56.925055 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:56.925011 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6ad039bb-dff9-4919-83bb-0a5813fa01b4-istio-data\") pod \"e2e-api-b-e9d840-openshift-default-5677d74ffc-twd6f\" (UID: \"6ad039bb-dff9-4919-83bb-0a5813fa01b4\") " pod="openshift-ingress/e2e-api-b-e9d840-openshift-default-5677d74ffc-twd6f" Jul 14 18:10:57.025817 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:57.025693 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6ad039bb-dff9-4919-83bb-0a5813fa01b4-istio-data\") pod \"e2e-api-b-e9d840-openshift-default-5677d74ffc-twd6f\" (UID: \"6ad039bb-dff9-4919-83bb-0a5813fa01b4\") " pod="openshift-ingress/e2e-api-b-e9d840-openshift-default-5677d74ffc-twd6f" Jul 14 18:10:57.025817 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:57.025770 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6ad039bb-dff9-4919-83bb-0a5813fa01b4-istio-podinfo\") pod \"e2e-api-b-e9d840-openshift-default-5677d74ffc-twd6f\" (UID: \"6ad039bb-dff9-4919-83bb-0a5813fa01b4\") " pod="openshift-ingress/e2e-api-b-e9d840-openshift-default-5677d74ffc-twd6f" Jul 14 18:10:57.025817 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:57.025799 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ad039bb-dff9-4919-83bb-0a5813fa01b4-workload-socket\") pod \"e2e-api-b-e9d840-openshift-default-5677d74ffc-twd6f\" (UID: \"6ad039bb-dff9-4919-83bb-0a5813fa01b4\") " pod="openshift-ingress/e2e-api-b-e9d840-openshift-default-5677d74ffc-twd6f" Jul 14 18:10:57.026126 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:57.025845 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rgshc\" (UniqueName: \"kubernetes.io/projected/6ad039bb-dff9-4919-83bb-0a5813fa01b4-kube-api-access-rgshc\") pod \"e2e-api-b-e9d840-openshift-default-5677d74ffc-twd6f\" (UID: \"6ad039bb-dff9-4919-83bb-0a5813fa01b4\") " pod="openshift-ingress/e2e-api-b-e9d840-openshift-default-5677d74ffc-twd6f" Jul 14 18:10:57.026126 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:57.025892 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6ad039bb-dff9-4919-83bb-0a5813fa01b4-workload-certs\") pod \"e2e-api-b-e9d840-openshift-default-5677d74ffc-twd6f\" (UID: \"6ad039bb-dff9-4919-83bb-0a5813fa01b4\") " pod="openshift-ingress/e2e-api-b-e9d840-openshift-default-5677d74ffc-twd6f" Jul 14 18:10:57.026126 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:57.025918 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6ad039bb-dff9-4919-83bb-0a5813fa01b4-istio-token\") pod \"e2e-api-b-e9d840-openshift-default-5677d74ffc-twd6f\" (UID: \"6ad039bb-dff9-4919-83bb-0a5813fa01b4\") " pod="openshift-ingress/e2e-api-b-e9d840-openshift-default-5677d74ffc-twd6f" Jul 14 18:10:57.026126 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:57.025957 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ad039bb-dff9-4919-83bb-0a5813fa01b4-credential-socket\") pod \"e2e-api-b-e9d840-openshift-default-5677d74ffc-twd6f\" (UID: \"6ad039bb-dff9-4919-83bb-0a5813fa01b4\") " pod="openshift-ingress/e2e-api-b-e9d840-openshift-default-5677d74ffc-twd6f" Jul 14 18:10:57.026126 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:57.025987 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6ad039bb-dff9-4919-83bb-0a5813fa01b4-istio-envoy\") pod \"e2e-api-b-e9d840-openshift-default-5677d74ffc-twd6f\" (UID: \"6ad039bb-dff9-4919-83bb-0a5813fa01b4\") " pod="openshift-ingress/e2e-api-b-e9d840-openshift-default-5677d74ffc-twd6f" Jul 14 18:10:57.026126 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:57.026039 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6ad039bb-dff9-4919-83bb-0a5813fa01b4-istiod-ca-cert\") pod \"e2e-api-b-e9d840-openshift-default-5677d74ffc-twd6f\" (UID: \"6ad039bb-dff9-4919-83bb-0a5813fa01b4\") " pod="openshift-ingress/e2e-api-b-e9d840-openshift-default-5677d74ffc-twd6f" Jul 14 18:10:57.026470 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:57.026161 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6ad039bb-dff9-4919-83bb-0a5813fa01b4-istio-data\") pod \"e2e-api-b-e9d840-openshift-default-5677d74ffc-twd6f\" (UID: \"6ad039bb-dff9-4919-83bb-0a5813fa01b4\") " pod="openshift-ingress/e2e-api-b-e9d840-openshift-default-5677d74ffc-twd6f" Jul 14 18:10:57.026470 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:57.026232 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ad039bb-dff9-4919-83bb-0a5813fa01b4-workload-socket\") pod \"e2e-api-b-e9d840-openshift-default-5677d74ffc-twd6f\" (UID: \"6ad039bb-dff9-4919-83bb-0a5813fa01b4\") " pod="openshift-ingress/e2e-api-b-e9d840-openshift-default-5677d74ffc-twd6f" Jul 14 18:10:57.026470 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:57.026251 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6ad039bb-dff9-4919-83bb-0a5813fa01b4-workload-certs\") pod \"e2e-api-b-e9d840-openshift-default-5677d74ffc-twd6f\" (UID: \"6ad039bb-dff9-4919-83bb-0a5813fa01b4\") " pod="openshift-ingress/e2e-api-b-e9d840-openshift-default-5677d74ffc-twd6f" Jul 14 18:10:57.026470 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:57.026460 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ad039bb-dff9-4919-83bb-0a5813fa01b4-credential-socket\") pod \"e2e-api-b-e9d840-openshift-default-5677d74ffc-twd6f\" (UID: \"6ad039bb-dff9-4919-83bb-0a5813fa01b4\") " pod="openshift-ingress/e2e-api-b-e9d840-openshift-default-5677d74ffc-twd6f" Jul 14 18:10:57.026749 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:57.026729 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6ad039bb-dff9-4919-83bb-0a5813fa01b4-istiod-ca-cert\") pod \"e2e-api-b-e9d840-openshift-default-5677d74ffc-twd6f\" (UID: \"6ad039bb-dff9-4919-83bb-0a5813fa01b4\") " pod="openshift-ingress/e2e-api-b-e9d840-openshift-default-5677d74ffc-twd6f" Jul 14 18:10:57.028454 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:57.028423 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6ad039bb-dff9-4919-83bb-0a5813fa01b4-istio-envoy\") pod \"e2e-api-b-e9d840-openshift-default-5677d74ffc-twd6f\" (UID: \"6ad039bb-dff9-4919-83bb-0a5813fa01b4\") " pod="openshift-ingress/e2e-api-b-e9d840-openshift-default-5677d74ffc-twd6f" Jul 14 18:10:57.028577 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:57.028477 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6ad039bb-dff9-4919-83bb-0a5813fa01b4-istio-podinfo\") pod \"e2e-api-b-e9d840-openshift-default-5677d74ffc-twd6f\" (UID: \"6ad039bb-dff9-4919-83bb-0a5813fa01b4\") " pod="openshift-ingress/e2e-api-b-e9d840-openshift-default-5677d74ffc-twd6f" Jul 14 18:10:57.039021 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:57.038972 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6ad039bb-dff9-4919-83bb-0a5813fa01b4-istio-token\") pod \"e2e-api-b-e9d840-openshift-default-5677d74ffc-twd6f\" (UID: \"6ad039bb-dff9-4919-83bb-0a5813fa01b4\") " pod="openshift-ingress/e2e-api-b-e9d840-openshift-default-5677d74ffc-twd6f" Jul 14 18:10:57.041674 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:57.041642 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rgshc\" (UniqueName: \"kubernetes.io/projected/6ad039bb-dff9-4919-83bb-0a5813fa01b4-kube-api-access-rgshc\") pod \"e2e-api-b-e9d840-openshift-default-5677d74ffc-twd6f\" (UID: \"6ad039bb-dff9-4919-83bb-0a5813fa01b4\") " pod="openshift-ingress/e2e-api-b-e9d840-openshift-default-5677d74ffc-twd6f" Jul 14 18:10:57.053481 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:57.053442 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-e9d840-openshift-default-5677d74ffc-twd6f" Jul 14 18:10:57.246161 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:57.246034 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-e9d840-openshift-default-5677d74ffc-twd6f"] Jul 14 18:10:57.248763 ip-10-0-129-209 kubenswrapper[2562]: W0714 18:10:57.248736 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ad039bb_dff9_4919_83bb_0a5813fa01b4.slice/crio-ebeea895d09afd44d972410d221b9acc8a79b1dc0094f75908f6042587b1a7c5 WatchSource:0}: Error finding container ebeea895d09afd44d972410d221b9acc8a79b1dc0094f75908f6042587b1a7c5: Status 404 returned error can't find the container with id ebeea895d09afd44d972410d221b9acc8a79b1dc0094f75908f6042587b1a7c5 Jul 14 18:10:57.250926 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:57.250896 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 14 18:10:57.251002 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:57.250959 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 14 18:10:57.251002 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:57.250986 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 14 18:10:57.423769 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:57.423732 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-e9d840-openshift-default-5677d74ffc-twd6f" event={"ID":"6ad039bb-dff9-4919-83bb-0a5813fa01b4","Type":"ContainerStarted","Data":"911a85478890c5b2a8de0e1e9daf2c3a9c43581ad35c3238c806ba6bc6ca22e4"} Jul 14 18:10:57.424062 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:57.423791 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-e9d840-openshift-default-5677d74ffc-twd6f" event={"ID":"6ad039bb-dff9-4919-83bb-0a5813fa01b4","Type":"ContainerStarted","Data":"ebeea895d09afd44d972410d221b9acc8a79b1dc0094f75908f6042587b1a7c5"} Jul 14 18:10:57.456237 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:57.456143 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-e9d840-openshift-default-5677d74ffc-twd6f" podStartSLOduration=1.456121355 podStartE2EDuration="1.456121355s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 18:10:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 18:10:57.454068943 +0000 UTC m=+2306.530526482" watchObservedRunningTime="2026-07-14 18:10:57.456121355 +0000 UTC m=+2306.532578894" Jul 14 18:10:58.054567 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:58.054536 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-e9d840-openshift-default-5677d74ffc-twd6f" Jul 14 18:10:58.055991 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:58.055948 2562 patch_prober.go:28] interesting pod/e2e-api-b-e9d840-openshift-default-5677d74ffc-twd6f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body= Jul 14 18:10:58.056123 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:58.056015 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-e9d840-openshift-default-5677d74ffc-twd6f" podUID="6ad039bb-dff9-4919-83bb-0a5813fa01b4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" Jul 14 18:10:59.063665 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:59.063627 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-e9d840-openshift-default-5677d74ffc-twd6f" Jul 14 18:10:59.433468 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:59.433435 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-e9d840-openshift-default-5677d74ffc-twd6f" Jul 14 18:10:59.434301 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:10:59.434285 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-e9d840-openshift-default-5677d74ffc-twd6f" Jul 14 18:11:04.462536 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:11:04.462487 2562 generic.go:356] "Generic (PLEG): container finished" podID="36c0e46a-2f62-479e-a42f-c910c0d5edaf" containerID="824a0fd38b12b1737b80fff3e595b8299cdc7409920b5f901bddc1295e6a56f6" exitCode=0 Jul 14 18:11:04.463079 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:11:04.462629 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-5a43f9-openshift-default-549d8686f9-ztvt5" event={"ID":"36c0e46a-2f62-479e-a42f-c910c0d5edaf","Type":"ContainerDied","Data":"824a0fd38b12b1737b80fff3e595b8299cdc7409920b5f901bddc1295e6a56f6"} Jul 14 18:11:04.463146 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:11:04.463084 2562 scope.go:117] "RemoveContainer" containerID="824a0fd38b12b1737b80fff3e595b8299cdc7409920b5f901bddc1295e6a56f6" Jul 14 18:11:04.464597 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:11:04.464558 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 14 18:11:04.464680 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:11:04.464654 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 14 18:11:04.464740 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:11:04.464699 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 14 18:11:05.390157 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:11:05.390105 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-5a43f9-openshift-default-549d8686f9-ztvt5" Jul 14 18:11:05.472358 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:11:05.472312 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-5a43f9-openshift-default-549d8686f9-ztvt5" event={"ID":"36c0e46a-2f62-479e-a42f-c910c0d5edaf","Type":"ContainerStarted","Data":"874f3f1db2e9c7e14d85769fb1913d35c63fa2ae92ff04b4e182ce951f9286d1"} Jul 14 18:11:06.273590 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:11:06.273554 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-5a43f9-openshift-default-549d8686f9-ztvt5" Jul 14 18:11:06.273951 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:11:06.273925 2562 patch_prober.go:28] interesting pod/e2e-api-a-5a43f9-openshift-default-549d8686f9-ztvt5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body= Jul 14 18:11:06.274148 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:11:06.274119 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-5a43f9-openshift-default-549d8686f9-ztvt5" podUID="36c0e46a-2f62-479e-a42f-c910c0d5edaf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" Jul 14 18:11:07.272414 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:11:07.272365 2562 patch_prober.go:28] interesting pod/e2e-api-a-5a43f9-openshift-default-549d8686f9-ztvt5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body= Jul 14 18:11:07.272944 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:11:07.272437 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-5a43f9-openshift-default-549d8686f9-ztvt5" podUID="36c0e46a-2f62-479e-a42f-c910c0d5edaf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" Jul 14 18:11:08.273254 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:11:08.272788 2562 patch_prober.go:28] interesting pod/e2e-api-a-5a43f9-openshift-default-549d8686f9-ztvt5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body= Jul 14 18:11:08.273254 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:11:08.272851 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-5a43f9-openshift-default-549d8686f9-ztvt5" podUID="36c0e46a-2f62-479e-a42f-c910c0d5edaf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" Jul 14 18:11:09.272291 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:11:09.272250 2562 patch_prober.go:28] interesting pod/e2e-api-a-5a43f9-openshift-default-549d8686f9-ztvt5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body= Jul 14 18:11:09.272502 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:11:09.272318 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-5a43f9-openshift-default-549d8686f9-ztvt5" podUID="36c0e46a-2f62-479e-a42f-c910c0d5edaf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" Jul 14 18:11:10.276144 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:11:10.276107 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-5a43f9-openshift-default-549d8686f9-ztvt5" Jul 14 18:11:10.276611 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:11:10.276559 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-5a43f9-openshift-default-549d8686f9-ztvt5" Jul 14 18:11:10.277292 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:11:10.277268 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-5a43f9-openshift-default-549d8686f9-ztvt5" Jul 14 18:11:11.509779 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:11:11.509697 2562 generic.go:356] "Generic (PLEG): container finished" podID="6ad039bb-dff9-4919-83bb-0a5813fa01b4" containerID="911a85478890c5b2a8de0e1e9daf2c3a9c43581ad35c3238c806ba6bc6ca22e4" exitCode=0 Jul 14 18:11:11.510212 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:11:11.509778 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-e9d840-openshift-default-5677d74ffc-twd6f" event={"ID":"6ad039bb-dff9-4919-83bb-0a5813fa01b4","Type":"ContainerDied","Data":"911a85478890c5b2a8de0e1e9daf2c3a9c43581ad35c3238c806ba6bc6ca22e4"} Jul 14 18:11:11.510300 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:11:11.510220 2562 scope.go:117] "RemoveContainer" containerID="911a85478890c5b2a8de0e1e9daf2c3a9c43581ad35c3238c806ba6bc6ca22e4" Jul 14 18:11:11.511111 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:11:11.511088 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 14 18:11:11.511160 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:11:11.511150 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 14 18:11:11.511221 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:11:11.511181 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 14 18:11:12.516217 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:11:12.516163 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-e9d840-openshift-default-5677d74ffc-twd6f" event={"ID":"6ad039bb-dff9-4919-83bb-0a5813fa01b4","Type":"ContainerStarted","Data":"1b595d93e84359a8291a032178706936991054319cd52c384d2c8cfdb84fd095"} Jul 14 18:11:13.054718 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:11:13.054680 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-e9d840-openshift-default-5677d74ffc-twd6f" Jul 14 18:11:13.056224 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:11:13.056162 2562 patch_prober.go:28] interesting pod/e2e-api-b-e9d840-openshift-default-5677d74ffc-twd6f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body= Jul 14 18:11:13.056931 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:11:13.056440 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-e9d840-openshift-default-5677d74ffc-twd6f" podUID="6ad039bb-dff9-4919-83bb-0a5813fa01b4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" Jul 14 18:11:14.054547 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:11:14.054512 2562 patch_prober.go:28] interesting pod/e2e-api-b-e9d840-openshift-default-5677d74ffc-twd6f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body= Jul 14 18:11:14.055097 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:11:14.055066 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-e9d840-openshift-default-5677d74ffc-twd6f" podUID="6ad039bb-dff9-4919-83bb-0a5813fa01b4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" Jul 14 18:11:14.434232 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:11:14.434179 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-e9d840-openshift-default-5677d74ffc-twd6f" Jul 14 18:11:15.054351 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:11:15.054316 2562 patch_prober.go:28] interesting pod/e2e-api-b-e9d840-openshift-default-5677d74ffc-twd6f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body= Jul 14 18:11:15.054651 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:11:15.054621 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-e9d840-openshift-default-5677d74ffc-twd6f" podUID="6ad039bb-dff9-4919-83bb-0a5813fa01b4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" Jul 14 18:11:16.055148 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:11:16.055113 2562 patch_prober.go:28] interesting pod/e2e-api-b-e9d840-openshift-default-5677d74ffc-twd6f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body= Jul 14 18:11:16.055752 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:11:16.055722 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-e9d840-openshift-default-5677d74ffc-twd6f" podUID="6ad039bb-dff9-4919-83bb-0a5813fa01b4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" Jul 14 18:11:17.058297 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:11:17.058266 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-e9d840-openshift-default-5677d74ffc-twd6f" Jul 14 18:11:17.059231 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:11:17.059213 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-e9d840-openshift-default-5677d74ffc-twd6f" Jul 14 18:11:37.838058 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:11:37.838026 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-5a43f9-openshift-default-549d8686f9-ztvt5"] Jul 14 18:11:37.838570 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:11:37.838245 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-5a43f9-openshift-default-549d8686f9-ztvt5" podUID="36c0e46a-2f62-479e-a42f-c910c0d5edaf" containerName="istio-proxy" containerID="cri-o://874f3f1db2e9c7e14d85769fb1913d35c63fa2ae92ff04b4e182ce951f9286d1" gracePeriod=30 Jul 14 18:11:40.278780 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:11:40.278737 2562 patch_prober.go:28] interesting pod/e2e-api-a-5a43f9-openshift-default-549d8686f9-ztvt5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 14 18:11:40.279145 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:11:40.278799 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-5a43f9-openshift-default-549d8686f9-ztvt5" podUID="36c0e46a-2f62-479e-a42f-c910c0d5edaf" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 14 18:11:43.089011 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:11:43.088988 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-5a43f9-openshift-default-549d8686f9-ztvt5" Jul 14 18:11:43.250022 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:11:43.249999 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-phzqc\" (UniqueName: \"kubernetes.io/projected/36c0e46a-2f62-479e-a42f-c910c0d5edaf-kube-api-access-phzqc\") pod \"36c0e46a-2f62-479e-a42f-c910c0d5edaf\" (UID: \"36c0e46a-2f62-479e-a42f-c910c0d5edaf\") " Jul 14 18:11:43.250232 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:11:43.250044 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/36c0e46a-2f62-479e-a42f-c910c0d5edaf-istio-token\") pod \"36c0e46a-2f62-479e-a42f-c910c0d5edaf\" (UID: \"36c0e46a-2f62-479e-a42f-c910c0d5edaf\") " Jul 14 18:11:43.250232 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:11:43.250081 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/36c0e46a-2f62-479e-a42f-c910c0d5edaf-istio-podinfo\") pod \"36c0e46a-2f62-479e-a42f-c910c0d5edaf\" (UID: \"36c0e46a-2f62-479e-a42f-c910c0d5edaf\") " Jul 14 18:11:43.250232 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:11:43.250111 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/36c0e46a-2f62-479e-a42f-c910c0d5edaf-workload-certs\") pod \"36c0e46a-2f62-479e-a42f-c910c0d5edaf\" (UID: \"36c0e46a-2f62-479e-a42f-c910c0d5edaf\") " Jul 14 18:11:43.250232 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:11:43.250127 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/36c0e46a-2f62-479e-a42f-c910c0d5edaf-credential-socket\") pod \"36c0e46a-2f62-479e-a42f-c910c0d5edaf\" (UID: \"36c0e46a-2f62-479e-a42f-c910c0d5edaf\") " Jul 14 18:11:43.250232 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:11:43.250154 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/36c0e46a-2f62-479e-a42f-c910c0d5edaf-workload-socket\") pod \"36c0e46a-2f62-479e-a42f-c910c0d5edaf\" (UID: \"36c0e46a-2f62-479e-a42f-c910c0d5edaf\") " Jul 14 18:11:43.250232 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:11:43.250175 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/36c0e46a-2f62-479e-a42f-c910c0d5edaf-istiod-ca-cert\") pod \"36c0e46a-2f62-479e-a42f-c910c0d5edaf\" (UID: \"36c0e46a-2f62-479e-a42f-c910c0d5edaf\") " Jul 14 18:11:43.250232 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:11:43.250227 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/36c0e46a-2f62-479e-a42f-c910c0d5edaf-istio-envoy\") pod \"36c0e46a-2f62-479e-a42f-c910c0d5edaf\" (UID: \"36c0e46a-2f62-479e-a42f-c910c0d5edaf\") " Jul 14 18:11:43.250608 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:11:43.250251 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/36c0e46a-2f62-479e-a42f-c910c0d5edaf-istio-data\") pod \"36c0e46a-2f62-479e-a42f-c910c0d5edaf\" (UID: \"36c0e46a-2f62-479e-a42f-c910c0d5edaf\") " Jul 14 18:11:43.250608 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:11:43.250539 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36c0e46a-2f62-479e-a42f-c910c0d5edaf-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "36c0e46a-2f62-479e-a42f-c910c0d5edaf" (UID: "36c0e46a-2f62-479e-a42f-c910c0d5edaf"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 18:11:43.250608 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:11:43.250573 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36c0e46a-2f62-479e-a42f-c910c0d5edaf-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "36c0e46a-2f62-479e-a42f-c910c0d5edaf" (UID: "36c0e46a-2f62-479e-a42f-c910c0d5edaf"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 18:11:43.250765 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:11:43.250611 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36c0e46a-2f62-479e-a42f-c910c0d5edaf-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "36c0e46a-2f62-479e-a42f-c910c0d5edaf" (UID: "36c0e46a-2f62-479e-a42f-c910c0d5edaf"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 18:11:43.250765 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:11:43.250690 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/36c0e46a-2f62-479e-a42f-c910c0d5edaf-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "36c0e46a-2f62-479e-a42f-c910c0d5edaf" (UID: "36c0e46a-2f62-479e-a42f-c910c0d5edaf"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 18:11:43.251654 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:11:43.251627 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36c0e46a-2f62-479e-a42f-c910c0d5edaf-istio-data" (OuterVolumeSpecName: "istio-data") pod "36c0e46a-2f62-479e-a42f-c910c0d5edaf" (UID: "36c0e46a-2f62-479e-a42f-c910c0d5edaf"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 18:11:43.252483 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:11:43.252456 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/36c0e46a-2f62-479e-a42f-c910c0d5edaf-kube-api-access-phzqc" (OuterVolumeSpecName: "kube-api-access-phzqc") pod "36c0e46a-2f62-479e-a42f-c910c0d5edaf" (UID: "36c0e46a-2f62-479e-a42f-c910c0d5edaf"). InnerVolumeSpecName "kube-api-access-phzqc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 18:11:43.252579 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:11:43.252502 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/36c0e46a-2f62-479e-a42f-c910c0d5edaf-istio-token" (OuterVolumeSpecName: "istio-token") pod "36c0e46a-2f62-479e-a42f-c910c0d5edaf" (UID: "36c0e46a-2f62-479e-a42f-c910c0d5edaf"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 18:11:43.252662 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:11:43.252636 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/36c0e46a-2f62-479e-a42f-c910c0d5edaf-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "36c0e46a-2f62-479e-a42f-c910c0d5edaf" (UID: "36c0e46a-2f62-479e-a42f-c910c0d5edaf"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 14 18:11:43.252872 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:11:43.252855 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36c0e46a-2f62-479e-a42f-c910c0d5edaf-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "36c0e46a-2f62-479e-a42f-c910c0d5edaf" (UID: "36c0e46a-2f62-479e-a42f-c910c0d5edaf"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 18:11:43.351391 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:11:43.351366 2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/36c0e46a-2f62-479e-a42f-c910c0d5edaf-workload-certs\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 18:11:43.351391 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:11:43.351389 2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/36c0e46a-2f62-479e-a42f-c910c0d5edaf-credential-socket\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 18:11:43.351522 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:11:43.351399 2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/36c0e46a-2f62-479e-a42f-c910c0d5edaf-workload-socket\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 18:11:43.351522 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:11:43.351408 2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/36c0e46a-2f62-479e-a42f-c910c0d5edaf-istiod-ca-cert\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 18:11:43.351522 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:11:43.351417 2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/36c0e46a-2f62-479e-a42f-c910c0d5edaf-istio-envoy\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 18:11:43.351522 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:11:43.351424 2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/36c0e46a-2f62-479e-a42f-c910c0d5edaf-istio-data\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 18:11:43.351522 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:11:43.351431 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-phzqc\" (UniqueName: \"kubernetes.io/projected/36c0e46a-2f62-479e-a42f-c910c0d5edaf-kube-api-access-phzqc\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 18:11:43.351522 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:11:43.351439 2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/36c0e46a-2f62-479e-a42f-c910c0d5edaf-istio-token\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 18:11:43.351522 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:11:43.351482 2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/36c0e46a-2f62-479e-a42f-c910c0d5edaf-istio-podinfo\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 18:11:43.647029 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:11:43.646994 2562 generic.go:356] "Generic (PLEG): container finished" podID="36c0e46a-2f62-479e-a42f-c910c0d5edaf" containerID="874f3f1db2e9c7e14d85769fb1913d35c63fa2ae92ff04b4e182ce951f9286d1" exitCode=0 Jul 14 18:11:43.647172 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:11:43.647046 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-5a43f9-openshift-default-549d8686f9-ztvt5" Jul 14 18:11:43.647172 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:11:43.647044 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-5a43f9-openshift-default-549d8686f9-ztvt5" event={"ID":"36c0e46a-2f62-479e-a42f-c910c0d5edaf","Type":"ContainerDied","Data":"874f3f1db2e9c7e14d85769fb1913d35c63fa2ae92ff04b4e182ce951f9286d1"} Jul 14 18:11:43.647270 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:11:43.647166 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-5a43f9-openshift-default-549d8686f9-ztvt5" event={"ID":"36c0e46a-2f62-479e-a42f-c910c0d5edaf","Type":"ContainerDied","Data":"26f01ffe8dd62c997a27dacf6a4538a79b8b77dbe18dce30efffbc6e37572aa5"} Jul 14 18:11:43.647270 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:11:43.647221 2562 scope.go:117] "RemoveContainer" containerID="874f3f1db2e9c7e14d85769fb1913d35c63fa2ae92ff04b4e182ce951f9286d1" Jul 14 18:11:43.655949 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:11:43.655927 2562 scope.go:117] "RemoveContainer" containerID="824a0fd38b12b1737b80fff3e595b8299cdc7409920b5f901bddc1295e6a56f6" Jul 14 18:11:43.663661 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:11:43.663642 2562 scope.go:117] "RemoveContainer" containerID="874f3f1db2e9c7e14d85769fb1913d35c63fa2ae92ff04b4e182ce951f9286d1" Jul 14 18:11:43.663906 ip-10-0-129-209 kubenswrapper[2562]: E0714 18:11:43.663885 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"874f3f1db2e9c7e14d85769fb1913d35c63fa2ae92ff04b4e182ce951f9286d1\": container with ID starting with 874f3f1db2e9c7e14d85769fb1913d35c63fa2ae92ff04b4e182ce951f9286d1 not found: ID does not exist" containerID="874f3f1db2e9c7e14d85769fb1913d35c63fa2ae92ff04b4e182ce951f9286d1" Jul 14 18:11:43.663976 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:11:43.663917 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"874f3f1db2e9c7e14d85769fb1913d35c63fa2ae92ff04b4e182ce951f9286d1"} err="failed to get container status \"874f3f1db2e9c7e14d85769fb1913d35c63fa2ae92ff04b4e182ce951f9286d1\": rpc error: code = NotFound desc = could not find container \"874f3f1db2e9c7e14d85769fb1913d35c63fa2ae92ff04b4e182ce951f9286d1\": container with ID starting with 874f3f1db2e9c7e14d85769fb1913d35c63fa2ae92ff04b4e182ce951f9286d1 not found: ID does not exist" Jul 14 18:11:43.663976 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:11:43.663943 2562 scope.go:117] "RemoveContainer" containerID="824a0fd38b12b1737b80fff3e595b8299cdc7409920b5f901bddc1295e6a56f6" Jul 14 18:11:43.664276 ip-10-0-129-209 kubenswrapper[2562]: E0714 18:11:43.664171 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"824a0fd38b12b1737b80fff3e595b8299cdc7409920b5f901bddc1295e6a56f6\": container with ID starting with 824a0fd38b12b1737b80fff3e595b8299cdc7409920b5f901bddc1295e6a56f6 not found: ID does not exist" containerID="824a0fd38b12b1737b80fff3e595b8299cdc7409920b5f901bddc1295e6a56f6" Jul 14 18:11:43.664276 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:11:43.664207 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"824a0fd38b12b1737b80fff3e595b8299cdc7409920b5f901bddc1295e6a56f6"} err="failed to get container status \"824a0fd38b12b1737b80fff3e595b8299cdc7409920b5f901bddc1295e6a56f6\": rpc error: code = NotFound desc = could not find container \"824a0fd38b12b1737b80fff3e595b8299cdc7409920b5f901bddc1295e6a56f6\": container with ID starting with 824a0fd38b12b1737b80fff3e595b8299cdc7409920b5f901bddc1295e6a56f6 not found: ID does not exist" Jul 14 18:11:43.672879 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:11:43.672856 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-5a43f9-openshift-default-549d8686f9-ztvt5"] Jul 14 18:11:43.680235 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:11:43.680215 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-5a43f9-openshift-default-549d8686f9-ztvt5"] Jul 14 18:11:45.519141 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:11:45.519101 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="36c0e46a-2f62-479e-a42f-c910c0d5edaf" path="/var/lib/kubelet/pods/36c0e46a-2f62-479e-a42f-c910c0d5edaf/volumes" Jul 14 18:12:04.030244 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:04.030209 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-e9d840-openshift-default-5677d74ffc-twd6f"] Jul 14 18:12:04.030848 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:04.030414 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-e9d840-openshift-default-5677d74ffc-twd6f" podUID="6ad039bb-dff9-4919-83bb-0a5813fa01b4" containerName="istio-proxy" containerID="cri-o://1b595d93e84359a8291a032178706936991054319cd52c384d2c8cfdb84fd095" gracePeriod=30 Jul 14 18:12:05.044011 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:05.043978 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-aa324e-openshift-default-686754ddc9-zzml5"] Jul 14 18:12:05.044526 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:05.044507 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36c0e46a-2f62-479e-a42f-c910c0d5edaf" containerName="istio-proxy" Jul 14 18:12:05.044603 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:05.044529 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="36c0e46a-2f62-479e-a42f-c910c0d5edaf" containerName="istio-proxy" Jul 14 18:12:05.044603 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:05.044554 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36c0e46a-2f62-479e-a42f-c910c0d5edaf" containerName="istio-proxy" Jul 14 18:12:05.044603 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:05.044563 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="36c0e46a-2f62-479e-a42f-c910c0d5edaf" containerName="istio-proxy" Jul 14 18:12:05.044751 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:05.044676 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="36c0e46a-2f62-479e-a42f-c910c0d5edaf" containerName="istio-proxy" Jul 14 18:12:05.047448 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:05.047429 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-aa324e-openshift-default-686754ddc9-zzml5" Jul 14 18:12:05.050582 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:05.050563 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-aa324e-openshift-default-dockercfg-89r6z\"" Jul 14 18:12:05.070107 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:05.070086 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-aa324e-openshift-default-686754ddc9-zzml5"] Jul 14 18:12:05.129435 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:05.129391 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/79d0ddff-f4c2-4e3c-8009-a1fc5bed6354-istio-data\") pod \"e2e-shared-a-aa324e-openshift-default-686754ddc9-zzml5\" (UID: \"79d0ddff-f4c2-4e3c-8009-a1fc5bed6354\") " pod="openshift-ingress/e2e-shared-a-aa324e-openshift-default-686754ddc9-zzml5" Jul 14 18:12:05.129633 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:05.129454 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/79d0ddff-f4c2-4e3c-8009-a1fc5bed6354-workload-certs\") pod \"e2e-shared-a-aa324e-openshift-default-686754ddc9-zzml5\" (UID: \"79d0ddff-f4c2-4e3c-8009-a1fc5bed6354\") " pod="openshift-ingress/e2e-shared-a-aa324e-openshift-default-686754ddc9-zzml5" Jul 14 18:12:05.129633 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:05.129506 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/79d0ddff-f4c2-4e3c-8009-a1fc5bed6354-workload-socket\") pod \"e2e-shared-a-aa324e-openshift-default-686754ddc9-zzml5\" (UID: \"79d0ddff-f4c2-4e3c-8009-a1fc5bed6354\") " pod="openshift-ingress/e2e-shared-a-aa324e-openshift-default-686754ddc9-zzml5" Jul 14 18:12:05.129633 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:05.129540 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/79d0ddff-f4c2-4e3c-8009-a1fc5bed6354-istio-podinfo\") pod \"e2e-shared-a-aa324e-openshift-default-686754ddc9-zzml5\" (UID: \"79d0ddff-f4c2-4e3c-8009-a1fc5bed6354\") " pod="openshift-ingress/e2e-shared-a-aa324e-openshift-default-686754ddc9-zzml5" Jul 14 18:12:05.129802 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:05.129640 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/79d0ddff-f4c2-4e3c-8009-a1fc5bed6354-credential-socket\") pod \"e2e-shared-a-aa324e-openshift-default-686754ddc9-zzml5\" (UID: \"79d0ddff-f4c2-4e3c-8009-a1fc5bed6354\") " pod="openshift-ingress/e2e-shared-a-aa324e-openshift-default-686754ddc9-zzml5" Jul 14 18:12:05.129802 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:05.129726 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/79d0ddff-f4c2-4e3c-8009-a1fc5bed6354-istiod-ca-cert\") pod \"e2e-shared-a-aa324e-openshift-default-686754ddc9-zzml5\" (UID: \"79d0ddff-f4c2-4e3c-8009-a1fc5bed6354\") " pod="openshift-ingress/e2e-shared-a-aa324e-openshift-default-686754ddc9-zzml5" Jul 14 18:12:05.129802 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:05.129784 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/79d0ddff-f4c2-4e3c-8009-a1fc5bed6354-istio-envoy\") pod \"e2e-shared-a-aa324e-openshift-default-686754ddc9-zzml5\" (UID: \"79d0ddff-f4c2-4e3c-8009-a1fc5bed6354\") " pod="openshift-ingress/e2e-shared-a-aa324e-openshift-default-686754ddc9-zzml5" Jul 14 18:12:05.130001 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:05.129810 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/79d0ddff-f4c2-4e3c-8009-a1fc5bed6354-istio-token\") pod \"e2e-shared-a-aa324e-openshift-default-686754ddc9-zzml5\" (UID: \"79d0ddff-f4c2-4e3c-8009-a1fc5bed6354\") " pod="openshift-ingress/e2e-shared-a-aa324e-openshift-default-686754ddc9-zzml5" Jul 14 18:12:05.130001 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:05.129847 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wsfwm\" (UniqueName: \"kubernetes.io/projected/79d0ddff-f4c2-4e3c-8009-a1fc5bed6354-kube-api-access-wsfwm\") pod \"e2e-shared-a-aa324e-openshift-default-686754ddc9-zzml5\" (UID: \"79d0ddff-f4c2-4e3c-8009-a1fc5bed6354\") " pod="openshift-ingress/e2e-shared-a-aa324e-openshift-default-686754ddc9-zzml5" Jul 14 18:12:05.230726 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:05.230689 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/79d0ddff-f4c2-4e3c-8009-a1fc5bed6354-workload-socket\") pod \"e2e-shared-a-aa324e-openshift-default-686754ddc9-zzml5\" (UID: \"79d0ddff-f4c2-4e3c-8009-a1fc5bed6354\") " pod="openshift-ingress/e2e-shared-a-aa324e-openshift-default-686754ddc9-zzml5" Jul 14 18:12:05.230726 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:05.230736 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/79d0ddff-f4c2-4e3c-8009-a1fc5bed6354-istio-podinfo\") pod \"e2e-shared-a-aa324e-openshift-default-686754ddc9-zzml5\" (UID: \"79d0ddff-f4c2-4e3c-8009-a1fc5bed6354\") " pod="openshift-ingress/e2e-shared-a-aa324e-openshift-default-686754ddc9-zzml5" Jul 14 18:12:05.230973 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:05.230764 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/79d0ddff-f4c2-4e3c-8009-a1fc5bed6354-credential-socket\") pod \"e2e-shared-a-aa324e-openshift-default-686754ddc9-zzml5\" (UID: \"79d0ddff-f4c2-4e3c-8009-a1fc5bed6354\") " pod="openshift-ingress/e2e-shared-a-aa324e-openshift-default-686754ddc9-zzml5" Jul 14 18:12:05.230973 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:05.230840 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/79d0ddff-f4c2-4e3c-8009-a1fc5bed6354-istiod-ca-cert\") pod \"e2e-shared-a-aa324e-openshift-default-686754ddc9-zzml5\" (UID: \"79d0ddff-f4c2-4e3c-8009-a1fc5bed6354\") " pod="openshift-ingress/e2e-shared-a-aa324e-openshift-default-686754ddc9-zzml5" Jul 14 18:12:05.230973 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:05.230871 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/79d0ddff-f4c2-4e3c-8009-a1fc5bed6354-istio-envoy\") pod \"e2e-shared-a-aa324e-openshift-default-686754ddc9-zzml5\" (UID: \"79d0ddff-f4c2-4e3c-8009-a1fc5bed6354\") " pod="openshift-ingress/e2e-shared-a-aa324e-openshift-default-686754ddc9-zzml5" Jul 14 18:12:05.230973 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:05.230898 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/79d0ddff-f4c2-4e3c-8009-a1fc5bed6354-istio-token\") pod \"e2e-shared-a-aa324e-openshift-default-686754ddc9-zzml5\" (UID: \"79d0ddff-f4c2-4e3c-8009-a1fc5bed6354\") " pod="openshift-ingress/e2e-shared-a-aa324e-openshift-default-686754ddc9-zzml5" Jul 14 18:12:05.230973 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:05.230923 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wsfwm\" (UniqueName: \"kubernetes.io/projected/79d0ddff-f4c2-4e3c-8009-a1fc5bed6354-kube-api-access-wsfwm\") pod \"e2e-shared-a-aa324e-openshift-default-686754ddc9-zzml5\" (UID: \"79d0ddff-f4c2-4e3c-8009-a1fc5bed6354\") " pod="openshift-ingress/e2e-shared-a-aa324e-openshift-default-686754ddc9-zzml5" Jul 14 18:12:05.231268 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:05.230979 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/79d0ddff-f4c2-4e3c-8009-a1fc5bed6354-istio-data\") pod \"e2e-shared-a-aa324e-openshift-default-686754ddc9-zzml5\" (UID: \"79d0ddff-f4c2-4e3c-8009-a1fc5bed6354\") " pod="openshift-ingress/e2e-shared-a-aa324e-openshift-default-686754ddc9-zzml5" Jul 14 18:12:05.231268 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:05.231008 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/79d0ddff-f4c2-4e3c-8009-a1fc5bed6354-workload-certs\") pod \"e2e-shared-a-aa324e-openshift-default-686754ddc9-zzml5\" (UID: \"79d0ddff-f4c2-4e3c-8009-a1fc5bed6354\") " pod="openshift-ingress/e2e-shared-a-aa324e-openshift-default-686754ddc9-zzml5" Jul 14 18:12:05.231268 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:05.231210 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/79d0ddff-f4c2-4e3c-8009-a1fc5bed6354-workload-socket\") pod \"e2e-shared-a-aa324e-openshift-default-686754ddc9-zzml5\" (UID: \"79d0ddff-f4c2-4e3c-8009-a1fc5bed6354\") " pod="openshift-ingress/e2e-shared-a-aa324e-openshift-default-686754ddc9-zzml5" Jul 14 18:12:05.231429 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:05.231375 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/79d0ddff-f4c2-4e3c-8009-a1fc5bed6354-workload-certs\") pod \"e2e-shared-a-aa324e-openshift-default-686754ddc9-zzml5\" (UID: \"79d0ddff-f4c2-4e3c-8009-a1fc5bed6354\") " pod="openshift-ingress/e2e-shared-a-aa324e-openshift-default-686754ddc9-zzml5" Jul 14 18:12:05.231654 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:05.231628 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/79d0ddff-f4c2-4e3c-8009-a1fc5bed6354-istio-data\") pod \"e2e-shared-a-aa324e-openshift-default-686754ddc9-zzml5\" (UID: \"79d0ddff-f4c2-4e3c-8009-a1fc5bed6354\") " pod="openshift-ingress/e2e-shared-a-aa324e-openshift-default-686754ddc9-zzml5" Jul 14 18:12:05.232143 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:05.232050 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/79d0ddff-f4c2-4e3c-8009-a1fc5bed6354-istiod-ca-cert\") pod \"e2e-shared-a-aa324e-openshift-default-686754ddc9-zzml5\" (UID: \"79d0ddff-f4c2-4e3c-8009-a1fc5bed6354\") " pod="openshift-ingress/e2e-shared-a-aa324e-openshift-default-686754ddc9-zzml5" Jul 14 18:12:05.232418 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:05.232218 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/79d0ddff-f4c2-4e3c-8009-a1fc5bed6354-credential-socket\") pod \"e2e-shared-a-aa324e-openshift-default-686754ddc9-zzml5\" (UID: \"79d0ddff-f4c2-4e3c-8009-a1fc5bed6354\") " pod="openshift-ingress/e2e-shared-a-aa324e-openshift-default-686754ddc9-zzml5" Jul 14 18:12:05.233959 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:05.233937 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/79d0ddff-f4c2-4e3c-8009-a1fc5bed6354-istio-envoy\") pod \"e2e-shared-a-aa324e-openshift-default-686754ddc9-zzml5\" (UID: \"79d0ddff-f4c2-4e3c-8009-a1fc5bed6354\") " pod="openshift-ingress/e2e-shared-a-aa324e-openshift-default-686754ddc9-zzml5" Jul 14 18:12:05.234144 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:05.234076 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/79d0ddff-f4c2-4e3c-8009-a1fc5bed6354-istio-podinfo\") pod \"e2e-shared-a-aa324e-openshift-default-686754ddc9-zzml5\" (UID: \"79d0ddff-f4c2-4e3c-8009-a1fc5bed6354\") " pod="openshift-ingress/e2e-shared-a-aa324e-openshift-default-686754ddc9-zzml5" Jul 14 18:12:05.243561 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:05.243534 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/79d0ddff-f4c2-4e3c-8009-a1fc5bed6354-istio-token\") pod \"e2e-shared-a-aa324e-openshift-default-686754ddc9-zzml5\" (UID: \"79d0ddff-f4c2-4e3c-8009-a1fc5bed6354\") " pod="openshift-ingress/e2e-shared-a-aa324e-openshift-default-686754ddc9-zzml5" Jul 14 18:12:05.243942 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:05.243924 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wsfwm\" (UniqueName: \"kubernetes.io/projected/79d0ddff-f4c2-4e3c-8009-a1fc5bed6354-kube-api-access-wsfwm\") pod \"e2e-shared-a-aa324e-openshift-default-686754ddc9-zzml5\" (UID: \"79d0ddff-f4c2-4e3c-8009-a1fc5bed6354\") " pod="openshift-ingress/e2e-shared-a-aa324e-openshift-default-686754ddc9-zzml5" Jul 14 18:12:05.358074 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:05.358000 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-aa324e-openshift-default-686754ddc9-zzml5" Jul 14 18:12:05.517154 ip-10-0-129-209 kubenswrapper[2562]: W0714 18:12:05.517126 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79d0ddff_f4c2_4e3c_8009_a1fc5bed6354.slice/crio-3cc4ed7582a209f07201ff849f85895b2296348758426167a19ceccd0d6aaa52 WatchSource:0}: Error finding container 3cc4ed7582a209f07201ff849f85895b2296348758426167a19ceccd0d6aaa52: Status 404 returned error can't find the container with id 3cc4ed7582a209f07201ff849f85895b2296348758426167a19ceccd0d6aaa52 Jul 14 18:12:05.519366 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:05.519331 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 14 18:12:05.519454 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:05.519398 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 14 18:12:05.519454 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:05.519447 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 14 18:12:05.520256 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:05.520238 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-aa324e-openshift-default-686754ddc9-zzml5"] Jul 14 18:12:05.744124 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:05.744091 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-aa324e-openshift-default-686754ddc9-zzml5" event={"ID":"79d0ddff-f4c2-4e3c-8009-a1fc5bed6354","Type":"ContainerStarted","Data":"061a29a5941b074ad346b56a649dfcffb3e22dc707828fbe2368c847490fd6af"} Jul 14 18:12:05.744344 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:05.744151 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-aa324e-openshift-default-686754ddc9-zzml5" event={"ID":"79d0ddff-f4c2-4e3c-8009-a1fc5bed6354","Type":"ContainerStarted","Data":"3cc4ed7582a209f07201ff849f85895b2296348758426167a19ceccd0d6aaa52"} Jul 14 18:12:05.781239 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:05.775974 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-aa324e-openshift-default-686754ddc9-zzml5" podStartSLOduration=0.775954557 podStartE2EDuration="775.954557ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 18:12:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 18:12:05.774057805 +0000 UTC m=+2374.850515346" watchObservedRunningTime="2026-07-14 18:12:05.775954557 +0000 UTC m=+2374.852412098" Jul 14 18:12:06.358511 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:06.358476 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-aa324e-openshift-default-686754ddc9-zzml5" Jul 14 18:12:06.363577 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:06.363550 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-aa324e-openshift-default-686754ddc9-zzml5" Jul 14 18:12:06.749058 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:06.749029 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-aa324e-openshift-default-686754ddc9-zzml5" Jul 14 18:12:06.749909 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:06.749890 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-aa324e-openshift-default-686754ddc9-zzml5" Jul 14 18:12:09.276881 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:09.276857 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-e9d840-openshift-default-5677d74ffc-twd6f" Jul 14 18:12:09.367574 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:09.367543 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6ad039bb-dff9-4919-83bb-0a5813fa01b4-istio-podinfo\") pod \"6ad039bb-dff9-4919-83bb-0a5813fa01b4\" (UID: \"6ad039bb-dff9-4919-83bb-0a5813fa01b4\") " Jul 14 18:12:09.367759 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:09.367585 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6ad039bb-dff9-4919-83bb-0a5813fa01b4-workload-certs\") pod \"6ad039bb-dff9-4919-83bb-0a5813fa01b4\" (UID: \"6ad039bb-dff9-4919-83bb-0a5813fa01b4\") " Jul 14 18:12:09.367759 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:09.367625 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rgshc\" (UniqueName: \"kubernetes.io/projected/6ad039bb-dff9-4919-83bb-0a5813fa01b4-kube-api-access-rgshc\") pod \"6ad039bb-dff9-4919-83bb-0a5813fa01b4\" (UID: \"6ad039bb-dff9-4919-83bb-0a5813fa01b4\") " Jul 14 18:12:09.367759 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:09.367680 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6ad039bb-dff9-4919-83bb-0a5813fa01b4-istiod-ca-cert\") pod \"6ad039bb-dff9-4919-83bb-0a5813fa01b4\" (UID: \"6ad039bb-dff9-4919-83bb-0a5813fa01b4\") " Jul 14 18:12:09.367759 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:09.367716 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ad039bb-dff9-4919-83bb-0a5813fa01b4-workload-socket\") pod \"6ad039bb-dff9-4919-83bb-0a5813fa01b4\" (UID: \"6ad039bb-dff9-4919-83bb-0a5813fa01b4\") " Jul 14 18:12:09.367759 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:09.367740 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6ad039bb-dff9-4919-83bb-0a5813fa01b4-istio-envoy\") pod \"6ad039bb-dff9-4919-83bb-0a5813fa01b4\" (UID: \"6ad039bb-dff9-4919-83bb-0a5813fa01b4\") " Jul 14 18:12:09.368033 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:09.367772 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ad039bb-dff9-4919-83bb-0a5813fa01b4-credential-socket\") pod \"6ad039bb-dff9-4919-83bb-0a5813fa01b4\" (UID: \"6ad039bb-dff9-4919-83bb-0a5813fa01b4\") " Jul 14 18:12:09.368033 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:09.367805 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6ad039bb-dff9-4919-83bb-0a5813fa01b4-istio-data\") pod \"6ad039bb-dff9-4919-83bb-0a5813fa01b4\" (UID: \"6ad039bb-dff9-4919-83bb-0a5813fa01b4\") " Jul 14 18:12:09.368033 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:09.367824 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ad039bb-dff9-4919-83bb-0a5813fa01b4-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6ad039bb-dff9-4919-83bb-0a5813fa01b4" (UID: "6ad039bb-dff9-4919-83bb-0a5813fa01b4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 18:12:09.368033 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:09.367838 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6ad039bb-dff9-4919-83bb-0a5813fa01b4-istio-token\") pod \"6ad039bb-dff9-4919-83bb-0a5813fa01b4\" (UID: \"6ad039bb-dff9-4919-83bb-0a5813fa01b4\") " Jul 14 18:12:09.368353 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:09.368013 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ad039bb-dff9-4919-83bb-0a5813fa01b4-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6ad039bb-dff9-4919-83bb-0a5813fa01b4" (UID: "6ad039bb-dff9-4919-83bb-0a5813fa01b4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 18:12:09.368353 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:09.368081 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ad039bb-dff9-4919-83bb-0a5813fa01b4-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6ad039bb-dff9-4919-83bb-0a5813fa01b4" (UID: "6ad039bb-dff9-4919-83bb-0a5813fa01b4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 18:12:09.368353 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:09.368084 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6ad039bb-dff9-4919-83bb-0a5813fa01b4-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6ad039bb-dff9-4919-83bb-0a5813fa01b4" (UID: "6ad039bb-dff9-4919-83bb-0a5813fa01b4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 18:12:09.368353 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:09.368256 2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6ad039bb-dff9-4919-83bb-0a5813fa01b4-workload-certs\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 18:12:09.368353 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:09.368274 2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6ad039bb-dff9-4919-83bb-0a5813fa01b4-istiod-ca-cert\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 18:12:09.368353 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:09.368286 2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ad039bb-dff9-4919-83bb-0a5813fa01b4-workload-socket\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 18:12:09.368353 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:09.368300 2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6ad039bb-dff9-4919-83bb-0a5813fa01b4-credential-socket\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 18:12:09.368998 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:09.368971 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ad039bb-dff9-4919-83bb-0a5813fa01b4-istio-data" (OuterVolumeSpecName: "istio-data") pod "6ad039bb-dff9-4919-83bb-0a5813fa01b4" (UID: "6ad039bb-dff9-4919-83bb-0a5813fa01b4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 18:12:09.369891 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:09.369867 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6ad039bb-dff9-4919-83bb-0a5813fa01b4-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6ad039bb-dff9-4919-83bb-0a5813fa01b4" (UID: "6ad039bb-dff9-4919-83bb-0a5813fa01b4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 14 18:12:09.370005 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:09.369987 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6ad039bb-dff9-4919-83bb-0a5813fa01b4-kube-api-access-rgshc" (OuterVolumeSpecName: "kube-api-access-rgshc") pod "6ad039bb-dff9-4919-83bb-0a5813fa01b4" (UID: "6ad039bb-dff9-4919-83bb-0a5813fa01b4"). InnerVolumeSpecName "kube-api-access-rgshc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 18:12:09.370109 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:09.370089 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6ad039bb-dff9-4919-83bb-0a5813fa01b4-istio-token" (OuterVolumeSpecName: "istio-token") pod "6ad039bb-dff9-4919-83bb-0a5813fa01b4" (UID: "6ad039bb-dff9-4919-83bb-0a5813fa01b4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 18:12:09.370314 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:09.370298 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ad039bb-dff9-4919-83bb-0a5813fa01b4-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6ad039bb-dff9-4919-83bb-0a5813fa01b4" (UID: "6ad039bb-dff9-4919-83bb-0a5813fa01b4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 18:12:09.468973 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:09.468940 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rgshc\" (UniqueName: \"kubernetes.io/projected/6ad039bb-dff9-4919-83bb-0a5813fa01b4-kube-api-access-rgshc\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 18:12:09.468973 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:09.468968 2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6ad039bb-dff9-4919-83bb-0a5813fa01b4-istio-envoy\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 18:12:09.468973 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:09.468978 2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6ad039bb-dff9-4919-83bb-0a5813fa01b4-istio-data\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 18:12:09.469215 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:09.468987 2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6ad039bb-dff9-4919-83bb-0a5813fa01b4-istio-token\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 18:12:09.469215 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:09.468996 2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6ad039bb-dff9-4919-83bb-0a5813fa01b4-istio-podinfo\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 18:12:09.764220 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:09.764102 2562 generic.go:356] "Generic (PLEG): container finished" podID="6ad039bb-dff9-4919-83bb-0a5813fa01b4" containerID="1b595d93e84359a8291a032178706936991054319cd52c384d2c8cfdb84fd095" exitCode=0 Jul 14 18:12:09.764220 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:09.764148 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-e9d840-openshift-default-5677d74ffc-twd6f" Jul 14 18:12:09.764459 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:09.764209 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-e9d840-openshift-default-5677d74ffc-twd6f" event={"ID":"6ad039bb-dff9-4919-83bb-0a5813fa01b4","Type":"ContainerDied","Data":"1b595d93e84359a8291a032178706936991054319cd52c384d2c8cfdb84fd095"} Jul 14 18:12:09.764459 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:09.764276 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-e9d840-openshift-default-5677d74ffc-twd6f" event={"ID":"6ad039bb-dff9-4919-83bb-0a5813fa01b4","Type":"ContainerDied","Data":"ebeea895d09afd44d972410d221b9acc8a79b1dc0094f75908f6042587b1a7c5"} Jul 14 18:12:09.764459 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:09.764309 2562 scope.go:117] "RemoveContainer" containerID="1b595d93e84359a8291a032178706936991054319cd52c384d2c8cfdb84fd095" Jul 14 18:12:09.774752 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:09.774731 2562 scope.go:117] "RemoveContainer" containerID="911a85478890c5b2a8de0e1e9daf2c3a9c43581ad35c3238c806ba6bc6ca22e4" Jul 14 18:12:09.782567 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:09.782552 2562 scope.go:117] "RemoveContainer" containerID="1b595d93e84359a8291a032178706936991054319cd52c384d2c8cfdb84fd095" Jul 14 18:12:09.782815 ip-10-0-129-209 kubenswrapper[2562]: E0714 18:12:09.782798 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1b595d93e84359a8291a032178706936991054319cd52c384d2c8cfdb84fd095\": container with ID starting with 1b595d93e84359a8291a032178706936991054319cd52c384d2c8cfdb84fd095 not found: ID does not exist" containerID="1b595d93e84359a8291a032178706936991054319cd52c384d2c8cfdb84fd095" Jul 14 18:12:09.782865 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:09.782822 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b595d93e84359a8291a032178706936991054319cd52c384d2c8cfdb84fd095"} err="failed to get container status \"1b595d93e84359a8291a032178706936991054319cd52c384d2c8cfdb84fd095\": rpc error: code = NotFound desc = could not find container \"1b595d93e84359a8291a032178706936991054319cd52c384d2c8cfdb84fd095\": container with ID starting with 1b595d93e84359a8291a032178706936991054319cd52c384d2c8cfdb84fd095 not found: ID does not exist" Jul 14 18:12:09.782865 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:09.782839 2562 scope.go:117] "RemoveContainer" containerID="911a85478890c5b2a8de0e1e9daf2c3a9c43581ad35c3238c806ba6bc6ca22e4" Jul 14 18:12:09.783081 ip-10-0-129-209 kubenswrapper[2562]: E0714 18:12:09.783058 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"911a85478890c5b2a8de0e1e9daf2c3a9c43581ad35c3238c806ba6bc6ca22e4\": container with ID starting with 911a85478890c5b2a8de0e1e9daf2c3a9c43581ad35c3238c806ba6bc6ca22e4 not found: ID does not exist" containerID="911a85478890c5b2a8de0e1e9daf2c3a9c43581ad35c3238c806ba6bc6ca22e4" Jul 14 18:12:09.783178 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:09.783091 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"911a85478890c5b2a8de0e1e9daf2c3a9c43581ad35c3238c806ba6bc6ca22e4"} err="failed to get container status \"911a85478890c5b2a8de0e1e9daf2c3a9c43581ad35c3238c806ba6bc6ca22e4\": rpc error: code = NotFound desc = could not find container \"911a85478890c5b2a8de0e1e9daf2c3a9c43581ad35c3238c806ba6bc6ca22e4\": container with ID starting with 911a85478890c5b2a8de0e1e9daf2c3a9c43581ad35c3238c806ba6bc6ca22e4 not found: ID does not exist" Jul 14 18:12:09.790407 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:09.790384 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-e9d840-openshift-default-5677d74ffc-twd6f"] Jul 14 18:12:09.797277 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:09.797255 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-e9d840-openshift-default-5677d74ffc-twd6f"] Jul 14 18:12:11.519488 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:11.519454 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6ad039bb-dff9-4919-83bb-0a5813fa01b4" path="/var/lib/kubelet/pods/6ad039bb-dff9-4919-83bb-0a5813fa01b4/volumes" Jul 14 18:12:12.872131 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:12.872099 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-877bd7-openshift-default-5cc7bb575-b5shn"] Jul 14 18:12:12.872520 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:12.872453 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ad039bb-dff9-4919-83bb-0a5813fa01b4" containerName="istio-proxy" Jul 14 18:12:12.872520 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:12.872464 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ad039bb-dff9-4919-83bb-0a5813fa01b4" containerName="istio-proxy" Jul 14 18:12:12.872611 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:12.872527 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="6ad039bb-dff9-4919-83bb-0a5813fa01b4" containerName="istio-proxy" Jul 14 18:12:12.872611 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:12.872534 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="36c0e46a-2f62-479e-a42f-c910c0d5edaf" containerName="istio-proxy" Jul 14 18:12:12.872611 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:12.872542 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="6ad039bb-dff9-4919-83bb-0a5813fa01b4" containerName="istio-proxy" Jul 14 18:12:12.875301 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:12.875282 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-877bd7-openshift-default-5cc7bb575-b5shn" Jul 14 18:12:12.877866 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:12.877847 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-877bd7-openshift-default-dockercfg-ssptb\"" Jul 14 18:12:12.897093 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:12.897064 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-877bd7-openshift-default-5cc7bb575-b5shn"] Jul 14 18:12:13.000133 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:13.000091 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/12201025-f9d5-4fd8-a048-f9d85c278670-istiod-ca-cert\") pod \"e2e-shared-b-877bd7-openshift-default-5cc7bb575-b5shn\" (UID: \"12201025-f9d5-4fd8-a048-f9d85c278670\") " pod="openshift-ingress/e2e-shared-b-877bd7-openshift-default-5cc7bb575-b5shn" Jul 14 18:12:13.000133 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:13.000132 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/12201025-f9d5-4fd8-a048-f9d85c278670-istio-data\") pod \"e2e-shared-b-877bd7-openshift-default-5cc7bb575-b5shn\" (UID: \"12201025-f9d5-4fd8-a048-f9d85c278670\") " pod="openshift-ingress/e2e-shared-b-877bd7-openshift-default-5cc7bb575-b5shn" Jul 14 18:12:13.000422 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:13.000267 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/12201025-f9d5-4fd8-a048-f9d85c278670-credential-socket\") pod \"e2e-shared-b-877bd7-openshift-default-5cc7bb575-b5shn\" (UID: \"12201025-f9d5-4fd8-a048-f9d85c278670\") " pod="openshift-ingress/e2e-shared-b-877bd7-openshift-default-5cc7bb575-b5shn" Jul 14 18:12:13.000422 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:13.000311 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/12201025-f9d5-4fd8-a048-f9d85c278670-istio-token\") pod \"e2e-shared-b-877bd7-openshift-default-5cc7bb575-b5shn\" (UID: \"12201025-f9d5-4fd8-a048-f9d85c278670\") " pod="openshift-ingress/e2e-shared-b-877bd7-openshift-default-5cc7bb575-b5shn" Jul 14 18:12:13.000422 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:13.000344 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/12201025-f9d5-4fd8-a048-f9d85c278670-istio-envoy\") pod \"e2e-shared-b-877bd7-openshift-default-5cc7bb575-b5shn\" (UID: \"12201025-f9d5-4fd8-a048-f9d85c278670\") " pod="openshift-ingress/e2e-shared-b-877bd7-openshift-default-5cc7bb575-b5shn" Jul 14 18:12:13.000422 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:13.000392 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/12201025-f9d5-4fd8-a048-f9d85c278670-istio-podinfo\") pod \"e2e-shared-b-877bd7-openshift-default-5cc7bb575-b5shn\" (UID: \"12201025-f9d5-4fd8-a048-f9d85c278670\") " pod="openshift-ingress/e2e-shared-b-877bd7-openshift-default-5cc7bb575-b5shn" Jul 14 18:12:13.000628 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:13.000450 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n7dsb\" (UniqueName: \"kubernetes.io/projected/12201025-f9d5-4fd8-a048-f9d85c278670-kube-api-access-n7dsb\") pod \"e2e-shared-b-877bd7-openshift-default-5cc7bb575-b5shn\" (UID: \"12201025-f9d5-4fd8-a048-f9d85c278670\") " pod="openshift-ingress/e2e-shared-b-877bd7-openshift-default-5cc7bb575-b5shn" Jul 14 18:12:13.000628 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:13.000518 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/12201025-f9d5-4fd8-a048-f9d85c278670-workload-socket\") pod \"e2e-shared-b-877bd7-openshift-default-5cc7bb575-b5shn\" (UID: \"12201025-f9d5-4fd8-a048-f9d85c278670\") " pod="openshift-ingress/e2e-shared-b-877bd7-openshift-default-5cc7bb575-b5shn" Jul 14 18:12:13.000628 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:13.000545 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/12201025-f9d5-4fd8-a048-f9d85c278670-workload-certs\") pod \"e2e-shared-b-877bd7-openshift-default-5cc7bb575-b5shn\" (UID: \"12201025-f9d5-4fd8-a048-f9d85c278670\") " pod="openshift-ingress/e2e-shared-b-877bd7-openshift-default-5cc7bb575-b5shn" Jul 14 18:12:13.101642 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:13.101595 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/12201025-f9d5-4fd8-a048-f9d85c278670-credential-socket\") pod \"e2e-shared-b-877bd7-openshift-default-5cc7bb575-b5shn\" (UID: \"12201025-f9d5-4fd8-a048-f9d85c278670\") " pod="openshift-ingress/e2e-shared-b-877bd7-openshift-default-5cc7bb575-b5shn" Jul 14 18:12:13.101835 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:13.101660 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/12201025-f9d5-4fd8-a048-f9d85c278670-istio-token\") pod \"e2e-shared-b-877bd7-openshift-default-5cc7bb575-b5shn\" (UID: \"12201025-f9d5-4fd8-a048-f9d85c278670\") " pod="openshift-ingress/e2e-shared-b-877bd7-openshift-default-5cc7bb575-b5shn" Jul 14 18:12:13.101835 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:13.101690 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/12201025-f9d5-4fd8-a048-f9d85c278670-istio-envoy\") pod \"e2e-shared-b-877bd7-openshift-default-5cc7bb575-b5shn\" (UID: \"12201025-f9d5-4fd8-a048-f9d85c278670\") " pod="openshift-ingress/e2e-shared-b-877bd7-openshift-default-5cc7bb575-b5shn" Jul 14 18:12:13.101835 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:13.101756 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/12201025-f9d5-4fd8-a048-f9d85c278670-istio-podinfo\") pod \"e2e-shared-b-877bd7-openshift-default-5cc7bb575-b5shn\" (UID: \"12201025-f9d5-4fd8-a048-f9d85c278670\") " pod="openshift-ingress/e2e-shared-b-877bd7-openshift-default-5cc7bb575-b5shn" Jul 14 18:12:13.101835 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:13.101783 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n7dsb\" (UniqueName: \"kubernetes.io/projected/12201025-f9d5-4fd8-a048-f9d85c278670-kube-api-access-n7dsb\") pod \"e2e-shared-b-877bd7-openshift-default-5cc7bb575-b5shn\" (UID: \"12201025-f9d5-4fd8-a048-f9d85c278670\") " pod="openshift-ingress/e2e-shared-b-877bd7-openshift-default-5cc7bb575-b5shn" Jul 14 18:12:13.101835 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:13.101834 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/12201025-f9d5-4fd8-a048-f9d85c278670-workload-socket\") pod \"e2e-shared-b-877bd7-openshift-default-5cc7bb575-b5shn\" (UID: \"12201025-f9d5-4fd8-a048-f9d85c278670\") " pod="openshift-ingress/e2e-shared-b-877bd7-openshift-default-5cc7bb575-b5shn" Jul 14 18:12:13.102116 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:13.101864 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/12201025-f9d5-4fd8-a048-f9d85c278670-workload-certs\") pod \"e2e-shared-b-877bd7-openshift-default-5cc7bb575-b5shn\" (UID: \"12201025-f9d5-4fd8-a048-f9d85c278670\") " pod="openshift-ingress/e2e-shared-b-877bd7-openshift-default-5cc7bb575-b5shn" Jul 14 18:12:13.102116 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:13.101930 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/12201025-f9d5-4fd8-a048-f9d85c278670-istiod-ca-cert\") pod \"e2e-shared-b-877bd7-openshift-default-5cc7bb575-b5shn\" (UID: \"12201025-f9d5-4fd8-a048-f9d85c278670\") " pod="openshift-ingress/e2e-shared-b-877bd7-openshift-default-5cc7bb575-b5shn" Jul 14 18:12:13.102116 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:13.101964 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/12201025-f9d5-4fd8-a048-f9d85c278670-istio-data\") pod \"e2e-shared-b-877bd7-openshift-default-5cc7bb575-b5shn\" (UID: \"12201025-f9d5-4fd8-a048-f9d85c278670\") " pod="openshift-ingress/e2e-shared-b-877bd7-openshift-default-5cc7bb575-b5shn" Jul 14 18:12:13.102116 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:13.101983 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/12201025-f9d5-4fd8-a048-f9d85c278670-credential-socket\") pod \"e2e-shared-b-877bd7-openshift-default-5cc7bb575-b5shn\" (UID: \"12201025-f9d5-4fd8-a048-f9d85c278670\") " pod="openshift-ingress/e2e-shared-b-877bd7-openshift-default-5cc7bb575-b5shn" Jul 14 18:12:13.102354 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:13.102265 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/12201025-f9d5-4fd8-a048-f9d85c278670-workload-socket\") pod \"e2e-shared-b-877bd7-openshift-default-5cc7bb575-b5shn\" (UID: \"12201025-f9d5-4fd8-a048-f9d85c278670\") " pod="openshift-ingress/e2e-shared-b-877bd7-openshift-default-5cc7bb575-b5shn" Jul 14 18:12:13.102354 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:13.102282 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/12201025-f9d5-4fd8-a048-f9d85c278670-istio-data\") pod \"e2e-shared-b-877bd7-openshift-default-5cc7bb575-b5shn\" (UID: \"12201025-f9d5-4fd8-a048-f9d85c278670\") " pod="openshift-ingress/e2e-shared-b-877bd7-openshift-default-5cc7bb575-b5shn" Jul 14 18:12:13.102473 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:13.102369 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/12201025-f9d5-4fd8-a048-f9d85c278670-workload-certs\") pod \"e2e-shared-b-877bd7-openshift-default-5cc7bb575-b5shn\" (UID: \"12201025-f9d5-4fd8-a048-f9d85c278670\") " pod="openshift-ingress/e2e-shared-b-877bd7-openshift-default-5cc7bb575-b5shn" Jul 14 18:12:13.102757 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:13.102734 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/12201025-f9d5-4fd8-a048-f9d85c278670-istiod-ca-cert\") pod \"e2e-shared-b-877bd7-openshift-default-5cc7bb575-b5shn\" (UID: \"12201025-f9d5-4fd8-a048-f9d85c278670\") " pod="openshift-ingress/e2e-shared-b-877bd7-openshift-default-5cc7bb575-b5shn" Jul 14 18:12:13.104314 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:13.104288 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/12201025-f9d5-4fd8-a048-f9d85c278670-istio-envoy\") pod \"e2e-shared-b-877bd7-openshift-default-5cc7bb575-b5shn\" (UID: \"12201025-f9d5-4fd8-a048-f9d85c278670\") " pod="openshift-ingress/e2e-shared-b-877bd7-openshift-default-5cc7bb575-b5shn" Jul 14 18:12:13.104444 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:13.104351 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/12201025-f9d5-4fd8-a048-f9d85c278670-istio-podinfo\") pod \"e2e-shared-b-877bd7-openshift-default-5cc7bb575-b5shn\" (UID: \"12201025-f9d5-4fd8-a048-f9d85c278670\") " pod="openshift-ingress/e2e-shared-b-877bd7-openshift-default-5cc7bb575-b5shn" Jul 14 18:12:13.115230 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:13.115208 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/12201025-f9d5-4fd8-a048-f9d85c278670-istio-token\") pod \"e2e-shared-b-877bd7-openshift-default-5cc7bb575-b5shn\" (UID: \"12201025-f9d5-4fd8-a048-f9d85c278670\") " pod="openshift-ingress/e2e-shared-b-877bd7-openshift-default-5cc7bb575-b5shn" Jul 14 18:12:13.115531 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:13.115510 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n7dsb\" (UniqueName: \"kubernetes.io/projected/12201025-f9d5-4fd8-a048-f9d85c278670-kube-api-access-n7dsb\") pod \"e2e-shared-b-877bd7-openshift-default-5cc7bb575-b5shn\" (UID: \"12201025-f9d5-4fd8-a048-f9d85c278670\") " pod="openshift-ingress/e2e-shared-b-877bd7-openshift-default-5cc7bb575-b5shn" Jul 14 18:12:13.185847 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:13.185755 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-877bd7-openshift-default-5cc7bb575-b5shn" Jul 14 18:12:13.388905 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:13.388877 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-877bd7-openshift-default-5cc7bb575-b5shn"] Jul 14 18:12:13.390352 ip-10-0-129-209 kubenswrapper[2562]: W0714 18:12:13.390312 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12201025_f9d5_4fd8_a048_f9d85c278670.slice/crio-3ba6faee352d87a253cf9b352ae14b2f718fe49e31af292c5cda53e12ab821cc WatchSource:0}: Error finding container 3ba6faee352d87a253cf9b352ae14b2f718fe49e31af292c5cda53e12ab821cc: Status 404 returned error can't find the container with id 3ba6faee352d87a253cf9b352ae14b2f718fe49e31af292c5cda53e12ab821cc Jul 14 18:12:13.392449 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:13.392421 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 14 18:12:13.392514 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:13.392484 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 14 18:12:13.392550 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:13.392514 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 14 18:12:13.785108 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:13.785013 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-877bd7-openshift-default-5cc7bb575-b5shn" event={"ID":"12201025-f9d5-4fd8-a048-f9d85c278670","Type":"ContainerStarted","Data":"e815999abab790f1558dbcd0b774a2d510ccc0cc6eb4c434f52930b84111d915"} Jul 14 18:12:13.785108 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:13.785065 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-877bd7-openshift-default-5cc7bb575-b5shn" event={"ID":"12201025-f9d5-4fd8-a048-f9d85c278670","Type":"ContainerStarted","Data":"3ba6faee352d87a253cf9b352ae14b2f718fe49e31af292c5cda53e12ab821cc"} Jul 14 18:12:13.816684 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:13.816634 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-877bd7-openshift-default-5cc7bb575-b5shn" podStartSLOduration=1.816614871 podStartE2EDuration="1.816614871s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-14 18:12:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 18:12:13.816589854 +0000 UTC m=+2382.893047419" watchObservedRunningTime="2026-07-14 18:12:13.816614871 +0000 UTC m=+2382.893072404" Jul 14 18:12:14.186072 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:14.186018 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-877bd7-openshift-default-5cc7bb575-b5shn" Jul 14 18:12:14.190992 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:14.190965 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-877bd7-openshift-default-5cc7bb575-b5shn" Jul 14 18:12:14.789775 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:14.789746 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-877bd7-openshift-default-5cc7bb575-b5shn" Jul 14 18:12:14.790779 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:14.790761 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-877bd7-openshift-default-5cc7bb575-b5shn" Jul 14 18:12:20.815973 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:20.815938 2562 generic.go:356] "Generic (PLEG): container finished" podID="79d0ddff-f4c2-4e3c-8009-a1fc5bed6354" containerID="061a29a5941b074ad346b56a649dfcffb3e22dc707828fbe2368c847490fd6af" exitCode=0 Jul 14 18:12:20.816392 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:20.815990 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-aa324e-openshift-default-686754ddc9-zzml5" event={"ID":"79d0ddff-f4c2-4e3c-8009-a1fc5bed6354","Type":"ContainerDied","Data":"061a29a5941b074ad346b56a649dfcffb3e22dc707828fbe2368c847490fd6af"} Jul 14 18:12:20.816392 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:20.816316 2562 scope.go:117] "RemoveContainer" containerID="061a29a5941b074ad346b56a649dfcffb3e22dc707828fbe2368c847490fd6af" Jul 14 18:12:20.817252 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:20.817227 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 14 18:12:20.817304 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:20.817294 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 14 18:12:20.817349 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:20.817326 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 14 18:12:21.749265 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:21.749219 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-aa324e-openshift-default-686754ddc9-zzml5" Jul 14 18:12:21.822507 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:21.822472 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-aa324e-openshift-default-686754ddc9-zzml5" event={"ID":"79d0ddff-f4c2-4e3c-8009-a1fc5bed6354","Type":"ContainerStarted","Data":"09f618c54d4df37190968306c87a6a51fe3b1eca210b255235c617a7dd3ba36a"} Jul 14 18:12:22.359161 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:22.359116 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-aa324e-openshift-default-686754ddc9-zzml5" Jul 14 18:12:22.361805 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:22.360381 2562 patch_prober.go:28] interesting pod/e2e-shared-a-aa324e-openshift-default-686754ddc9-zzml5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body= Jul 14 18:12:22.361805 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:22.360440 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-aa324e-openshift-default-686754ddc9-zzml5" podUID="79d0ddff-f4c2-4e3c-8009-a1fc5bed6354" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" Jul 14 18:12:23.358643 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:23.358593 2562 patch_prober.go:28] interesting pod/e2e-shared-a-aa324e-openshift-default-686754ddc9-zzml5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body= Jul 14 18:12:23.359145 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:23.358682 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-aa324e-openshift-default-686754ddc9-zzml5" podUID="79d0ddff-f4c2-4e3c-8009-a1fc5bed6354" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" Jul 14 18:12:24.359396 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:24.359355 2562 patch_prober.go:28] interesting pod/e2e-shared-a-aa324e-openshift-default-686754ddc9-zzml5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body= Jul 14 18:12:24.359994 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:24.359961 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-aa324e-openshift-default-686754ddc9-zzml5" podUID="79d0ddff-f4c2-4e3c-8009-a1fc5bed6354" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" Jul 14 18:12:25.359062 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:25.359017 2562 patch_prober.go:28] interesting pod/e2e-shared-a-aa324e-openshift-default-686754ddc9-zzml5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body= Jul 14 18:12:25.359291 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:25.359086 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-aa324e-openshift-default-686754ddc9-zzml5" podUID="79d0ddff-f4c2-4e3c-8009-a1fc5bed6354" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" Jul 14 18:12:26.373810 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:26.373775 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-aa324e-openshift-default-686754ddc9-zzml5" Jul 14 18:12:26.375414 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:26.375384 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-aa324e-openshift-default-686754ddc9-zzml5" Jul 14 18:12:26.376517 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:26.376492 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-aa324e-openshift-default-686754ddc9-zzml5" Jul 14 18:12:30.831347 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:30.831302 2562 patch_prober.go:28] interesting pod/e2e-shared-b-877bd7-openshift-default-5cc7bb575-b5shn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 14 18:12:30.831749 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:30.831376 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-877bd7-openshift-default-5cc7bb575-b5shn" podUID="12201025-f9d5-4fd8-a048-f9d85c278670" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 14 18:12:31.529919 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:31.529854 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2sbww_c3d00b23-d0d5-4d18-a722-464d7b114745/ovn-acl-logging/0.log" Jul 14 18:12:31.534552 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:31.534532 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2sbww_c3d00b23-d0d5-4d18-a722-464d7b114745/ovn-acl-logging/0.log" Jul 14 18:12:31.876729 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:31.876695 2562 generic.go:356] "Generic (PLEG): container finished" podID="12201025-f9d5-4fd8-a048-f9d85c278670" containerID="e815999abab790f1558dbcd0b774a2d510ccc0cc6eb4c434f52930b84111d915" exitCode=0 Jul 14 18:12:31.877095 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:31.876747 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-877bd7-openshift-default-5cc7bb575-b5shn" event={"ID":"12201025-f9d5-4fd8-a048-f9d85c278670","Type":"ContainerDied","Data":"e815999abab790f1558dbcd0b774a2d510ccc0cc6eb4c434f52930b84111d915"} Jul 14 18:12:31.877158 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:31.877142 2562 scope.go:117] "RemoveContainer" containerID="e815999abab790f1558dbcd0b774a2d510ccc0cc6eb4c434f52930b84111d915" Jul 14 18:12:31.878178 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:31.878152 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 14 18:12:31.878248 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:31.878231 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 14 18:12:31.878290 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:31.878263 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 14 18:12:32.882963 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:32.882934 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-877bd7-openshift-default-5cc7bb575-b5shn" event={"ID":"12201025-f9d5-4fd8-a048-f9d85c278670","Type":"ContainerStarted","Data":"5868d564303822c71bb4eb769e6d11e33a595adfe899a27e9dccb15701f941b9"} Jul 14 18:12:33.186335 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:33.186229 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-877bd7-openshift-default-5cc7bb575-b5shn" Jul 14 18:12:33.186502 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:33.186478 2562 patch_prober.go:28] interesting pod/e2e-shared-b-877bd7-openshift-default-5cc7bb575-b5shn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body= Jul 14 18:12:33.186546 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:33.186527 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-877bd7-openshift-default-5cc7bb575-b5shn" podUID="12201025-f9d5-4fd8-a048-f9d85c278670" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" Jul 14 18:12:34.186419 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:34.186381 2562 patch_prober.go:28] interesting pod/e2e-shared-b-877bd7-openshift-default-5cc7bb575-b5shn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body= Jul 14 18:12:34.186842 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:34.186438 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-877bd7-openshift-default-5cc7bb575-b5shn" podUID="12201025-f9d5-4fd8-a048-f9d85c278670" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" Jul 14 18:12:35.187025 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:35.186981 2562 patch_prober.go:28] interesting pod/e2e-shared-b-877bd7-openshift-default-5cc7bb575-b5shn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body= Jul 14 18:12:35.187530 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:35.187046 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-877bd7-openshift-default-5cc7bb575-b5shn" podUID="12201025-f9d5-4fd8-a048-f9d85c278670" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" Jul 14 18:12:36.187087 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:36.187044 2562 patch_prober.go:28] interesting pod/e2e-shared-b-877bd7-openshift-default-5cc7bb575-b5shn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body= Jul 14 18:12:36.187658 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:36.187104 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-877bd7-openshift-default-5cc7bb575-b5shn" podUID="12201025-f9d5-4fd8-a048-f9d85c278670" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" Jul 14 18:12:37.186921 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:37.186887 2562 patch_prober.go:28] interesting pod/e2e-shared-b-877bd7-openshift-default-5cc7bb575-b5shn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body= Jul 14 18:12:37.187094 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:37.186946 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-877bd7-openshift-default-5cc7bb575-b5shn" podUID="12201025-f9d5-4fd8-a048-f9d85c278670" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" Jul 14 18:12:38.186457 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:38.186420 2562 patch_prober.go:28] interesting pod/e2e-shared-b-877bd7-openshift-default-5cc7bb575-b5shn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body= Jul 14 18:12:38.186879 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:38.186480 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-877bd7-openshift-default-5cc7bb575-b5shn" podUID="12201025-f9d5-4fd8-a048-f9d85c278670" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" Jul 14 18:12:39.189930 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:39.189901 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-877bd7-openshift-default-5cc7bb575-b5shn" Jul 14 18:12:39.190395 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:39.190171 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-877bd7-openshift-default-5cc7bb575-b5shn" Jul 14 18:12:39.190920 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:12:39.190905 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-877bd7-openshift-default-5cc7bb575-b5shn" Jul 14 18:17:31.563713 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:17:31.563683 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2sbww_c3d00b23-d0d5-4d18-a722-464d7b114745/ovn-acl-logging/0.log" Jul 14 18:17:31.567573 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:17:31.567548 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2sbww_c3d00b23-d0d5-4d18-a722-464d7b114745/ovn-acl-logging/0.log" Jul 14 18:18:46.531089 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:18:46.531058 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-aa324e-openshift-default-686754ddc9-zzml5"] Jul 14 18:18:46.531633 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:18:46.531262 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-aa324e-openshift-default-686754ddc9-zzml5" podUID="79d0ddff-f4c2-4e3c-8009-a1fc5bed6354" containerName="istio-proxy" containerID="cri-o://09f618c54d4df37190968306c87a6a51fe3b1eca210b255235c617a7dd3ba36a" gracePeriod=30 Jul 14 18:18:51.772955 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:18:51.772933 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-aa324e-openshift-default-686754ddc9-zzml5" Jul 14 18:18:51.873598 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:18:51.873575 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/79d0ddff-f4c2-4e3c-8009-a1fc5bed6354-istiod-ca-cert\") pod \"79d0ddff-f4c2-4e3c-8009-a1fc5bed6354\" (UID: \"79d0ddff-f4c2-4e3c-8009-a1fc5bed6354\") " Jul 14 18:18:51.873755 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:18:51.873639 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wsfwm\" (UniqueName: \"kubernetes.io/projected/79d0ddff-f4c2-4e3c-8009-a1fc5bed6354-kube-api-access-wsfwm\") pod \"79d0ddff-f4c2-4e3c-8009-a1fc5bed6354\" (UID: \"79d0ddff-f4c2-4e3c-8009-a1fc5bed6354\") " Jul 14 18:18:51.873755 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:18:51.873687 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/79d0ddff-f4c2-4e3c-8009-a1fc5bed6354-istio-data\") pod \"79d0ddff-f4c2-4e3c-8009-a1fc5bed6354\" (UID: \"79d0ddff-f4c2-4e3c-8009-a1fc5bed6354\") " Jul 14 18:18:51.873755 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:18:51.873728 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/79d0ddff-f4c2-4e3c-8009-a1fc5bed6354-istio-podinfo\") pod \"79d0ddff-f4c2-4e3c-8009-a1fc5bed6354\" (UID: \"79d0ddff-f4c2-4e3c-8009-a1fc5bed6354\") " Jul 14 18:18:51.873755 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:18:51.873750 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/79d0ddff-f4c2-4e3c-8009-a1fc5bed6354-workload-socket\") pod \"79d0ddff-f4c2-4e3c-8009-a1fc5bed6354\" (UID: \"79d0ddff-f4c2-4e3c-8009-a1fc5bed6354\") " Jul 14 18:18:51.873982 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:18:51.873776 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/79d0ddff-f4c2-4e3c-8009-a1fc5bed6354-workload-certs\") pod \"79d0ddff-f4c2-4e3c-8009-a1fc5bed6354\" (UID: \"79d0ddff-f4c2-4e3c-8009-a1fc5bed6354\") " Jul 14 18:18:51.873982 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:18:51.873798 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/79d0ddff-f4c2-4e3c-8009-a1fc5bed6354-istio-envoy\") pod \"79d0ddff-f4c2-4e3c-8009-a1fc5bed6354\" (UID: \"79d0ddff-f4c2-4e3c-8009-a1fc5bed6354\") " Jul 14 18:18:51.873982 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:18:51.873900 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/79d0ddff-f4c2-4e3c-8009-a1fc5bed6354-credential-socket\") pod \"79d0ddff-f4c2-4e3c-8009-a1fc5bed6354\" (UID: \"79d0ddff-f4c2-4e3c-8009-a1fc5bed6354\") " Jul 14 18:18:51.874156 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:18:51.873990 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/79d0ddff-f4c2-4e3c-8009-a1fc5bed6354-istio-token\") pod \"79d0ddff-f4c2-4e3c-8009-a1fc5bed6354\" (UID: \"79d0ddff-f4c2-4e3c-8009-a1fc5bed6354\") " Jul 14 18:18:51.874156 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:18:51.874051 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79d0ddff-f4c2-4e3c-8009-a1fc5bed6354-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "79d0ddff-f4c2-4e3c-8009-a1fc5bed6354" (UID: "79d0ddff-f4c2-4e3c-8009-a1fc5bed6354"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 18:18:51.874156 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:18:51.874065 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/79d0ddff-f4c2-4e3c-8009-a1fc5bed6354-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "79d0ddff-f4c2-4e3c-8009-a1fc5bed6354" (UID: "79d0ddff-f4c2-4e3c-8009-a1fc5bed6354"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 18:18:51.874156 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:18:51.874069 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79d0ddff-f4c2-4e3c-8009-a1fc5bed6354-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "79d0ddff-f4c2-4e3c-8009-a1fc5bed6354" (UID: "79d0ddff-f4c2-4e3c-8009-a1fc5bed6354"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 18:18:51.874396 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:18:51.874319 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79d0ddff-f4c2-4e3c-8009-a1fc5bed6354-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "79d0ddff-f4c2-4e3c-8009-a1fc5bed6354" (UID: "79d0ddff-f4c2-4e3c-8009-a1fc5bed6354"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 18:18:51.874396 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:18:51.874351 2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/79d0ddff-f4c2-4e3c-8009-a1fc5bed6354-workload-socket\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 18:18:51.874396 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:18:51.874369 2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/79d0ddff-f4c2-4e3c-8009-a1fc5bed6354-workload-certs\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 18:18:51.874396 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:18:51.874382 2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/79d0ddff-f4c2-4e3c-8009-a1fc5bed6354-istiod-ca-cert\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 18:18:51.874880 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:18:51.874855 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79d0ddff-f4c2-4e3c-8009-a1fc5bed6354-istio-data" (OuterVolumeSpecName: "istio-data") pod "79d0ddff-f4c2-4e3c-8009-a1fc5bed6354" (UID: "79d0ddff-f4c2-4e3c-8009-a1fc5bed6354"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 18:18:51.876105 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:18:51.876073 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/79d0ddff-f4c2-4e3c-8009-a1fc5bed6354-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "79d0ddff-f4c2-4e3c-8009-a1fc5bed6354" (UID: "79d0ddff-f4c2-4e3c-8009-a1fc5bed6354"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 14 18:18:51.876230 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:18:51.876129 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/79d0ddff-f4c2-4e3c-8009-a1fc5bed6354-kube-api-access-wsfwm" (OuterVolumeSpecName: "kube-api-access-wsfwm") pod "79d0ddff-f4c2-4e3c-8009-a1fc5bed6354" (UID: "79d0ddff-f4c2-4e3c-8009-a1fc5bed6354"). InnerVolumeSpecName "kube-api-access-wsfwm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 18:18:51.876535 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:18:51.876517 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79d0ddff-f4c2-4e3c-8009-a1fc5bed6354-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "79d0ddff-f4c2-4e3c-8009-a1fc5bed6354" (UID: "79d0ddff-f4c2-4e3c-8009-a1fc5bed6354"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 18:18:51.876590 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:18:51.876568 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/79d0ddff-f4c2-4e3c-8009-a1fc5bed6354-istio-token" (OuterVolumeSpecName: "istio-token") pod "79d0ddff-f4c2-4e3c-8009-a1fc5bed6354" (UID: "79d0ddff-f4c2-4e3c-8009-a1fc5bed6354"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 18:18:51.975718 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:18:51.975689 2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/79d0ddff-f4c2-4e3c-8009-a1fc5bed6354-istio-token\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 18:18:51.975718 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:18:51.975712 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wsfwm\" (UniqueName: \"kubernetes.io/projected/79d0ddff-f4c2-4e3c-8009-a1fc5bed6354-kube-api-access-wsfwm\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 18:18:51.975718 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:18:51.975722 2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/79d0ddff-f4c2-4e3c-8009-a1fc5bed6354-istio-data\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 18:18:51.975927 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:18:51.975731 2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/79d0ddff-f4c2-4e3c-8009-a1fc5bed6354-istio-podinfo\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 18:18:51.975927 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:18:51.975741 2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/79d0ddff-f4c2-4e3c-8009-a1fc5bed6354-istio-envoy\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 18:18:51.975927 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:18:51.975750 2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/79d0ddff-f4c2-4e3c-8009-a1fc5bed6354-credential-socket\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 18:18:52.529741 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:18:52.529707 2562 generic.go:356] "Generic (PLEG): container finished" podID="79d0ddff-f4c2-4e3c-8009-a1fc5bed6354" containerID="09f618c54d4df37190968306c87a6a51fe3b1eca210b255235c617a7dd3ba36a" exitCode=0 Jul 14 18:18:52.529928 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:18:52.529755 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-aa324e-openshift-default-686754ddc9-zzml5" Jul 14 18:18:52.529928 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:18:52.529787 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-aa324e-openshift-default-686754ddc9-zzml5" event={"ID":"79d0ddff-f4c2-4e3c-8009-a1fc5bed6354","Type":"ContainerDied","Data":"09f618c54d4df37190968306c87a6a51fe3b1eca210b255235c617a7dd3ba36a"} Jul 14 18:18:52.529928 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:18:52.529837 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-aa324e-openshift-default-686754ddc9-zzml5" event={"ID":"79d0ddff-f4c2-4e3c-8009-a1fc5bed6354","Type":"ContainerDied","Data":"3cc4ed7582a209f07201ff849f85895b2296348758426167a19ceccd0d6aaa52"} Jul 14 18:18:52.529928 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:18:52.529859 2562 scope.go:117] "RemoveContainer" containerID="09f618c54d4df37190968306c87a6a51fe3b1eca210b255235c617a7dd3ba36a" Jul 14 18:18:52.538595 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:18:52.538580 2562 scope.go:117] "RemoveContainer" containerID="061a29a5941b074ad346b56a649dfcffb3e22dc707828fbe2368c847490fd6af" Jul 14 18:18:52.546770 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:18:52.546753 2562 scope.go:117] "RemoveContainer" containerID="09f618c54d4df37190968306c87a6a51fe3b1eca210b255235c617a7dd3ba36a" Jul 14 18:18:52.546994 ip-10-0-129-209 kubenswrapper[2562]: E0714 18:18:52.546977 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"09f618c54d4df37190968306c87a6a51fe3b1eca210b255235c617a7dd3ba36a\": container with ID starting with 09f618c54d4df37190968306c87a6a51fe3b1eca210b255235c617a7dd3ba36a not found: ID does not exist" containerID="09f618c54d4df37190968306c87a6a51fe3b1eca210b255235c617a7dd3ba36a" Jul 14 18:18:52.547067 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:18:52.547006 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"09f618c54d4df37190968306c87a6a51fe3b1eca210b255235c617a7dd3ba36a"} err="failed to get container status \"09f618c54d4df37190968306c87a6a51fe3b1eca210b255235c617a7dd3ba36a\": rpc error: code = NotFound desc = could not find container \"09f618c54d4df37190968306c87a6a51fe3b1eca210b255235c617a7dd3ba36a\": container with ID starting with 09f618c54d4df37190968306c87a6a51fe3b1eca210b255235c617a7dd3ba36a not found: ID does not exist" Jul 14 18:18:52.547067 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:18:52.547029 2562 scope.go:117] "RemoveContainer" containerID="061a29a5941b074ad346b56a649dfcffb3e22dc707828fbe2368c847490fd6af" Jul 14 18:18:52.547269 ip-10-0-129-209 kubenswrapper[2562]: E0714 18:18:52.547242 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"061a29a5941b074ad346b56a649dfcffb3e22dc707828fbe2368c847490fd6af\": container with ID starting with 061a29a5941b074ad346b56a649dfcffb3e22dc707828fbe2368c847490fd6af not found: ID does not exist" containerID="061a29a5941b074ad346b56a649dfcffb3e22dc707828fbe2368c847490fd6af" Jul 14 18:18:52.547330 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:18:52.547271 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"061a29a5941b074ad346b56a649dfcffb3e22dc707828fbe2368c847490fd6af"} err="failed to get container status \"061a29a5941b074ad346b56a649dfcffb3e22dc707828fbe2368c847490fd6af\": rpc error: code = NotFound desc = could not find container \"061a29a5941b074ad346b56a649dfcffb3e22dc707828fbe2368c847490fd6af\": container with ID starting with 061a29a5941b074ad346b56a649dfcffb3e22dc707828fbe2368c847490fd6af not found: ID does not exist" Jul 14 18:18:52.551702 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:18:52.551681 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-aa324e-openshift-default-686754ddc9-zzml5"] Jul 14 18:18:52.557550 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:18:52.557531 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-aa324e-openshift-default-686754ddc9-zzml5"] Jul 14 18:18:53.518987 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:18:53.518952 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="79d0ddff-f4c2-4e3c-8009-a1fc5bed6354" path="/var/lib/kubelet/pods/79d0ddff-f4c2-4e3c-8009-a1fc5bed6354/volumes" Jul 14 18:19:12.765118 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:12.765084 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-877bd7-openshift-default-5cc7bb575-b5shn"] Jul 14 18:19:12.765523 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:12.765332 2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-877bd7-openshift-default-5cc7bb575-b5shn" podUID="12201025-f9d5-4fd8-a048-f9d85c278670" containerName="istio-proxy" containerID="cri-o://5868d564303822c71bb4eb769e6d11e33a595adfe899a27e9dccb15701f941b9" gracePeriod=30 Jul 14 18:19:18.010237 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:18.010215 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-877bd7-openshift-default-5cc7bb575-b5shn" Jul 14 18:19:18.105557 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:18.105483 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/12201025-f9d5-4fd8-a048-f9d85c278670-istio-podinfo\") pod \"12201025-f9d5-4fd8-a048-f9d85c278670\" (UID: \"12201025-f9d5-4fd8-a048-f9d85c278670\") " Jul 14 18:19:18.105557 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:18.105519 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n7dsb\" (UniqueName: \"kubernetes.io/projected/12201025-f9d5-4fd8-a048-f9d85c278670-kube-api-access-n7dsb\") pod \"12201025-f9d5-4fd8-a048-f9d85c278670\" (UID: \"12201025-f9d5-4fd8-a048-f9d85c278670\") " Jul 14 18:19:18.105557 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:18.105550 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/12201025-f9d5-4fd8-a048-f9d85c278670-istio-envoy\") pod \"12201025-f9d5-4fd8-a048-f9d85c278670\" (UID: \"12201025-f9d5-4fd8-a048-f9d85c278670\") " Jul 14 18:19:18.105843 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:18.105734 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/12201025-f9d5-4fd8-a048-f9d85c278670-istio-data\") pod \"12201025-f9d5-4fd8-a048-f9d85c278670\" (UID: \"12201025-f9d5-4fd8-a048-f9d85c278670\") " Jul 14 18:19:18.105843 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:18.105804 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/12201025-f9d5-4fd8-a048-f9d85c278670-workload-socket\") pod \"12201025-f9d5-4fd8-a048-f9d85c278670\" (UID: \"12201025-f9d5-4fd8-a048-f9d85c278670\") " Jul 14 18:19:18.105843 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:18.105838 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/12201025-f9d5-4fd8-a048-f9d85c278670-istiod-ca-cert\") pod \"12201025-f9d5-4fd8-a048-f9d85c278670\" (UID: \"12201025-f9d5-4fd8-a048-f9d85c278670\") " Jul 14 18:19:18.105987 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:18.105868 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/12201025-f9d5-4fd8-a048-f9d85c278670-workload-certs\") pod \"12201025-f9d5-4fd8-a048-f9d85c278670\" (UID: \"12201025-f9d5-4fd8-a048-f9d85c278670\") " Jul 14 18:19:18.105987 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:18.105896 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/12201025-f9d5-4fd8-a048-f9d85c278670-credential-socket\") pod \"12201025-f9d5-4fd8-a048-f9d85c278670\" (UID: \"12201025-f9d5-4fd8-a048-f9d85c278670\") " Jul 14 18:19:18.105987 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:18.105919 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/12201025-f9d5-4fd8-a048-f9d85c278670-istio-token\") pod \"12201025-f9d5-4fd8-a048-f9d85c278670\" (UID: \"12201025-f9d5-4fd8-a048-f9d85c278670\") " Jul 14 18:19:18.106104 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:18.106066 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12201025-f9d5-4fd8-a048-f9d85c278670-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "12201025-f9d5-4fd8-a048-f9d85c278670" (UID: "12201025-f9d5-4fd8-a048-f9d85c278670"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 18:19:18.106145 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:18.106113 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12201025-f9d5-4fd8-a048-f9d85c278670-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "12201025-f9d5-4fd8-a048-f9d85c278670" (UID: "12201025-f9d5-4fd8-a048-f9d85c278670"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 18:19:18.106344 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:18.106325 2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/12201025-f9d5-4fd8-a048-f9d85c278670-workload-certs\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 18:19:18.106389 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:18.106352 2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/12201025-f9d5-4fd8-a048-f9d85c278670-workload-socket\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 18:19:18.106389 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:18.106325 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12201025-f9d5-4fd8-a048-f9d85c278670-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "12201025-f9d5-4fd8-a048-f9d85c278670" (UID: "12201025-f9d5-4fd8-a048-f9d85c278670"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 18:19:18.106853 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:18.106705 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/12201025-f9d5-4fd8-a048-f9d85c278670-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "12201025-f9d5-4fd8-a048-f9d85c278670" (UID: "12201025-f9d5-4fd8-a048-f9d85c278670"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 14 18:19:18.106956 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:18.106910 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12201025-f9d5-4fd8-a048-f9d85c278670-istio-data" (OuterVolumeSpecName: "istio-data") pod "12201025-f9d5-4fd8-a048-f9d85c278670" (UID: "12201025-f9d5-4fd8-a048-f9d85c278670"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 18:19:18.107942 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:18.107912 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/12201025-f9d5-4fd8-a048-f9d85c278670-kube-api-access-n7dsb" (OuterVolumeSpecName: "kube-api-access-n7dsb") pod "12201025-f9d5-4fd8-a048-f9d85c278670" (UID: "12201025-f9d5-4fd8-a048-f9d85c278670"). InnerVolumeSpecName "kube-api-access-n7dsb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 18:19:18.108284 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:18.108252 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12201025-f9d5-4fd8-a048-f9d85c278670-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "12201025-f9d5-4fd8-a048-f9d85c278670" (UID: "12201025-f9d5-4fd8-a048-f9d85c278670"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 14 18:19:18.108367 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:18.108311 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/12201025-f9d5-4fd8-a048-f9d85c278670-istio-token" (OuterVolumeSpecName: "istio-token") pod "12201025-f9d5-4fd8-a048-f9d85c278670" (UID: "12201025-f9d5-4fd8-a048-f9d85c278670"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 14 18:19:18.108367 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:18.108327 2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/12201025-f9d5-4fd8-a048-f9d85c278670-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "12201025-f9d5-4fd8-a048-f9d85c278670" (UID: "12201025-f9d5-4fd8-a048-f9d85c278670"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 14 18:19:18.206921 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:18.206898 2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/12201025-f9d5-4fd8-a048-f9d85c278670-istiod-ca-cert\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 18:19:18.206921 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:18.206919 2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/12201025-f9d5-4fd8-a048-f9d85c278670-credential-socket\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 18:19:18.207052 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:18.206929 2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/12201025-f9d5-4fd8-a048-f9d85c278670-istio-token\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 18:19:18.207052 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:18.206937 2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/12201025-f9d5-4fd8-a048-f9d85c278670-istio-podinfo\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 18:19:18.207052 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:18.206945 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n7dsb\" (UniqueName: \"kubernetes.io/projected/12201025-f9d5-4fd8-a048-f9d85c278670-kube-api-access-n7dsb\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 18:19:18.207052 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:18.206954 2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/12201025-f9d5-4fd8-a048-f9d85c278670-istio-envoy\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 18:19:18.207052 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:18.206961 2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/12201025-f9d5-4fd8-a048-f9d85c278670-istio-data\") on node \"ip-10-0-129-209.ec2.internal\" DevicePath \"\"" Jul 14 18:19:18.638360 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:18.638329 2562 generic.go:356] "Generic (PLEG): container finished" podID="12201025-f9d5-4fd8-a048-f9d85c278670" containerID="5868d564303822c71bb4eb769e6d11e33a595adfe899a27e9dccb15701f941b9" exitCode=0 Jul 14 18:19:18.638531 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:18.638374 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-877bd7-openshift-default-5cc7bb575-b5shn" Jul 14 18:19:18.638531 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:18.638373 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-877bd7-openshift-default-5cc7bb575-b5shn" event={"ID":"12201025-f9d5-4fd8-a048-f9d85c278670","Type":"ContainerDied","Data":"5868d564303822c71bb4eb769e6d11e33a595adfe899a27e9dccb15701f941b9"} Jul 14 18:19:18.638531 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:18.638500 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-877bd7-openshift-default-5cc7bb575-b5shn" event={"ID":"12201025-f9d5-4fd8-a048-f9d85c278670","Type":"ContainerDied","Data":"3ba6faee352d87a253cf9b352ae14b2f718fe49e31af292c5cda53e12ab821cc"} Jul 14 18:19:18.638531 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:18.638531 2562 scope.go:117] "RemoveContainer" containerID="5868d564303822c71bb4eb769e6d11e33a595adfe899a27e9dccb15701f941b9" Jul 14 18:19:18.647327 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:18.647311 2562 scope.go:117] "RemoveContainer" containerID="e815999abab790f1558dbcd0b774a2d510ccc0cc6eb4c434f52930b84111d915" Jul 14 18:19:18.655398 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:18.655377 2562 scope.go:117] "RemoveContainer" containerID="5868d564303822c71bb4eb769e6d11e33a595adfe899a27e9dccb15701f941b9" Jul 14 18:19:18.655672 ip-10-0-129-209 kubenswrapper[2562]: E0714 18:19:18.655652 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5868d564303822c71bb4eb769e6d11e33a595adfe899a27e9dccb15701f941b9\": container with ID starting with 5868d564303822c71bb4eb769e6d11e33a595adfe899a27e9dccb15701f941b9 not found: ID does not exist" containerID="5868d564303822c71bb4eb769e6d11e33a595adfe899a27e9dccb15701f941b9" Jul 14 18:19:18.655758 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:18.655677 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5868d564303822c71bb4eb769e6d11e33a595adfe899a27e9dccb15701f941b9"} err="failed to get container status \"5868d564303822c71bb4eb769e6d11e33a595adfe899a27e9dccb15701f941b9\": rpc error: code = NotFound desc = could not find container \"5868d564303822c71bb4eb769e6d11e33a595adfe899a27e9dccb15701f941b9\": container with ID starting with 5868d564303822c71bb4eb769e6d11e33a595adfe899a27e9dccb15701f941b9 not found: ID does not exist" Jul 14 18:19:18.655758 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:18.655697 2562 scope.go:117] "RemoveContainer" containerID="e815999abab790f1558dbcd0b774a2d510ccc0cc6eb4c434f52930b84111d915" Jul 14 18:19:18.655978 ip-10-0-129-209 kubenswrapper[2562]: E0714 18:19:18.655955 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e815999abab790f1558dbcd0b774a2d510ccc0cc6eb4c434f52930b84111d915\": container with ID starting with e815999abab790f1558dbcd0b774a2d510ccc0cc6eb4c434f52930b84111d915 not found: ID does not exist" containerID="e815999abab790f1558dbcd0b774a2d510ccc0cc6eb4c434f52930b84111d915" Jul 14 18:19:18.656050 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:18.655984 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e815999abab790f1558dbcd0b774a2d510ccc0cc6eb4c434f52930b84111d915"} err="failed to get container status \"e815999abab790f1558dbcd0b774a2d510ccc0cc6eb4c434f52930b84111d915\": rpc error: code = NotFound desc = could not find container \"e815999abab790f1558dbcd0b774a2d510ccc0cc6eb4c434f52930b84111d915\": container with ID starting with e815999abab790f1558dbcd0b774a2d510ccc0cc6eb4c434f52930b84111d915 not found: ID does not exist" Jul 14 18:19:18.664471 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:18.664447 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-877bd7-openshift-default-5cc7bb575-b5shn"] Jul 14 18:19:18.669316 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:18.669288 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-877bd7-openshift-default-5cc7bb575-b5shn"] Jul 14 18:19:18.937250 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:18.937149 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7558c48c44-gnp72_3f0ef414-c4bb-492c-98d8-9bc1f07a9b6f/copy-manifests/0.log" Jul 14 18:19:18.944742 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:18.944722 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7558c48c44-gnp72_3f0ef414-c4bb-492c-98d8-9bc1f07a9b6f/manager/0.log" Jul 14 18:19:19.519620 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:19.519589 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="12201025-f9d5-4fd8-a048-f9d85c278670" path="/var/lib/kubelet/pods/12201025-f9d5-4fd8-a048-f9d85c278670/volumes" Jul 14 18:19:23.865588 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:23.865560 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf46rt48_1a4c56d5-9623-4732-9bbd-d6d539fe8049/istio-proxy/0.log" Jul 14 18:19:23.991076 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:23.991047 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-ln9vg_e6e12fb7-7384-4f6d-9f71-8fb407254355/discovery/0.log" Jul 14 18:19:24.101724 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:24.101696 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-fd75767d5-gwn8q_ece6bedb-d95d-4355-8108-b4dba7886c3b/kube-auth-proxy/0.log" Jul 14 18:19:35.892357 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:35.892311 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-kv2bk_ac89d28e-0c57-4be5-8c3a-f0482ce0ba6f/global-pull-secret-syncer/0.log" Jul 14 18:19:36.070606 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:36.070580 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-hxbnp_650b148f-78d8-456f-bc24-1fbe4a75a24a/konnectivity-agent/0.log" Jul 14 18:19:36.133488 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:36.133463 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-129-209.ec2.internal_8b1a7c81fd72faef90d5a3084bb532e1/haproxy/0.log" Jul 14 18:19:43.417996 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:43.417944 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6f004740-868c-4ad2-9c9c-bf0d903bfa87/alertmanager/0.log" Jul 14 18:19:43.465830 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:43.465765 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6f004740-868c-4ad2-9c9c-bf0d903bfa87/config-reloader/0.log" Jul 14 18:19:43.509533 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:43.509510 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6f004740-868c-4ad2-9c9c-bf0d903bfa87/kube-rbac-proxy-web/0.log" Jul 14 18:19:43.559568 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:43.559547 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6f004740-868c-4ad2-9c9c-bf0d903bfa87/kube-rbac-proxy/0.log" Jul 14 18:19:43.611543 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:43.611519 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6f004740-868c-4ad2-9c9c-bf0d903bfa87/kube-rbac-proxy-metric/0.log" Jul 14 18:19:43.661707 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:43.661683 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6f004740-868c-4ad2-9c9c-bf0d903bfa87/prom-label-proxy/0.log" Jul 14 18:19:43.707750 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:43.707734 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_6f004740-868c-4ad2-9c9c-bf0d903bfa87/init-config-reloader/0.log" Jul 14 18:19:43.929327 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:43.929293 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-69dd5d8d98-25vlb_15eb5778-e2da-43d6-b724-dde07774086a/metrics-server/0.log" Jul 14 18:19:44.114590 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:44.114532 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-bwhfp_ba05a4f0-94e4-4462-9bc4-7d0d7cc8e230/node-exporter/0.log" Jul 14 18:19:44.154637 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:44.154613 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-bwhfp_ba05a4f0-94e4-4462-9bc4-7d0d7cc8e230/kube-rbac-proxy/0.log" Jul 14 18:19:44.198644 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:44.198594 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-bwhfp_ba05a4f0-94e4-4462-9bc4-7d0d7cc8e230/init-textfile/0.log" Jul 14 18:19:44.485512 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:44.485427 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_4a8f645b-43a7-4f0b-a290-3d4c92ae2499/prometheus/0.log" Jul 14 18:19:44.514883 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:44.514821 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_4a8f645b-43a7-4f0b-a290-3d4c92ae2499/config-reloader/0.log" Jul 14 18:19:44.546951 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:44.546925 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_4a8f645b-43a7-4f0b-a290-3d4c92ae2499/thanos-sidecar/0.log" Jul 14 18:19:44.574119 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:44.574095 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_4a8f645b-43a7-4f0b-a290-3d4c92ae2499/kube-rbac-proxy-web/0.log" Jul 14 18:19:44.599923 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:44.599890 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_4a8f645b-43a7-4f0b-a290-3d4c92ae2499/kube-rbac-proxy/0.log" Jul 14 18:19:44.630398 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:44.630361 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_4a8f645b-43a7-4f0b-a290-3d4c92ae2499/kube-rbac-proxy-thanos/0.log" Jul 14 18:19:44.660745 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:44.660704 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_4a8f645b-43a7-4f0b-a290-3d4c92ae2499/init-config-reloader/0.log" Jul 14 18:19:44.784729 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:44.784662 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6fd6d747b5-qcc65_2bcb63de-5908-4bdf-9c18-86937b58b55d/telemeter-client/0.log" Jul 14 18:19:44.821105 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:44.821085 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6fd6d747b5-qcc65_2bcb63de-5908-4bdf-9c18-86937b58b55d/reload/0.log" Jul 14 18:19:44.846434 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:44.846418 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6fd6d747b5-qcc65_2bcb63de-5908-4bdf-9c18-86937b58b55d/kube-rbac-proxy/0.log" Jul 14 18:19:44.878076 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:44.878045 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-57d5c4cc6b-4j54g_c20e734f-caf7-4ac3-a4f2-4297dfe34ec1/thanos-query/0.log" Jul 14 18:19:44.911286 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:44.911267 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-57d5c4cc6b-4j54g_c20e734f-caf7-4ac3-a4f2-4297dfe34ec1/kube-rbac-proxy-web/0.log" Jul 14 18:19:44.937388 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:44.937360 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-57d5c4cc6b-4j54g_c20e734f-caf7-4ac3-a4f2-4297dfe34ec1/kube-rbac-proxy/0.log" Jul 14 18:19:44.971514 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:44.971487 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-57d5c4cc6b-4j54g_c20e734f-caf7-4ac3-a4f2-4297dfe34ec1/prom-label-proxy/0.log" Jul 14 18:19:44.995182 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:44.995153 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-57d5c4cc6b-4j54g_c20e734f-caf7-4ac3-a4f2-4297dfe34ec1/kube-rbac-proxy-rules/0.log" Jul 14 18:19:45.023659 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:45.023641 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-57d5c4cc6b-4j54g_c20e734f-caf7-4ac3-a4f2-4297dfe34ec1/kube-rbac-proxy-metrics/0.log" Jul 14 18:19:45.567148 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:45.567113 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-g2pvf/perf-node-gather-daemonset-tcddz"] Jul 14 18:19:45.567713 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:45.567601 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ad039bb-dff9-4919-83bb-0a5813fa01b4" containerName="istio-proxy" Jul 14 18:19:45.567713 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:45.567622 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ad039bb-dff9-4919-83bb-0a5813fa01b4" containerName="istio-proxy" Jul 14 18:19:45.567713 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:45.567635 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="79d0ddff-f4c2-4e3c-8009-a1fc5bed6354" containerName="istio-proxy" Jul 14 18:19:45.567713 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:45.567643 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="79d0ddff-f4c2-4e3c-8009-a1fc5bed6354" containerName="istio-proxy" Jul 14 18:19:45.567713 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:45.567654 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="12201025-f9d5-4fd8-a048-f9d85c278670" containerName="istio-proxy" Jul 14 18:19:45.567713 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:45.567662 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="12201025-f9d5-4fd8-a048-f9d85c278670" containerName="istio-proxy" Jul 14 18:19:45.567713 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:45.567682 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="12201025-f9d5-4fd8-a048-f9d85c278670" containerName="istio-proxy" Jul 14 18:19:45.567713 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:45.567690 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="12201025-f9d5-4fd8-a048-f9d85c278670" containerName="istio-proxy" Jul 14 18:19:45.568174 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:45.567777 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="12201025-f9d5-4fd8-a048-f9d85c278670" containerName="istio-proxy" Jul 14 18:19:45.568174 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:45.567790 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="79d0ddff-f4c2-4e3c-8009-a1fc5bed6354" containerName="istio-proxy" Jul 14 18:19:45.568174 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:45.567801 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="79d0ddff-f4c2-4e3c-8009-a1fc5bed6354" containerName="istio-proxy" Jul 14 18:19:45.569863 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:45.569841 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-g2pvf/perf-node-gather-daemonset-tcddz" Jul 14 18:19:45.572643 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:45.572625 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-g2pvf\"/\"default-dockercfg-4w9h4\"" Jul 14 18:19:45.572643 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:45.572638 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-g2pvf\"/\"openshift-service-ca.crt\"" Jul 14 18:19:45.573157 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:45.573139 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-g2pvf\"/\"kube-root-ca.crt\"" Jul 14 18:19:45.591656 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:45.591635 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-g2pvf/perf-node-gather-daemonset-tcddz"] Jul 14 18:19:45.734459 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:45.734427 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6a7b8965-8be4-4080-9a57-04095c710ce0-podres\") pod \"perf-node-gather-daemonset-tcddz\" (UID: \"6a7b8965-8be4-4080-9a57-04095c710ce0\") " pod="openshift-must-gather-g2pvf/perf-node-gather-daemonset-tcddz" Jul 14 18:19:45.734619 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:45.734475 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6a7b8965-8be4-4080-9a57-04095c710ce0-proc\") pod \"perf-node-gather-daemonset-tcddz\" (UID: \"6a7b8965-8be4-4080-9a57-04095c710ce0\") " pod="openshift-must-gather-g2pvf/perf-node-gather-daemonset-tcddz" Jul 14 18:19:45.734619 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:45.734565 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6a7b8965-8be4-4080-9a57-04095c710ce0-lib-modules\") pod \"perf-node-gather-daemonset-tcddz\" (UID: \"6a7b8965-8be4-4080-9a57-04095c710ce0\") " pod="openshift-must-gather-g2pvf/perf-node-gather-daemonset-tcddz" Jul 14 18:19:45.734619 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:45.734607 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cs2s4\" (UniqueName: \"kubernetes.io/projected/6a7b8965-8be4-4080-9a57-04095c710ce0-kube-api-access-cs2s4\") pod \"perf-node-gather-daemonset-tcddz\" (UID: \"6a7b8965-8be4-4080-9a57-04095c710ce0\") " pod="openshift-must-gather-g2pvf/perf-node-gather-daemonset-tcddz" Jul 14 18:19:45.734727 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:45.734639 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6a7b8965-8be4-4080-9a57-04095c710ce0-sys\") pod \"perf-node-gather-daemonset-tcddz\" (UID: \"6a7b8965-8be4-4080-9a57-04095c710ce0\") " pod="openshift-must-gather-g2pvf/perf-node-gather-daemonset-tcddz" Jul 14 18:19:45.835469 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:45.835389 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6a7b8965-8be4-4080-9a57-04095c710ce0-sys\") pod \"perf-node-gather-daemonset-tcddz\" (UID: \"6a7b8965-8be4-4080-9a57-04095c710ce0\") " pod="openshift-must-gather-g2pvf/perf-node-gather-daemonset-tcddz" Jul 14 18:19:45.835469 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:45.835432 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6a7b8965-8be4-4080-9a57-04095c710ce0-podres\") pod \"perf-node-gather-daemonset-tcddz\" (UID: \"6a7b8965-8be4-4080-9a57-04095c710ce0\") " pod="openshift-must-gather-g2pvf/perf-node-gather-daemonset-tcddz" Jul 14 18:19:45.835469 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:45.835472 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6a7b8965-8be4-4080-9a57-04095c710ce0-proc\") pod \"perf-node-gather-daemonset-tcddz\" (UID: \"6a7b8965-8be4-4080-9a57-04095c710ce0\") " pod="openshift-must-gather-g2pvf/perf-node-gather-daemonset-tcddz" Jul 14 18:19:45.835675 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:45.835505 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6a7b8965-8be4-4080-9a57-04095c710ce0-sys\") pod \"perf-node-gather-daemonset-tcddz\" (UID: \"6a7b8965-8be4-4080-9a57-04095c710ce0\") " pod="openshift-must-gather-g2pvf/perf-node-gather-daemonset-tcddz" Jul 14 18:19:45.835675 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:45.835532 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6a7b8965-8be4-4080-9a57-04095c710ce0-lib-modules\") pod \"perf-node-gather-daemonset-tcddz\" (UID: \"6a7b8965-8be4-4080-9a57-04095c710ce0\") " pod="openshift-must-gather-g2pvf/perf-node-gather-daemonset-tcddz" Jul 14 18:19:45.835675 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:45.835551 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6a7b8965-8be4-4080-9a57-04095c710ce0-proc\") pod \"perf-node-gather-daemonset-tcddz\" (UID: \"6a7b8965-8be4-4080-9a57-04095c710ce0\") " pod="openshift-must-gather-g2pvf/perf-node-gather-daemonset-tcddz" Jul 14 18:19:45.835675 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:45.835568 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cs2s4\" (UniqueName: \"kubernetes.io/projected/6a7b8965-8be4-4080-9a57-04095c710ce0-kube-api-access-cs2s4\") pod \"perf-node-gather-daemonset-tcddz\" (UID: \"6a7b8965-8be4-4080-9a57-04095c710ce0\") " pod="openshift-must-gather-g2pvf/perf-node-gather-daemonset-tcddz" Jul 14 18:19:45.835675 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:45.835581 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6a7b8965-8be4-4080-9a57-04095c710ce0-podres\") pod \"perf-node-gather-daemonset-tcddz\" (UID: \"6a7b8965-8be4-4080-9a57-04095c710ce0\") " pod="openshift-must-gather-g2pvf/perf-node-gather-daemonset-tcddz" Jul 14 18:19:45.835828 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:45.835687 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6a7b8965-8be4-4080-9a57-04095c710ce0-lib-modules\") pod \"perf-node-gather-daemonset-tcddz\" (UID: \"6a7b8965-8be4-4080-9a57-04095c710ce0\") " pod="openshift-must-gather-g2pvf/perf-node-gather-daemonset-tcddz" Jul 14 18:19:45.845664 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:45.845632 2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cs2s4\" (UniqueName: \"kubernetes.io/projected/6a7b8965-8be4-4080-9a57-04095c710ce0-kube-api-access-cs2s4\") pod \"perf-node-gather-daemonset-tcddz\" (UID: \"6a7b8965-8be4-4080-9a57-04095c710ce0\") " pod="openshift-must-gather-g2pvf/perf-node-gather-daemonset-tcddz" Jul 14 18:19:45.879685 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:45.879666 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-g2pvf/perf-node-gather-daemonset-tcddz" Jul 14 18:19:46.009934 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:46.009912 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-g2pvf/perf-node-gather-daemonset-tcddz"] Jul 14 18:19:46.012204 ip-10-0-129-209 kubenswrapper[2562]: W0714 18:19:46.012164 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod6a7b8965_8be4_4080_9a57_04095c710ce0.slice/crio-becca28dd4bcebde4ee47700832f23256287d9ee1306d492acadb0f204573233 WatchSource:0}: Error finding container becca28dd4bcebde4ee47700832f23256287d9ee1306d492acadb0f204573233: Status 404 returned error can't find the container with id becca28dd4bcebde4ee47700832f23256287d9ee1306d492acadb0f204573233 Jul 14 18:19:46.755909 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:46.755873 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-g2pvf/perf-node-gather-daemonset-tcddz" event={"ID":"6a7b8965-8be4-4080-9a57-04095c710ce0","Type":"ContainerStarted","Data":"c62644081ac9c0b433ccdf7997da9e546a4d5c1eb27b912ddb6cd3a7cdbb6e4d"} Jul 14 18:19:46.756335 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:46.755923 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-g2pvf/perf-node-gather-daemonset-tcddz" event={"ID":"6a7b8965-8be4-4080-9a57-04095c710ce0","Type":"ContainerStarted","Data":"becca28dd4bcebde4ee47700832f23256287d9ee1306d492acadb0f204573233"} Jul 14 18:19:46.756335 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:46.756004 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-g2pvf/perf-node-gather-daemonset-tcddz" Jul 14 18:19:46.775088 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:46.775040 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-g2pvf/perf-node-gather-daemonset-tcddz" podStartSLOduration=1.77502833 podStartE2EDuration="1.77502833s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-14 18:19:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-14 18:19:46.773361065 +0000 UTC m=+2835.849818627" watchObservedRunningTime="2026-07-14 18:19:46.77502833 +0000 UTC m=+2835.851485869" Jul 14 18:19:49.300544 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:49.300517 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-wjkk8_95b1cf03-081d-49cb-a74d-9c6e5e165e7d/dns/0.log" Jul 14 18:19:49.331324 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:49.331299 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-wjkk8_95b1cf03-081d-49cb-a74d-9c6e5e165e7d/kube-rbac-proxy/0.log" Jul 14 18:19:49.391066 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:49.391049 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-7fdn8_91ee1d15-3741-4cf5-821e-d3b216b7e246/dns-node-resolver/0.log" Jul 14 18:19:50.136256 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:50.136223 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-thnpp_17370624-81e7-42ff-b916-e3f323b3e0a5/node-ca/0.log" Jul 14 18:19:51.221231 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:51.221198 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf46rt48_1a4c56d5-9623-4732-9bbd-d6d539fe8049/istio-proxy/0.log" Jul 14 18:19:51.350695 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:51.350666 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-ln9vg_e6e12fb7-7384-4f6d-9f71-8fb407254355/discovery/0.log" Jul 14 18:19:51.378134 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:51.378104 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-fd75767d5-gwn8q_ece6bedb-d95d-4355-8108-b4dba7886c3b/kube-auth-proxy/0.log" Jul 14 18:19:52.354152 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:52.354124 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-z4vr6_ebe882b9-ebc1-4be5-acba-06f13a445bd2/serve-healthcheck-canary/0.log" Jul 14 18:19:52.770393 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:52.770364 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-g2pvf/perf-node-gather-daemonset-tcddz" Jul 14 18:19:52.890068 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:52.890044 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-8r6jg_7635e71d-e195-4ef9-8eff-980969d9f360/kube-rbac-proxy/0.log" Jul 14 18:19:52.914817 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:52.914798 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-8r6jg_7635e71d-e195-4ef9-8eff-980969d9f360/exporter/0.log" Jul 14 18:19:52.948612 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:52.948588 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-8r6jg_7635e71d-e195-4ef9-8eff-980969d9f360/extractor/0.log" Jul 14 18:19:56.290936 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:56.290894 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7558c48c44-gnp72_3f0ef414-c4bb-492c-98d8-9bc1f07a9b6f/manager/0.log" Jul 14 18:19:56.326760 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:56.326735 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7558c48c44-gnp72_3f0ef414-c4bb-492c-98d8-9bc1f07a9b6f/copy-manifests/0.log" Jul 14 18:19:56.371291 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:56.371266 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf46rt48_1a4c56d5-9623-4732-9bbd-d6d539fe8049/istio-proxy/0.log" Jul 14 18:19:56.499451 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:56.499427 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-ln9vg_e6e12fb7-7384-4f6d-9f71-8fb407254355/discovery/0.log" Jul 14 18:19:56.514779 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:56.514755 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-fd75767d5-gwn8q_ece6bedb-d95d-4355-8108-b4dba7886c3b/kube-auth-proxy/0.log" Jul 14 18:19:57.890141 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:57.890110 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-8448dcd647-77wvx_d1441010-f67e-4b0a-b81e-ef383d4d05a8/manager/0.log" Jul 14 18:19:57.918051 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:57.918026 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf46rt48_1a4c56d5-9623-4732-9bbd-d6d539fe8049/istio-proxy/0.log" Jul 14 18:19:58.032973 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:58.032945 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-ln9vg_e6e12fb7-7384-4f6d-9f71-8fb407254355/discovery/0.log" Jul 14 18:19:58.049377 ip-10-0-129-209 kubenswrapper[2562]: I0714 18:19:58.049355 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-fd75767d5-gwn8q_ece6bedb-d95d-4355-8108-b4dba7886c3b/kube-auth-proxy/0.log"