Jul 06 22:08:40.309323 ip-10-0-137-101 systemd[1]: Starting Kubernetes Kubelet... Jul 06 22:08:40.819695 ip-10-0-137-101 kubenswrapper[2581]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 06 22:08:40.819695 ip-10-0-137-101 kubenswrapper[2581]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 06 22:08:40.819695 ip-10-0-137-101 kubenswrapper[2581]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 06 22:08:40.819695 ip-10-0-137-101 kubenswrapper[2581]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 06 22:08:40.819695 ip-10-0-137-101 kubenswrapper[2581]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 06 22:08:40.819695 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.763566 2581 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 06 22:08:40.819695 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.767835 2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 22:08:40.819695 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.767847 2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 22:08:40.819695 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.767851 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 22:08:40.819695 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.767854 2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 22:08:40.819695 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.767857 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 22:08:40.819695 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.767861 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 22:08:40.819695 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.767864 2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 22:08:40.819695 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.767867 2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 22:08:41.110202 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.767870 2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 22:08:41.110202 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.767873 2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 22:08:41.110202 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.767875 2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 22:08:41.110202 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.767878 2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 22:08:41.110202 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.767881 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 22:08:41.110202 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.767883 2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 22:08:41.110202 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.767886 2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 22:08:41.110202 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.767889 2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 22:08:41.110202 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.767891 2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 22:08:41.110202 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.767894 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 22:08:41.110202 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.767896 2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 22:08:41.110202 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.767899 2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 22:08:41.110202 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.767902 2581 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 22:08:41.110202 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.767906 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 22:08:41.110202 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.767908 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 22:08:41.110202 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.767911 2581 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 22:08:41.110202 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.767914 2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 22:08:41.110202 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.767917 2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 22:08:41.110202 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.767920 2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 22:08:41.110202 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.767924 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 22:08:41.122158 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.767926 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 22:08:41.122158 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.767929 2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 22:08:41.122158 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.767932 2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 22:08:41.122158 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.767934 2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 22:08:41.122158 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.767937 2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 22:08:41.122158 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.767939 2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 22:08:41.122158 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.767942 2581 feature_gate.go:328] unrecognized feature gate: Example Jul 06 22:08:41.122158 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.767945 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 22:08:41.122158 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.767947 2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 22:08:41.122158 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.767950 2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 22:08:41.122158 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.767953 2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 22:08:41.122158 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.767956 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 22:08:41.122158 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.767958 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 22:08:41.122158 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.767961 2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 22:08:41.122158 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.767963 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 22:08:41.122158 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.767966 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 22:08:41.122158 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.767968 2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 22:08:41.122158 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.767971 2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 22:08:41.122158 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.767973 2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 22:08:40.847802 ip-10-0-137-101 systemd[1]: Started Kubernetes Kubelet. Jul 06 22:08:41.123354 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.767976 2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 22:08:41.123354 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.767978 2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 22:08:41.123354 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.767981 2581 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 22:08:41.123354 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.767985 2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 22:08:41.123354 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.767989 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 22:08:41.123354 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.767992 2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 22:08:41.123354 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.767995 2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 22:08:41.123354 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.767998 2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 22:08:41.123354 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.768001 2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 22:08:41.123354 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.768005 2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 22:08:41.123354 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.768008 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 22:08:41.123354 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.768010 2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 22:08:41.123354 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.768014 2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 22:08:41.123354 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.768018 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 22:08:41.123354 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.768021 2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 22:08:41.123354 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.768023 2581 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 22:08:41.123354 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.768026 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 22:08:41.123354 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.768028 2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 22:08:41.123354 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.768031 2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 22:08:41.124499 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.768033 2581 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 22:08:41.124499 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.768036 2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 22:08:41.124499 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.768039 2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 22:08:41.124499 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.768041 2581 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 22:08:41.124499 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.768044 2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 22:08:41.124499 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.768047 2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 22:08:41.124499 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.768051 2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 22:08:41.124499 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.768053 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 22:08:41.124499 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.768057 2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 22:08:41.124499 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.768059 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 22:08:41.124499 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.768082 2581 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 22:08:41.124499 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.768087 2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 22:08:41.124499 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.768090 2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 22:08:41.124499 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.768093 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 22:08:41.124499 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.768095 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 22:08:41.124499 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.768098 2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 22:08:41.124499 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.768101 2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 22:08:41.124499 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.768103 2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 22:08:41.124499 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.768105 2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 22:08:41.124499 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.768108 2581 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 22:08:41.124499 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.768111 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 22:08:41.125337 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.768113 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 22:08:41.125337 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.768116 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 22:08:41.125337 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.768119 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 22:08:41.125337 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.768121 2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 22:08:41.125337 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.768125 2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 22:08:41.125337 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.768128 2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 22:08:41.125337 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.768131 2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 22:08:41.125337 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.768133 2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 22:08:41.125337 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.768136 2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 22:08:41.125337 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.768140 2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 22:08:41.125337 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.768143 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 22:08:41.125337 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.768146 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 22:08:41.125337 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.768876 2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 22:08:41.125337 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.768883 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 22:08:41.125337 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.768887 2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 22:08:41.125337 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.768890 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 22:08:41.125337 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.768892 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 22:08:41.125337 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.768896 2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 22:08:41.125337 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.768898 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 22:08:41.125987 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.768901 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 22:08:41.125987 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.768904 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 22:08:41.125987 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.768906 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 22:08:41.125987 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.768909 2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 22:08:41.125987 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.768912 2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 22:08:41.125987 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.768914 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 22:08:41.125987 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.768917 2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 22:08:41.125987 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.768920 2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 22:08:41.125987 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.768922 2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 22:08:41.125987 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.768925 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 22:08:41.125987 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.768928 2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 22:08:41.125987 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.768931 2581 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 22:08:41.125987 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.768933 2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 22:08:41.125987 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.768936 2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 22:08:41.125987 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.768938 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 22:08:41.125987 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.768941 2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 22:08:41.125987 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.768944 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 22:08:41.125987 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.768947 2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 22:08:41.125987 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.768950 2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 22:08:41.125987 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.768952 2581 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 22:08:41.125987 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.768955 2581 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 22:08:41.126812 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.768958 2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 22:08:41.126812 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.768960 2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 22:08:41.126812 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.768963 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 22:08:41.126812 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.768965 2581 feature_gate.go:328] unrecognized feature gate: Example Jul 06 22:08:41.126812 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.768968 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 22:08:41.126812 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.768971 2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 22:08:41.126812 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.768977 2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 22:08:41.126812 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.768979 2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 22:08:41.126812 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.768982 2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 22:08:41.126812 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.768985 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 22:08:41.126812 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.768988 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 22:08:41.126812 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.768991 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 22:08:41.126812 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.768994 2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 22:08:41.126812 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.768997 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 22:08:41.126812 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769000 2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 22:08:41.126812 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769002 2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 22:08:41.126812 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769005 2581 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 22:08:41.126812 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769007 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 22:08:41.126812 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769010 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 22:08:41.126812 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769013 2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 22:08:41.127661 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769015 2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 22:08:41.127661 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769017 2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 22:08:41.127661 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769020 2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 22:08:41.127661 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769023 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 22:08:41.127661 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769026 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 22:08:41.127661 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769029 2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 22:08:41.127661 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769031 2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 22:08:41.127661 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769034 2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 22:08:41.127661 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769036 2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 22:08:41.127661 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769039 2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 22:08:41.127661 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769041 2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 22:08:41.127661 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769044 2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 22:08:41.127661 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769047 2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 22:08:41.127661 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769049 2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 22:08:41.127661 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769052 2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 22:08:41.127661 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769056 2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 22:08:41.127661 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769059 2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 22:08:41.127661 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769080 2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 22:08:41.127661 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769086 2581 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 22:08:41.128280 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769089 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 22:08:41.128280 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769092 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 22:08:41.128280 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769095 2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 22:08:41.128280 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769098 2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 22:08:41.128280 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769101 2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 22:08:41.128280 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769104 2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 22:08:41.128280 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769107 2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 22:08:41.128280 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769109 2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 22:08:41.128280 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769112 2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 22:08:41.128280 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769114 2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 22:08:41.128280 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769117 2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 22:08:41.128280 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769119 2581 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 22:08:41.128280 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769123 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 22:08:41.128280 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769128 2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 22:08:41.128280 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769131 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 22:08:41.128280 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769133 2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 22:08:41.128280 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769136 2581 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 22:08:41.128280 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769138 2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 22:08:41.128280 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769141 2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 22:08:41.128280 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769143 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 22:08:41.128920 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769146 2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 22:08:41.128920 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769148 2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 22:08:41.128920 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769151 2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 22:08:41.128920 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769153 2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 22:08:41.128920 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769156 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 22:08:41.128920 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769158 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 22:08:41.128920 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769161 2581 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 22:08:41.128920 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769163 2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 22:08:41.128920 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769166 2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 22:08:41.128920 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769168 2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 22:08:41.128920 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769171 2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 22:08:41.128920 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769176 2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 22:08:41.128920 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769546 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 22:08:41.128920 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769551 2581 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 22:08:41.128920 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769554 2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 22:08:41.128920 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769557 2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 22:08:41.128920 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769560 2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 22:08:41.128920 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769563 2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 22:08:41.128920 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769566 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 22:08:41.129551 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769569 2581 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 22:08:41.129551 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769573 2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 22:08:41.129551 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769576 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 22:08:41.129551 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769579 2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 22:08:41.129551 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769581 2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 22:08:41.129551 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769585 2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 22:08:41.129551 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769587 2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 22:08:41.129551 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769590 2581 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 22:08:41.129551 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769595 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 22:08:41.129551 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769597 2581 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 22:08:41.129551 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769600 2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 22:08:41.129551 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769602 2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 22:08:41.129551 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769605 2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 22:08:41.129551 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769608 2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 22:08:41.129551 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769610 2581 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 22:08:41.129551 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769613 2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 22:08:41.129551 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769615 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 22:08:41.129551 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769618 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 22:08:41.129551 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769621 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 22:08:41.129551 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769623 2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 22:08:41.130215 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769626 2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 22:08:41.130215 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769629 2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 22:08:41.130215 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769631 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 22:08:41.130215 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769634 2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 22:08:41.130215 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769637 2581 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 22:08:41.130215 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769640 2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 22:08:41.130215 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769643 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 22:08:41.130215 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769645 2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 22:08:41.130215 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769648 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 22:08:41.130215 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769651 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 22:08:41.130215 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769654 2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 22:08:41.130215 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769656 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 22:08:41.130215 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769659 2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 22:08:41.130215 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769661 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 22:08:41.130215 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769664 2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 22:08:41.130215 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769666 2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 22:08:41.130215 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769669 2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 22:08:41.130215 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769671 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 22:08:41.130215 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769674 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 22:08:41.130215 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769676 2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 22:08:41.130847 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769680 2581 feature_gate.go:328] unrecognized feature gate: Example Jul 06 22:08:41.130847 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769682 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 22:08:41.130847 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769685 2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 22:08:41.130847 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769687 2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 22:08:41.130847 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769690 2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 22:08:41.130847 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769692 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 22:08:41.130847 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769695 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 22:08:41.130847 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769697 2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 22:08:41.130847 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769700 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 22:08:41.130847 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769703 2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 22:08:41.130847 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769706 2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 22:08:41.130847 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769708 2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 22:08:41.130847 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769711 2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 22:08:41.130847 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769713 2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 22:08:41.130847 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769716 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 22:08:41.130847 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769718 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 22:08:41.130847 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769722 2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 22:08:41.130847 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769725 2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 22:08:41.130847 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769729 2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 22:08:41.131495 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769732 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 22:08:41.131495 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769735 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 22:08:41.131495 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769738 2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 22:08:41.131495 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769741 2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 22:08:41.131495 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769744 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 22:08:41.131495 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769748 2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 22:08:41.131495 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769750 2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 22:08:41.131495 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769753 2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 22:08:41.131495 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769756 2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 22:08:41.131495 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769759 2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 22:08:41.131495 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769761 2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 22:08:41.131495 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769764 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 22:08:41.131495 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769766 2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 22:08:41.131495 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769770 2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 22:08:41.131495 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769772 2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 22:08:41.131495 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769774 2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 22:08:41.131495 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769777 2581 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 22:08:41.131495 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769779 2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 22:08:41.131495 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769782 2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 22:08:41.131495 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769784 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 22:08:41.132354 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769787 2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 22:08:41.132354 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769789 2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 22:08:41.132354 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769792 2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 22:08:41.132354 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769794 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 22:08:41.132354 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769797 2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 22:08:41.132354 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769801 2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 22:08:41.132354 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769804 2581 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 22:08:41.132354 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769808 2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 22:08:41.132354 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769812 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 22:08:41.132354 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769816 2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 22:08:41.132354 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769819 2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 22:08:41.132354 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769822 2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 22:08:41.132354 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.769824 2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 22:08:41.132354 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.769894 2581 flags.go:64] FLAG: --address="0.0.0.0" Jul 06 22:08:41.132354 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.769903 2581 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 06 22:08:41.132354 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.769912 2581 flags.go:64] FLAG: --anonymous-auth="true" Jul 06 22:08:41.132354 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.769926 2581 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 06 22:08:41.132354 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.769932 2581 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 06 22:08:41.132354 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.769936 2581 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 06 22:08:41.132354 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.769939 2581 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 06 22:08:41.132997 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.769943 2581 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 06 22:08:41.132997 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.769947 2581 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 06 22:08:41.132997 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.769950 2581 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 06 22:08:41.132997 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.769954 2581 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 06 22:08:41.132997 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.769957 2581 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 06 22:08:41.132997 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.769961 2581 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 06 22:08:41.132997 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.769964 2581 flags.go:64] FLAG: --cgroup-root="" Jul 06 22:08:41.132997 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.769967 2581 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 06 22:08:41.132997 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.769970 2581 flags.go:64] FLAG: --client-ca-file="" Jul 06 22:08:41.132997 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.769973 2581 flags.go:64] FLAG: --cloud-provider="external" Jul 06 22:08:41.132997 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.769976 2581 flags.go:64] FLAG: --cluster-dns="[]" Jul 06 22:08:41.132997 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.769979 2581 flags.go:64] FLAG: --cluster-domain="" Jul 06 22:08:41.132997 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.769982 2581 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 06 22:08:41.132997 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.769985 2581 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 06 22:08:41.132997 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.769989 2581 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 06 22:08:41.132997 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.769992 2581 flags.go:64] FLAG: --container-log-max-files="5" Jul 06 22:08:41.132997 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.769995 2581 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 06 22:08:41.132997 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.769998 2581 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 06 22:08:41.132997 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.770001 2581 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 06 22:08:41.132997 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.770005 2581 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 06 22:08:41.132997 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.770008 2581 flags.go:64] FLAG: --contention-profiling="false" Jul 06 22:08:41.132997 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.770012 2581 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 06 22:08:41.132997 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.770015 2581 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 06 22:08:41.132997 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.770018 2581 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 06 22:08:41.133793 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.770021 2581 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 06 22:08:41.133793 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.770025 2581 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 06 22:08:41.133793 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.770028 2581 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 06 22:08:41.133793 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.770031 2581 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 06 22:08:41.133793 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.770034 2581 flags.go:64] FLAG: --enable-load-reader="false" Jul 06 22:08:41.133793 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.770037 2581 flags.go:64] FLAG: --enable-server="true" Jul 06 22:08:41.133793 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.770040 2581 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 06 22:08:41.133793 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.770045 2581 flags.go:64] FLAG: --event-burst="100" Jul 06 22:08:41.133793 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.770048 2581 flags.go:64] FLAG: --event-qps="50" Jul 06 22:08:41.133793 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.770051 2581 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 06 22:08:41.133793 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.770054 2581 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 06 22:08:41.133793 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.770057 2581 flags.go:64] FLAG: --eviction-hard="" Jul 06 22:08:41.133793 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.770061 2581 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 06 22:08:41.133793 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.770085 2581 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 06 22:08:41.133793 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.770090 2581 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 06 22:08:41.133793 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.770093 2581 flags.go:64] FLAG: --eviction-soft="" Jul 06 22:08:41.133793 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.770096 2581 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 06 22:08:41.133793 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.770099 2581 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 06 22:08:41.133793 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.770102 2581 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 06 22:08:41.133793 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.770105 2581 flags.go:64] FLAG: --experimental-mounter-path="" Jul 06 22:08:41.133793 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.770108 2581 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 06 22:08:41.133793 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.770111 2581 flags.go:64] FLAG: --fail-swap-on="true" Jul 06 22:08:41.133793 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.770113 2581 flags.go:64] FLAG: --feature-gates="" Jul 06 22:08:41.133793 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.770117 2581 flags.go:64] FLAG: --file-check-frequency="20s" Jul 06 22:08:41.133793 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.770120 2581 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 06 22:08:41.134602 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.770123 2581 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 06 22:08:41.134602 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.770127 2581 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 06 22:08:41.134602 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.770130 2581 flags.go:64] FLAG: --healthz-port="10248" Jul 06 22:08:41.134602 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.770133 2581 flags.go:64] FLAG: --help="false" Jul 06 22:08:41.134602 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.770137 2581 flags.go:64] FLAG: --hostname-override="ip-10-0-137-101.ec2.internal" Jul 06 22:08:41.134602 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.770140 2581 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 06 22:08:41.134602 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.770143 2581 flags.go:64] FLAG: --http-check-frequency="20s" Jul 06 22:08:41.134602 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.770145 2581 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 06 22:08:41.134602 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.770149 2581 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 06 22:08:41.134602 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.770152 2581 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 06 22:08:41.134602 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.770155 2581 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 06 22:08:41.134602 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.770157 2581 flags.go:64] FLAG: --image-service-endpoint="" Jul 06 22:08:41.134602 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.770160 2581 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 06 22:08:41.134602 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.770162 2581 flags.go:64] FLAG: --kube-api-burst="100" Jul 06 22:08:41.134602 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.770166 2581 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 06 22:08:41.134602 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.770169 2581 flags.go:64] FLAG: --kube-api-qps="50" Jul 06 22:08:41.134602 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.770172 2581 flags.go:64] FLAG: --kube-reserved="" Jul 06 22:08:41.134602 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.770175 2581 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 06 22:08:41.134602 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.770178 2581 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 06 22:08:41.134602 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.770181 2581 flags.go:64] FLAG: --kubelet-cgroups="" Jul 06 22:08:41.134602 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.770184 2581 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 06 22:08:41.134602 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.770187 2581 flags.go:64] FLAG: --lock-file="" Jul 06 22:08:41.134602 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.770190 2581 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 06 22:08:41.134602 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.770193 2581 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 06 22:08:41.135246 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.770196 2581 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 06 22:08:41.135246 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.770201 2581 flags.go:64] FLAG: --log-json-split-stream="false" Jul 06 22:08:41.135246 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.770204 2581 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 06 22:08:41.135246 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.770207 2581 flags.go:64] FLAG: --log-text-split-stream="false" Jul 06 22:08:41.135246 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.770209 2581 flags.go:64] FLAG: --logging-format="text" Jul 06 22:08:41.135246 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.770212 2581 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 06 22:08:41.135246 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.770215 2581 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 06 22:08:41.135246 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.770218 2581 flags.go:64] FLAG: --manifest-url="" Jul 06 22:08:41.135246 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.770221 2581 flags.go:64] FLAG: --manifest-url-header="" Jul 06 22:08:41.135246 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.770228 2581 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 06 22:08:41.135246 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.770231 2581 flags.go:64] FLAG: --max-open-files="1000000" Jul 06 22:08:41.135246 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.770236 2581 flags.go:64] FLAG: --max-pods="110" Jul 06 22:08:41.135246 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.770240 2581 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 06 22:08:41.135246 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.770243 2581 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 06 22:08:41.135246 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.770246 2581 flags.go:64] FLAG: --memory-manager-policy="None" Jul 06 22:08:41.135246 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.770249 2581 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 06 22:08:41.135246 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.770251 2581 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 06 22:08:41.135246 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.770254 2581 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 06 22:08:41.135246 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.770257 2581 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 06 22:08:41.135246 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.770265 2581 flags.go:64] FLAG: --node-status-max-images="50" Jul 06 22:08:41.135246 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.770268 2581 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 06 22:08:41.135246 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.770272 2581 flags.go:64] FLAG: --oom-score-adj="-999" Jul 06 22:08:41.135246 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.770274 2581 flags.go:64] FLAG: --pod-cidr="" Jul 06 22:08:41.135891 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.770278 2581 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 06 22:08:41.135891 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.770283 2581 flags.go:64] FLAG: --pod-manifest-path="" Jul 06 22:08:41.135891 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.770286 2581 flags.go:64] FLAG: --pod-max-pids="-1" Jul 06 22:08:41.135891 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.770289 2581 flags.go:64] FLAG: --pods-per-core="0" Jul 06 22:08:41.135891 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.770292 2581 flags.go:64] FLAG: --port="10250" Jul 06 22:08:41.135891 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.770297 2581 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 06 22:08:41.135891 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.770299 2581 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0dbda52bb1a00e6b1" Jul 06 22:08:41.135891 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.770303 2581 flags.go:64] FLAG: --qos-reserved="" Jul 06 22:08:41.135891 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.770306 2581 flags.go:64] FLAG: --read-only-port="10255" Jul 06 22:08:41.135891 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.770308 2581 flags.go:64] FLAG: --register-node="true" Jul 06 22:08:41.135891 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.770311 2581 flags.go:64] FLAG: --register-with-taints="" Jul 06 22:08:41.135891 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.770315 2581 flags.go:64] FLAG: --registry-burst="10" Jul 06 22:08:41.135891 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.770318 2581 flags.go:64] FLAG: --registry-qps="5" Jul 06 22:08:41.135891 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.770321 2581 flags.go:64] FLAG: --reserved-cpus="" Jul 06 22:08:41.135891 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.770323 2581 flags.go:64] FLAG: --reserved-memory="" Jul 06 22:08:41.135891 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.770327 2581 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 06 22:08:41.135891 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.770330 2581 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 06 22:08:41.135891 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.770333 2581 flags.go:64] FLAG: --rotate-certificates="false" Jul 06 22:08:41.135891 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.770335 2581 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 06 22:08:41.135891 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.770338 2581 flags.go:64] FLAG: --runonce="false" Jul 06 22:08:41.135891 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.770341 2581 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 06 22:08:41.135891 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.770344 2581 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 06 22:08:41.135891 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.770348 2581 flags.go:64] FLAG: --seccomp-default="false" Jul 06 22:08:41.135891 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.770351 2581 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 06 22:08:41.135891 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.770353 2581 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 06 22:08:41.136868 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.770356 2581 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 06 22:08:41.136868 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.770359 2581 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 06 22:08:41.136868 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.770362 2581 flags.go:64] FLAG: --storage-driver-password="root" Jul 06 22:08:41.136868 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.770365 2581 flags.go:64] FLAG: --storage-driver-secure="false" Jul 06 22:08:41.136868 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.770368 2581 flags.go:64] FLAG: --storage-driver-table="stats" Jul 06 22:08:41.136868 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.770371 2581 flags.go:64] FLAG: --storage-driver-user="root" Jul 06 22:08:41.136868 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.770373 2581 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 06 22:08:41.136868 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.770376 2581 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 06 22:08:41.136868 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.770380 2581 flags.go:64] FLAG: --system-cgroups="" Jul 06 22:08:41.136868 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.770382 2581 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 06 22:08:41.136868 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.770387 2581 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 06 22:08:41.136868 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.770390 2581 flags.go:64] FLAG: --tls-cert-file="" Jul 06 22:08:41.136868 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.770394 2581 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 06 22:08:41.136868 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.770398 2581 flags.go:64] FLAG: --tls-min-version="" Jul 06 22:08:41.136868 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.770400 2581 flags.go:64] FLAG: --tls-private-key-file="" Jul 06 22:08:41.136868 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.770403 2581 flags.go:64] FLAG: --topology-manager-policy="none" Jul 06 22:08:41.136868 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.770406 2581 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 06 22:08:41.136868 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.770409 2581 flags.go:64] FLAG: --topology-manager-scope="container" Jul 06 22:08:41.136868 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.770412 2581 flags.go:64] FLAG: --v="2" Jul 06 22:08:41.136868 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.770415 2581 flags.go:64] FLAG: --version="false" Jul 06 22:08:41.136868 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.770420 2581 flags.go:64] FLAG: --vmodule="" Jul 06 22:08:41.136868 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.770423 2581 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 06 22:08:41.136868 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.770427 2581 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 06 22:08:41.136868 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.770542 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 22:08:41.136868 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.770545 2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 22:08:41.137578 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.770548 2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 22:08:41.137578 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.770551 2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 22:08:41.137578 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.770554 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 22:08:41.137578 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.770557 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 22:08:41.137578 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.770561 2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 22:08:41.137578 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.770564 2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 22:08:41.137578 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.770566 2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 22:08:41.137578 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.770569 2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 22:08:41.137578 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.770572 2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 22:08:41.137578 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.770575 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 22:08:41.137578 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.770578 2581 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 22:08:41.137578 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.770580 2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 22:08:41.137578 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.770583 2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 22:08:41.137578 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.770585 2581 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 22:08:41.137578 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.770588 2581 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 22:08:41.137578 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.770590 2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 22:08:41.137578 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.770593 2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 22:08:41.137578 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.770596 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 22:08:41.137578 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.770599 2581 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 22:08:41.137578 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.770602 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 22:08:41.247142 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.770605 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 22:08:41.247142 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.770608 2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 22:08:41.247142 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.770610 2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 22:08:41.247142 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.770613 2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 22:08:41.247142 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.770615 2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 22:08:41.247142 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.770619 2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 22:08:41.247142 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.770623 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 22:08:41.247142 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.770626 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 22:08:41.247142 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.770629 2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 22:08:41.247142 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.770631 2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 22:08:41.247142 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.770634 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 22:08:41.247142 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.770637 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 22:08:41.247142 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.770639 2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 22:08:41.247142 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.770642 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 22:08:41.247142 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.770644 2581 feature_gate.go:328] unrecognized feature gate: Example Jul 06 22:08:41.247142 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.770647 2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 22:08:41.247142 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.770650 2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 22:08:41.247142 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.770653 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 22:08:41.247142 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.770655 2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 22:08:41.247945 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.770658 2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 22:08:41.247945 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.770661 2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 22:08:41.247945 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.770663 2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 22:08:41.247945 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.770666 2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 22:08:41.247945 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.770669 2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 22:08:41.247945 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.770673 2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 22:08:41.247945 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.770677 2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 22:08:41.247945 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.770680 2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 22:08:41.247945 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.770683 2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 22:08:41.247945 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.770686 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 22:08:41.247945 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.770689 2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 22:08:41.247945 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.770691 2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 22:08:41.247945 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.770695 2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 22:08:41.247945 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.770698 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 22:08:41.247945 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.770701 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 22:08:41.247945 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.770703 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 22:08:41.247945 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.770706 2581 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 22:08:41.247945 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.770708 2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 22:08:41.247945 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.770711 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 22:08:41.248749 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.770713 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 22:08:41.248749 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.770715 2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 22:08:41.248749 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.770718 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 22:08:41.248749 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.770720 2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 22:08:41.248749 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.770723 2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 22:08:41.248749 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.770725 2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 22:08:41.248749 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.770728 2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 22:08:41.248749 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.770730 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 22:08:41.248749 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.770733 2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 22:08:41.248749 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.770735 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 22:08:41.248749 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.770740 2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 22:08:41.248749 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.770742 2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 22:08:41.248749 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.770745 2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 22:08:41.248749 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.770747 2581 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 22:08:41.248749 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.770750 2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 22:08:41.248749 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.770752 2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 22:08:41.248749 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.770755 2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 22:08:41.248749 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.770757 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 22:08:41.248749 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.770760 2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 22:08:41.249544 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.770763 2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 22:08:41.249544 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.770765 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 22:08:41.249544 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.770768 2581 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 22:08:41.249544 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.770771 2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 22:08:41.249544 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.770774 2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 22:08:41.249544 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.770776 2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 22:08:41.249544 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.770780 2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 22:08:41.249544 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.770784 2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 22:08:41.249544 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.770787 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 22:08:41.249544 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.770790 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 22:08:41.249544 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.770792 2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 22:08:41.249544 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.770795 2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 22:08:41.249544 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.770798 2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 22:08:41.249544 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.770801 2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 22:08:41.249544 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.770803 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 22:08:41.249544 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.770806 2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 22:08:41.249544 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.770809 2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 22:08:41.249544 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.770811 2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 22:08:41.249544 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.770814 2581 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 22:08:41.249544 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.770816 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 22:08:41.250361 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.771584 2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 06 22:08:41.250361 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.780117 2581 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 06 22:08:41.250361 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.780133 2581 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 06 22:08:41.250361 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.780153 2581 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 06 22:08:41.250361 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.780162 2581 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 06 22:08:41.250361 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780219 2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 22:08:41.250361 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780224 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 22:08:41.250361 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780227 2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 22:08:41.250361 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780231 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 22:08:41.250361 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780234 2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 22:08:41.250361 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780236 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 22:08:41.250361 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780239 2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 22:08:41.250361 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780242 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 22:08:41.250361 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780248 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 22:08:41.250361 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780251 2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 22:08:41.251550 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780254 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 22:08:41.251550 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780256 2581 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 22:08:41.251550 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780259 2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 22:08:41.251550 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780262 2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 22:08:41.251550 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780265 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 22:08:41.251550 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780267 2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 22:08:41.251550 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780270 2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 22:08:41.251550 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780272 2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 22:08:41.251550 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780275 2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 22:08:41.251550 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780278 2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 22:08:41.251550 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780280 2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 22:08:41.251550 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780283 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 22:08:41.251550 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780285 2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 22:08:41.251550 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780288 2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 22:08:41.251550 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780290 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 22:08:41.251550 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780293 2581 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 22:08:41.251550 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780295 2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 22:08:41.251550 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780299 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 22:08:41.251550 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780301 2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 22:08:41.251550 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780304 2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 22:08:41.252354 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780306 2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 22:08:41.252354 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780309 2581 feature_gate.go:328] unrecognized feature gate: Example Jul 06 22:08:41.252354 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780312 2581 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 22:08:41.252354 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780314 2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 22:08:41.252354 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780317 2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 22:08:41.252354 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780319 2581 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 22:08:41.252354 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780322 2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 22:08:41.252354 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780326 2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 22:08:41.252354 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780329 2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 22:08:41.252354 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780331 2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 22:08:41.252354 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780335 2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 22:08:41.252354 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780338 2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 22:08:41.252354 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780340 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 22:08:41.252354 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780343 2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 22:08:41.252354 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780346 2581 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 22:08:41.252354 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780349 2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 22:08:41.252354 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780352 2581 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 22:08:41.252354 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780354 2581 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 22:08:41.252354 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780357 2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 22:08:41.252354 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780360 2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 22:08:41.253550 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780362 2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 22:08:41.253550 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780366 2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 22:08:41.253550 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780371 2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 22:08:41.253550 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780375 2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 22:08:41.253550 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780377 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 22:08:41.253550 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780380 2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 22:08:41.253550 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780383 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 22:08:41.253550 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780385 2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 22:08:41.253550 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780388 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 22:08:41.253550 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780391 2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 22:08:41.253550 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780393 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 22:08:41.253550 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780396 2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 22:08:41.253550 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780398 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 22:08:41.253550 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780401 2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 22:08:41.253550 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780403 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 22:08:41.253550 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780406 2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 22:08:41.253550 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780408 2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 22:08:41.253550 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780411 2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 22:08:41.253550 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780414 2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 22:08:41.254367 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780418 2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 22:08:41.254367 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780421 2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 22:08:41.254367 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780424 2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 22:08:41.254367 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780428 2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 22:08:41.254367 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780431 2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 22:08:41.254367 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780435 2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 22:08:41.254367 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780438 2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 22:08:41.254367 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780441 2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 22:08:41.254367 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780443 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 22:08:41.254367 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780446 2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 22:08:41.254367 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780449 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 22:08:41.254367 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780452 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 22:08:41.254367 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780455 2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 22:08:41.254367 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780457 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 22:08:41.254367 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780460 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 22:08:41.254367 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780462 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 22:08:41.254367 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780465 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 22:08:41.254367 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780468 2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 22:08:41.254367 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780470 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 22:08:41.255545 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780473 2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 22:08:41.255545 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780475 2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 22:08:41.255545 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780478 2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 22:08:41.255545 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780480 2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 22:08:41.255545 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780483 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 22:08:41.255545 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780486 2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 22:08:41.255545 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780488 2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 22:08:41.255545 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780491 2581 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 22:08:41.255545 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780493 2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 22:08:41.255545 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780496 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 22:08:41.255545 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780512 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 22:08:41.255545 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.780517 2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 06 22:08:41.255545 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780627 2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 22:08:41.255545 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780632 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 22:08:41.255545 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780638 2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 22:08:41.256354 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780641 2581 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 22:08:41.256354 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780646 2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 22:08:41.256354 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780650 2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 22:08:41.256354 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780654 2581 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 22:08:41.256354 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780656 2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 22:08:41.256354 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780659 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 22:08:41.256354 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780661 2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 22:08:41.256354 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780664 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 22:08:41.256354 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780667 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 22:08:41.256354 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780670 2581 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 22:08:41.256354 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780672 2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 22:08:41.256354 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780675 2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 22:08:41.256354 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780677 2581 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 22:08:41.256354 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780680 2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 22:08:41.256354 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780682 2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 22:08:41.256354 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780685 2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 22:08:41.256354 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780687 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 22:08:41.256354 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780690 2581 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 22:08:41.256354 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780692 2581 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 22:08:41.256354 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780695 2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 22:08:41.257129 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780697 2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 22:08:41.257129 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780701 2581 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 22:08:41.257129 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780704 2581 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 22:08:41.257129 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780707 2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 22:08:41.257129 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780710 2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 22:08:41.257129 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780713 2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 22:08:41.257129 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780716 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 22:08:41.257129 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780719 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 22:08:41.257129 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780722 2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 22:08:41.257129 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780725 2581 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 22:08:41.257129 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780727 2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 22:08:41.257129 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780731 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 22:08:41.257129 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780734 2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 22:08:41.257129 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780737 2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 22:08:41.257129 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780740 2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 22:08:41.257129 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780743 2581 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 22:08:41.257129 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780745 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 22:08:41.257129 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780748 2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 22:08:41.257129 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780750 2581 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 22:08:41.257932 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780753 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 22:08:41.257932 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780755 2581 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 22:08:41.257932 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780758 2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 22:08:41.257932 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780761 2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 22:08:41.257932 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780764 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 22:08:41.257932 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780767 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 22:08:41.257932 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780769 2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 22:08:41.257932 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780772 2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 22:08:41.257932 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780774 2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 22:08:41.257932 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780776 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 22:08:41.257932 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780779 2581 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 22:08:41.257932 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780782 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 22:08:41.257932 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780784 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 22:08:41.257932 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780787 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 22:08:41.257932 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780789 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 22:08:41.257932 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780792 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 22:08:41.257932 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780794 2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 22:08:41.257932 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780797 2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 22:08:41.257932 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780799 2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 22:08:41.257932 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780802 2581 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 22:08:41.258742 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780804 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 22:08:41.258742 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780807 2581 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 22:08:41.258742 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780809 2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 22:08:41.258742 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780812 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 22:08:41.258742 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780816 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 22:08:41.258742 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780818 2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 22:08:41.258742 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780822 2581 feature_gate.go:328] unrecognized feature gate: Example Jul 06 22:08:41.258742 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780825 2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 22:08:41.258742 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780827 2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 22:08:41.258742 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780830 2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 22:08:41.258742 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780833 2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 22:08:41.258742 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780835 2581 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 22:08:41.258742 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780838 2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 22:08:41.258742 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780840 2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 22:08:41.258742 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780843 2581 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 22:08:41.258742 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780845 2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 22:08:41.258742 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780848 2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 22:08:41.258742 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780851 2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 22:08:41.258742 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780853 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 22:08:41.259535 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780856 2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 22:08:41.259535 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780859 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 22:08:41.259535 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780861 2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 22:08:41.259535 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780864 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 22:08:41.259535 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780867 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 22:08:41.259535 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780870 2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 22:08:41.259535 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780872 2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 22:08:41.259535 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780875 2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 22:08:41.259535 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780877 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 22:08:41.259535 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780880 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 22:08:41.259535 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780883 2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 22:08:41.259535 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780885 2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 22:08:41.259535 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780888 2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 22:08:41.259535 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780890 2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 22:08:41.259535 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780893 2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 22:08:41.259535 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780895 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 22:08:41.259535 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780897 2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 22:08:41.259535 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:40.780901 2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 22:08:41.260343 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.780906 2581 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 06 22:08:41.260343 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.781008 2581 server.go:962] "Client rotation is on, will bootstrap in background" Jul 06 22:08:41.260343 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.783500 2581 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 06 22:08:41.260343 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.785222 2581 server.go:1019] "Starting client certificate rotation" Jul 06 22:08:41.260343 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.785323 2581 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 06 22:08:41.260343 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.785354 2581 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 06 22:08:41.260343 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.787978 2581 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 06 22:08:41.260343 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.793709 2581 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 06 22:08:41.260343 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.805850 2581 log.go:25] "Validated CRI v1 runtime API" Jul 06 22:08:41.260343 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.808779 2581 log.go:25] "Validated CRI v1 image API" Jul 06 22:08:41.260343 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.809585 2581 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 06 22:08:41.260343 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.812253 2581 fs.go:135] Filesystem UUIDs: map[2e9e5efc-00e7-48bb-be8c-f6480516ffcf:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 b259b232-39a8-4008-b07c-e2d378d73d38:/dev/nvme0n1p3] Jul 06 22:08:41.260762 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.812275 2581 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 06 22:08:41.260762 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.816038 2581 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 06 22:08:41.261998 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.818946 2581 manager.go:216] Machine: {Timestamp:2026-07-06 22:08:40.817545687 +0000 UTC m=+0.376340999 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3018391 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec29067864ac3ae6ad0546e0cb822591 SystemUUID:ec290678-64ac-3ae6-ad05-46e0cb822591 BootID:fa7796f9-a950-43a6-8169-ef4bb984c3d6 Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:b5:1b:f5:bf:fb Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:b5:1b:f5:bf:fb Speed:0 Mtu:9001} {Name:ovs-system MacAddress:26:8e:a2:6e:d6:ed Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 06 22:08:41.261998 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.819044 2581 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 06 22:08:41.261998 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.819136 2581 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 06 22:08:41.261998 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.823592 2581 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 06 22:08:41.261998 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.823778 2581 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-137-101.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 06 22:08:41.261998 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.825468 2581 topology_manager.go:138] "Creating topology manager with none policy" Jul 06 22:08:41.261998 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.825480 2581 container_manager_linux.go:309] "Creating device plugin manager" Jul 06 22:08:41.261998 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.825488 2581 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 06 22:08:41.261998 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.826237 2581 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 06 22:08:41.261998 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.829313 2581 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 06 22:08:41.261998 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.834258 2581 state_mem.go:36] "Initialized new in-memory state store" Jul 06 22:08:41.261998 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.834381 2581 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 06 22:08:41.261998 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.834409 2581 kubelet.go:486] "Attempting to sync node with API server" Jul 06 22:08:41.261998 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.834425 2581 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 06 22:08:41.261998 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.834439 2581 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 06 22:08:41.261998 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.834448 2581 kubelet.go:398] "Adding apiserver pod source" Jul 06 22:08:41.261998 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.834456 2581 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 06 22:08:41.261998 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.834796 2581 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 06 22:08:41.262805 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.834812 2581 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 06 22:08:41.262805 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.837177 2581 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 06 22:08:41.262805 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.839536 2581 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 06 22:08:41.262805 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.839564 2581 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 06 22:08:41.262805 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.840937 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 06 22:08:41.262805 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.840950 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 06 22:08:41.262805 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.840956 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 06 22:08:41.262805 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.840961 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 06 22:08:41.262805 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.840970 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 06 22:08:41.262805 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.840975 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 06 22:08:41.262805 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.840983 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 06 22:08:41.262805 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.840988 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 06 22:08:41.262805 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.840995 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 06 22:08:41.262805 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.841000 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 06 22:08:41.262805 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.841009 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 06 22:08:41.262805 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.841014 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 06 22:08:41.262805 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.841873 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 06 22:08:41.262805 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.841880 2581 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 06 22:08:41.262805 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:08:40.844972 2581 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-137-101.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 06 22:08:41.262805 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:08:40.844976 2581 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 06 22:08:41.262805 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.845091 2581 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-137-101.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 06 22:08:41.262805 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.846962 2581 server.go:1268] "Started kubelet" Jul 06 22:08:41.262805 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.847033 2581 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 06 22:08:41.262805 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.847038 2581 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 06 22:08:41.262805 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.847143 2581 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 06 22:08:41.262805 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.849252 2581 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 06 22:08:41.262805 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.849829 2581 server.go:310] "Adding debug handlers to kubelet server" Jul 06 22:08:41.262805 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:08:40.853647 2581 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 06 22:08:41.262805 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:08:40.853306 2581 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-101.ec2.internal.18bfd2c4f6233178 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-101.ec2.internal,UID:ip-10-0-137-101.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-137-101.ec2.internal,},FirstTimestamp:2026-07-06 22:08:40.846930296 +0000 UTC m=+0.405725607,LastTimestamp:2026-07-06 22:08:40.846930296 +0000 UTC m=+0.405725607,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-101.ec2.internal,}" Jul 06 22:08:41.262805 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.857350 2581 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 06 22:08:41.262805 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.857356 2581 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 06 22:08:41.262805 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:08:40.857462 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-101.ec2.internal\" not found" Jul 06 22:08:41.262805 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.858103 2581 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 06 22:08:41.262805 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.858122 2581 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 06 22:08:41.262805 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.858103 2581 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 06 22:08:41.262805 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.858275 2581 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 06 22:08:41.262805 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.858291 2581 factory.go:55] Registering systemd factory Jul 06 22:08:41.262805 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.858302 2581 factory.go:223] Registration of the systemd container factory successfully Jul 06 22:08:41.262805 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.858308 2581 reconstruct.go:97] "Volume reconstruction finished" Jul 06 22:08:41.262805 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.858314 2581 reconciler.go:29] "Reconciler: start to sync state" Jul 06 22:08:41.268080 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.858487 2581 factory.go:153] Registering CRI-O factory Jul 06 22:08:41.268080 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.858502 2581 factory.go:223] Registration of the crio container factory successfully Jul 06 22:08:41.268080 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.858523 2581 factory.go:103] Registering Raw factory Jul 06 22:08:41.268080 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.858537 2581 manager.go:1196] Started watching for new ooms in manager Jul 06 22:08:41.268080 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.859001 2581 manager.go:349] Starting recovery of all containers Jul 06 22:08:41.268080 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:08:40.859402 2581 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-137-101.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 06 22:08:41.268080 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:08:40.860157 2581 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 06 22:08:41.268080 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.865559 2581 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-2gjld" Jul 06 22:08:41.268080 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.870689 2581 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-2gjld" Jul 06 22:08:41.268080 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.871342 2581 manager.go:354] Recovery completed Jul 06 22:08:41.268080 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.876776 2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 22:08:41.268080 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.878829 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-101.ec2.internal" event="NodeHasSufficientMemory" Jul 06 22:08:41.268080 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.878852 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-101.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 22:08:41.268080 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.878863 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-101.ec2.internal" event="NodeHasSufficientPID" Jul 06 22:08:41.268080 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.879454 2581 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 06 22:08:41.268080 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.879464 2581 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 06 22:08:41.268080 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.879478 2581 state_mem.go:36] "Initialized new in-memory state store" Jul 06 22:08:41.268080 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.879509 2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 22:08:41.268080 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.880125 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-101.ec2.internal" event="NodeHasSufficientMemory" Jul 06 22:08:41.268080 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.880150 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-101.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 22:08:41.268080 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.880165 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-101.ec2.internal" event="NodeHasSufficientPID" Jul 06 22:08:41.268080 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.881508 2581 policy_none.go:49] "None policy: Start" Jul 06 22:08:41.268080 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.881526 2581 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 06 22:08:41.268080 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.881536 2581 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 06 22:08:41.268080 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.883433 2581 policy_none.go:47] "Start" Jul 06 22:08:41.268080 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:08:40.957770 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-101.ec2.internal\" not found" Jul 06 22:08:41.268080 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.964658 2581 manager.go:337] "Starting Device Plugin manager" Jul 06 22:08:41.268080 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:08:40.964701 2581 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 06 22:08:41.268080 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.964713 2581 server.go:87] "Starting device plugin registration server" Jul 06 22:08:41.268080 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.964956 2581 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 06 22:08:41.268080 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.964968 2581 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 06 22:08:41.268080 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.965137 2581 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 06 22:08:41.268080 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.965213 2581 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 06 22:08:41.268080 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.965220 2581 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 06 22:08:41.268080 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:08:40.965679 2581 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 06 22:08:41.268080 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:08:40.965718 2581 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-137-101.ec2.internal\" not found" Jul 06 22:08:41.268080 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.968225 2581 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 06 22:08:41.268080 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.969467 2581 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 06 22:08:41.268080 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.969503 2581 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 06 22:08:41.268080 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.969522 2581 kubelet.go:2448] "Starting kubelet main sync loop" Jul 06 22:08:41.268080 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:08:40.969556 2581 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 06 22:08:41.268080 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:40.971895 2581 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 22:08:41.268080 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:41.065616 2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 22:08:41.268080 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:08:41.066329 2581 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-137-101.ec2.internal\" not found" node="ip-10-0-137-101.ec2.internal" Jul 06 22:08:41.268080 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:41.066577 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-101.ec2.internal" event="NodeHasSufficientMemory" Jul 06 22:08:41.268080 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:41.066606 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-101.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 22:08:41.269716 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:41.066622 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-101.ec2.internal" event="NodeHasSufficientPID" Jul 06 22:08:41.269716 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:41.066644 2581 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-137-101.ec2.internal" Jul 06 22:08:41.269716 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:41.069778 2581 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-101.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-137-101.ec2.internal"] Jul 06 22:08:41.269716 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:41.069845 2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 22:08:41.269716 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:41.070593 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-101.ec2.internal" event="NodeHasSufficientMemory" Jul 06 22:08:41.269716 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:41.070616 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-101.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 22:08:41.269716 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:41.070629 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-101.ec2.internal" event="NodeHasSufficientPID" Jul 06 22:08:41.269716 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:41.072849 2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 22:08:41.269716 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:41.072963 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-101.ec2.internal" Jul 06 22:08:41.269716 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:41.072991 2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 22:08:41.269716 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:41.073482 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-101.ec2.internal" event="NodeHasSufficientMemory" Jul 06 22:08:41.269716 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:41.073516 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-101.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 22:08:41.269716 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:41.073516 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-101.ec2.internal" event="NodeHasSufficientMemory" Jul 06 22:08:41.269716 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:41.073526 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-101.ec2.internal" event="NodeHasSufficientPID" Jul 06 22:08:41.269716 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:41.073538 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-101.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 22:08:41.269716 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:41.073550 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-101.ec2.internal" event="NodeHasSufficientPID" Jul 06 22:08:41.269716 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:41.075622 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-101.ec2.internal" Jul 06 22:08:41.269716 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:41.075642 2581 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 22:08:41.269716 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:41.076306 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-101.ec2.internal" event="NodeHasSufficientMemory" Jul 06 22:08:41.269716 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:41.076332 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-101.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 22:08:41.269716 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:41.076363 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-101.ec2.internal" event="NodeHasSufficientPID" Jul 06 22:08:41.269716 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:41.078051 2581 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-137-101.ec2.internal" Jul 06 22:08:41.269716 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:08:41.078079 2581 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-137-101.ec2.internal\": node \"ip-10-0-137-101.ec2.internal\" not found" Jul 06 22:08:41.269716 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:08:41.104807 2581 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-137-101.ec2.internal\" not found" node="ip-10-0-137-101.ec2.internal" Jul 06 22:08:41.269716 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:08:41.106180 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-101.ec2.internal\" not found" Jul 06 22:08:41.269716 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:08:41.112597 2581 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-137-101.ec2.internal\" not found" node="ip-10-0-137-101.ec2.internal" Jul 06 22:08:41.269716 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:41.159852 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/42603ae9f82038ef3f0cca72faabcc35-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-101.ec2.internal\" (UID: \"42603ae9f82038ef3f0cca72faabcc35\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-101.ec2.internal" Jul 06 22:08:41.269716 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:41.159902 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/42603ae9f82038ef3f0cca72faabcc35-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-101.ec2.internal\" (UID: \"42603ae9f82038ef3f0cca72faabcc35\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-101.ec2.internal" Jul 06 22:08:41.269716 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:41.159933 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/2573bcaf102b258c7dbacbd9609f526e-config\") pod \"kube-apiserver-proxy-ip-10-0-137-101.ec2.internal\" (UID: \"2573bcaf102b258c7dbacbd9609f526e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-101.ec2.internal" Jul 06 22:08:41.269716 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:08:41.206904 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-101.ec2.internal\" not found" Jul 06 22:08:41.269716 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:41.260674 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/2573bcaf102b258c7dbacbd9609f526e-config\") pod \"kube-apiserver-proxy-ip-10-0-137-101.ec2.internal\" (UID: \"2573bcaf102b258c7dbacbd9609f526e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-101.ec2.internal" Jul 06 22:08:41.379791 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:41.260703 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/42603ae9f82038ef3f0cca72faabcc35-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-101.ec2.internal\" (UID: \"42603ae9f82038ef3f0cca72faabcc35\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-101.ec2.internal" Jul 06 22:08:41.379791 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:41.260733 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/42603ae9f82038ef3f0cca72faabcc35-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-101.ec2.internal\" (UID: \"42603ae9f82038ef3f0cca72faabcc35\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-101.ec2.internal" Jul 06 22:08:41.379791 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:41.260769 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/2573bcaf102b258c7dbacbd9609f526e-config\") pod \"kube-apiserver-proxy-ip-10-0-137-101.ec2.internal\" (UID: \"2573bcaf102b258c7dbacbd9609f526e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-101.ec2.internal" Jul 06 22:08:41.379791 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:41.260795 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/42603ae9f82038ef3f0cca72faabcc35-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-101.ec2.internal\" (UID: \"42603ae9f82038ef3f0cca72faabcc35\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-101.ec2.internal" Jul 06 22:08:41.379791 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:41.260861 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/42603ae9f82038ef3f0cca72faabcc35-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-101.ec2.internal\" (UID: \"42603ae9f82038ef3f0cca72faabcc35\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-101.ec2.internal" Jul 06 22:08:41.379791 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:08:41.307955 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-101.ec2.internal\" not found" Jul 06 22:08:41.507717 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:08:41.408573 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-101.ec2.internal\" not found" Jul 06 22:08:41.507717 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:41.408601 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-101.ec2.internal" Jul 06 22:08:41.507717 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:41.416714 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-101.ec2.internal" Jul 06 22:08:41.509449 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:08:41.508662 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-101.ec2.internal\" not found" Jul 06 22:08:41.617831 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:08:41.609204 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-101.ec2.internal\" not found" Jul 06 22:08:41.744159 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:08:41.709334 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-101.ec2.internal\" not found" Jul 06 22:08:41.877442 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:41.784265 2581 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 06 22:08:41.877442 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:41.784445 2581 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 06 22:08:41.877442 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:08:41.810027 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-101.ec2.internal\" not found" Jul 06 22:08:41.877442 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:41.863590 2581 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 22:08:41.877442 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:41.874115 2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-05 22:03:41 +0000 UTC" deadline="2028-02-25 16:04:29.596702406 +0000 UTC" Jul 06 22:08:41.877442 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:41.874142 2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14369h55m47.722563099s" Jul 06 22:08:42.000585 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:08:41.910287 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-101.ec2.internal\" not found" Jul 06 22:08:42.019385 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:08:42.012614 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-101.ec2.internal\" not found" Jul 06 22:08:42.130533 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:08:42.113814 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-101.ec2.internal\" not found" Jul 06 22:08:42.138839 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.138204 2581 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 22:08:42.252628 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:08:42.214812 2581 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-101.ec2.internal\" not found" Jul 06 22:08:42.252628 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.249087 2581 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 22:08:42.259694 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.258469 2581 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-101.ec2.internal" Jul 06 22:08:42.275020 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.273437 2581 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 06 22:08:42.275020 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.274453 2581 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-101.ec2.internal" Jul 06 22:08:42.382201 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.285253 2581 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 06 22:08:42.784155 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.764920 2581 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 22:08:42.920541 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.835637 2581 apiserver.go:52] "Watching apiserver" Jul 06 22:08:42.920541 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.845339 2581 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 06 22:08:42.920541 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.845688 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-additional-cni-plugins-5nrhc","kube-system/global-pull-secret-syncer-klc48","openshift-image-registry/node-ca-cmtz2","openshift-multus/multus-gdwzl","openshift-multus/network-metrics-daemon-n9hgj","openshift-network-diagnostics/network-check-target-l5rc4","openshift-network-operator/iptables-alerter-fk58k","openshift-ovn-kubernetes/ovnkube-node-vt6d6","kube-system/konnectivity-agent-66tt7","kube-system/kube-apiserver-proxy-ip-10-0-137-101.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-st84l","openshift-cluster-node-tuning-operator/tuned-w7z7d","openshift-dns/node-resolver-4b7pr","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-101.ec2.internal"] Jul 06 22:08:42.920541 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.848401 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-fk58k" Jul 06 22:08:42.920541 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.850645 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-4b7pr" Jul 06 22:08:42.920541 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.851500 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 06 22:08:42.920541 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.851502 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 06 22:08:42.920541 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.851806 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-d2xcg\"" Jul 06 22:08:42.920541 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.851830 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 06 22:08:42.920541 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.852949 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-cmtz2" Jul 06 22:08:42.920541 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.853124 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 06 22:08:42.920541 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.853276 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 06 22:08:42.920541 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.853707 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-ncx2v\"" Jul 06 22:08:42.920541 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.855353 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-gdwzl" Jul 06 22:08:42.920541 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.855436 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 06 22:08:42.920541 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.855587 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 06 22:08:42.920541 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.855771 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 06 22:08:42.920541 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.856076 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-lzpnm\"" Jul 06 22:08:42.920541 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.857663 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n9hgj" Jul 06 22:08:42.920541 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.857764 2581 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 06 22:08:42.920541 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.857856 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 06 22:08:42.920541 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:08:42.857755 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n9hgj" podUID="8a83eb6f-f275-4562-b014-0b48c710b624" Jul 06 22:08:42.920541 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.858160 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 06 22:08:42.920541 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.858216 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 06 22:08:42.920541 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.858221 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-qgn5c\"" Jul 06 22:08:42.920541 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.858284 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 06 22:08:42.920541 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.859929 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l5rc4" Jul 06 22:08:42.920541 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:08:42.859987 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l5rc4" podUID="9ade138e-0e88-4087-8042-92fe2db1e1d8" Jul 06 22:08:42.920541 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.862204 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-66tt7" Jul 06 22:08:42.920541 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.864562 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-5nrhc" Jul 06 22:08:42.920541 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.865142 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-b9xc7\"" Jul 06 22:08:42.920541 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.865145 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 06 22:08:42.920541 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.865171 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 06 22:08:42.921847 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.866867 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-klc48" Jul 06 22:08:42.921847 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:08:42.866912 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-klc48" podUID="b42224bc-b677-43bf-94fb-fc75223b963a" Jul 06 22:08:42.921847 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.867310 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 06 22:08:42.921847 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.867398 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-d76rs\"" Jul 06 22:08:42.921847 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.867416 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 06 22:08:42.921847 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.869378 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-vt6d6" Jul 06 22:08:42.921847 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.870369 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w8xnp\" (UniqueName: \"kubernetes.io/projected/9a1fd480-e89a-4522-b82a-87b8c72124d5-kube-api-access-w8xnp\") pod \"node-ca-cmtz2\" (UID: \"9a1fd480-e89a-4522-b82a-87b8c72124d5\") " pod="openshift-image-registry/node-ca-cmtz2" Jul 06 22:08:42.921847 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.870403 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f2e37c39-7aed-412b-8a5d-2fc1481f707e-multus-cni-dir\") pod \"multus-gdwzl\" (UID: \"f2e37c39-7aed-412b-8a5d-2fc1481f707e\") " pod="openshift-multus/multus-gdwzl" Jul 06 22:08:42.921847 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.870429 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f2e37c39-7aed-412b-8a5d-2fc1481f707e-os-release\") pod \"multus-gdwzl\" (UID: \"f2e37c39-7aed-412b-8a5d-2fc1481f707e\") " pod="openshift-multus/multus-gdwzl" Jul 06 22:08:42.921847 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.870467 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f2e37c39-7aed-412b-8a5d-2fc1481f707e-cni-binary-copy\") pod \"multus-gdwzl\" (UID: \"f2e37c39-7aed-412b-8a5d-2fc1481f707e\") " pod="openshift-multus/multus-gdwzl" Jul 06 22:08:42.921847 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.870505 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f2e37c39-7aed-412b-8a5d-2fc1481f707e-multus-daemon-config\") pod \"multus-gdwzl\" (UID: \"f2e37c39-7aed-412b-8a5d-2fc1481f707e\") " pod="openshift-multus/multus-gdwzl" Jul 06 22:08:42.921847 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.870548 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f2e37c39-7aed-412b-8a5d-2fc1481f707e-host-run-netns\") pod \"multus-gdwzl\" (UID: \"f2e37c39-7aed-412b-8a5d-2fc1481f707e\") " pod="openshift-multus/multus-gdwzl" Jul 06 22:08:42.921847 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.870596 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f2e37c39-7aed-412b-8a5d-2fc1481f707e-system-cni-dir\") pod \"multus-gdwzl\" (UID: \"f2e37c39-7aed-412b-8a5d-2fc1481f707e\") " pod="openshift-multus/multus-gdwzl" Jul 06 22:08:42.921847 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.870613 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f2e37c39-7aed-412b-8a5d-2fc1481f707e-multus-socket-dir-parent\") pod \"multus-gdwzl\" (UID: \"f2e37c39-7aed-412b-8a5d-2fc1481f707e\") " pod="openshift-multus/multus-gdwzl" Jul 06 22:08:42.921847 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.870660 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/314ffdbc-f9e6-45c0-b37a-d216700a1de5-hosts-file\") pod \"node-resolver-4b7pr\" (UID: \"314ffdbc-f9e6-45c0-b37a-d216700a1de5\") " pod="openshift-dns/node-resolver-4b7pr" Jul 06 22:08:42.921847 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.870679 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f2e37c39-7aed-412b-8a5d-2fc1481f707e-etc-kubernetes\") pod \"multus-gdwzl\" (UID: \"f2e37c39-7aed-412b-8a5d-2fc1481f707e\") " pod="openshift-multus/multus-gdwzl" Jul 06 22:08:42.921847 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.870694 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kltv7\" (UniqueName: \"kubernetes.io/projected/8a83eb6f-f275-4562-b014-0b48c710b624-kube-api-access-kltv7\") pod \"network-metrics-daemon-n9hgj\" (UID: \"8a83eb6f-f275-4562-b014-0b48c710b624\") " pod="openshift-multus/network-metrics-daemon-n9hgj" Jul 06 22:08:42.921847 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.870717 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9a1fd480-e89a-4522-b82a-87b8c72124d5-host\") pod \"node-ca-cmtz2\" (UID: \"9a1fd480-e89a-4522-b82a-87b8c72124d5\") " pod="openshift-image-registry/node-ca-cmtz2" Jul 06 22:08:42.921847 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.870740 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/9a1fd480-e89a-4522-b82a-87b8c72124d5-serviceca\") pod \"node-ca-cmtz2\" (UID: \"9a1fd480-e89a-4522-b82a-87b8c72124d5\") " pod="openshift-image-registry/node-ca-cmtz2" Jul 06 22:08:42.921847 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.870774 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f2e37c39-7aed-412b-8a5d-2fc1481f707e-host-var-lib-cni-bin\") pod \"multus-gdwzl\" (UID: \"f2e37c39-7aed-412b-8a5d-2fc1481f707e\") " pod="openshift-multus/multus-gdwzl" Jul 06 22:08:42.924993 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.870792 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f2e37c39-7aed-412b-8a5d-2fc1481f707e-multus-conf-dir\") pod \"multus-gdwzl\" (UID: \"f2e37c39-7aed-412b-8a5d-2fc1481f707e\") " pod="openshift-multus/multus-gdwzl" Jul 06 22:08:42.924993 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.870829 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tlvkd\" (UniqueName: \"kubernetes.io/projected/9ade138e-0e88-4087-8042-92fe2db1e1d8-kube-api-access-tlvkd\") pod \"network-check-target-l5rc4\" (UID: \"9ade138e-0e88-4087-8042-92fe2db1e1d8\") " pod="openshift-network-diagnostics/network-check-target-l5rc4" Jul 06 22:08:42.924993 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.870872 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e5b109e4-bcab-4c91-8791-3c7fcd077f27-iptables-alerter-script\") pod \"iptables-alerter-fk58k\" (UID: \"e5b109e4-bcab-4c91-8791-3c7fcd077f27\") " pod="openshift-network-operator/iptables-alerter-fk58k" Jul 06 22:08:42.924993 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.870890 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/314ffdbc-f9e6-45c0-b37a-d216700a1de5-tmp-dir\") pod \"node-resolver-4b7pr\" (UID: \"314ffdbc-f9e6-45c0-b37a-d216700a1de5\") " pod="openshift-dns/node-resolver-4b7pr" Jul 06 22:08:42.924993 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.870916 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f2e37c39-7aed-412b-8a5d-2fc1481f707e-host-var-lib-kubelet\") pod \"multus-gdwzl\" (UID: \"f2e37c39-7aed-412b-8a5d-2fc1481f707e\") " pod="openshift-multus/multus-gdwzl" Jul 06 22:08:42.924993 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.870936 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f2e37c39-7aed-412b-8a5d-2fc1481f707e-hostroot\") pod \"multus-gdwzl\" (UID: \"f2e37c39-7aed-412b-8a5d-2fc1481f707e\") " pod="openshift-multus/multus-gdwzl" Jul 06 22:08:42.924993 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.870965 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wlnmw\" (UniqueName: \"kubernetes.io/projected/f2e37c39-7aed-412b-8a5d-2fc1481f707e-kube-api-access-wlnmw\") pod \"multus-gdwzl\" (UID: \"f2e37c39-7aed-412b-8a5d-2fc1481f707e\") " pod="openshift-multus/multus-gdwzl" Jul 06 22:08:42.924993 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.870983 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f2e37c39-7aed-412b-8a5d-2fc1481f707e-host-run-multus-certs\") pod \"multus-gdwzl\" (UID: \"f2e37c39-7aed-412b-8a5d-2fc1481f707e\") " pod="openshift-multus/multus-gdwzl" Jul 06 22:08:42.924993 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.870998 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qj46v\" (UniqueName: \"kubernetes.io/projected/314ffdbc-f9e6-45c0-b37a-d216700a1de5-kube-api-access-qj46v\") pod \"node-resolver-4b7pr\" (UID: \"314ffdbc-f9e6-45c0-b37a-d216700a1de5\") " pod="openshift-dns/node-resolver-4b7pr" Jul 06 22:08:42.924993 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.871017 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f2e37c39-7aed-412b-8a5d-2fc1481f707e-host-run-k8s-cni-cncf-io\") pod \"multus-gdwzl\" (UID: \"f2e37c39-7aed-412b-8a5d-2fc1481f707e\") " pod="openshift-multus/multus-gdwzl" Jul 06 22:08:42.924993 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.871041 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8a83eb6f-f275-4562-b014-0b48c710b624-metrics-certs\") pod \"network-metrics-daemon-n9hgj\" (UID: \"8a83eb6f-f275-4562-b014-0b48c710b624\") " pod="openshift-multus/network-metrics-daemon-n9hgj" Jul 06 22:08:42.924993 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.871055 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e5b109e4-bcab-4c91-8791-3c7fcd077f27-host-slash\") pod \"iptables-alerter-fk58k\" (UID: \"e5b109e4-bcab-4c91-8791-3c7fcd077f27\") " pod="openshift-network-operator/iptables-alerter-fk58k" Jul 06 22:08:42.924993 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.871088 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f2e37c39-7aed-412b-8a5d-2fc1481f707e-cnibin\") pod \"multus-gdwzl\" (UID: \"f2e37c39-7aed-412b-8a5d-2fc1481f707e\") " pod="openshift-multus/multus-gdwzl" Jul 06 22:08:42.924993 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.871110 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f2e37c39-7aed-412b-8a5d-2fc1481f707e-host-var-lib-cni-multus\") pod \"multus-gdwzl\" (UID: \"f2e37c39-7aed-412b-8a5d-2fc1481f707e\") " pod="openshift-multus/multus-gdwzl" Jul 06 22:08:42.924993 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.871126 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fdhw6\" (UniqueName: \"kubernetes.io/projected/e5b109e4-bcab-4c91-8791-3c7fcd077f27-kube-api-access-fdhw6\") pod \"iptables-alerter-fk58k\" (UID: \"e5b109e4-bcab-4c91-8791-3c7fcd077f27\") " pod="openshift-network-operator/iptables-alerter-fk58k" Jul 06 22:08:42.924993 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.872054 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-st84l" Jul 06 22:08:42.924993 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.872235 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-xpkj6\"" Jul 06 22:08:42.924993 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.872569 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 06 22:08:42.924993 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.874065 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 06 22:08:42.925893 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.874169 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 06 22:08:42.925893 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.874315 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 06 22:08:42.925893 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.874470 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 06 22:08:42.925893 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.874542 2581 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 06 22:08:42.925893 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.874603 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 06 22:08:42.925893 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.874617 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 06 22:08:42.925893 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.875243 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 06 22:08:42.925893 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.875500 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-t8zkp\"" Jul 06 22:08:42.925893 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.875525 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 06 22:08:42.925893 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.877000 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-w7z7d" Jul 06 22:08:42.925893 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.879682 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 06 22:08:42.925893 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.879699 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 06 22:08:42.925893 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.879699 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-wd69f\"" Jul 06 22:08:42.925893 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.902723 2581 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-2h4df" Jul 06 22:08:42.925893 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.911005 2581 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-2h4df" Jul 06 22:08:43.041240 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.959452 2581 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 06 22:08:43.041240 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.971283 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f1de085f-40f2-4f95-a5bb-78aba60de69e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-5nrhc\" (UID: \"f1de085f-40f2-4f95-a5bb-78aba60de69e\") " pod="openshift-multus/multus-additional-cni-plugins-5nrhc" Jul 06 22:08:43.041240 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.971323 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a7a62305-2195-4227-b5b7-5dec03864283-run-ovn\") pod \"ovnkube-node-vt6d6\" (UID: \"a7a62305-2195-4227-b5b7-5dec03864283\") " pod="openshift-ovn-kubernetes/ovnkube-node-vt6d6" Jul 06 22:08:43.041240 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.971351 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a7a62305-2195-4227-b5b7-5dec03864283-env-overrides\") pod \"ovnkube-node-vt6d6\" (UID: \"a7a62305-2195-4227-b5b7-5dec03864283\") " pod="openshift-ovn-kubernetes/ovnkube-node-vt6d6" Jul 06 22:08:43.041240 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.971382 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f2e37c39-7aed-412b-8a5d-2fc1481f707e-cnibin\") pod \"multus-gdwzl\" (UID: \"f2e37c39-7aed-412b-8a5d-2fc1481f707e\") " pod="openshift-multus/multus-gdwzl" Jul 06 22:08:43.041240 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.971408 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f2e37c39-7aed-412b-8a5d-2fc1481f707e-host-var-lib-cni-multus\") pod \"multus-gdwzl\" (UID: \"f2e37c39-7aed-412b-8a5d-2fc1481f707e\") " pod="openshift-multus/multus-gdwzl" Jul 06 22:08:43.041240 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.971454 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fdhw6\" (UniqueName: \"kubernetes.io/projected/e5b109e4-bcab-4c91-8791-3c7fcd077f27-kube-api-access-fdhw6\") pod \"iptables-alerter-fk58k\" (UID: \"e5b109e4-bcab-4c91-8791-3c7fcd077f27\") " pod="openshift-network-operator/iptables-alerter-fk58k" Jul 06 22:08:43.041240 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.971480 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/0938d2ec-6cb6-4f74-882c-06081dbbfb88-etc-modprobe-d\") pod \"tuned-w7z7d\" (UID: \"0938d2ec-6cb6-4f74-882c-06081dbbfb88\") " pod="openshift-cluster-node-tuning-operator/tuned-w7z7d" Jul 06 22:08:43.041240 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.971503 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/0938d2ec-6cb6-4f74-882c-06081dbbfb88-etc-sysctl-conf\") pod \"tuned-w7z7d\" (UID: \"0938d2ec-6cb6-4f74-882c-06081dbbfb88\") " pod="openshift-cluster-node-tuning-operator/tuned-w7z7d" Jul 06 22:08:43.041240 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.971507 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f2e37c39-7aed-412b-8a5d-2fc1481f707e-cnibin\") pod \"multus-gdwzl\" (UID: \"f2e37c39-7aed-412b-8a5d-2fc1481f707e\") " pod="openshift-multus/multus-gdwzl" Jul 06 22:08:43.041240 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.971531 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w8xnp\" (UniqueName: \"kubernetes.io/projected/9a1fd480-e89a-4522-b82a-87b8c72124d5-kube-api-access-w8xnp\") pod \"node-ca-cmtz2\" (UID: \"9a1fd480-e89a-4522-b82a-87b8c72124d5\") " pod="openshift-image-registry/node-ca-cmtz2" Jul 06 22:08:43.041240 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.971507 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f2e37c39-7aed-412b-8a5d-2fc1481f707e-host-var-lib-cni-multus\") pod \"multus-gdwzl\" (UID: \"f2e37c39-7aed-412b-8a5d-2fc1481f707e\") " pod="openshift-multus/multus-gdwzl" Jul 06 22:08:43.041240 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.971577 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a7a62305-2195-4227-b5b7-5dec03864283-host-slash\") pod \"ovnkube-node-vt6d6\" (UID: \"a7a62305-2195-4227-b5b7-5dec03864283\") " pod="openshift-ovn-kubernetes/ovnkube-node-vt6d6" Jul 06 22:08:43.041240 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.971607 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a7a62305-2195-4227-b5b7-5dec03864283-host-run-netns\") pod \"ovnkube-node-vt6d6\" (UID: \"a7a62305-2195-4227-b5b7-5dec03864283\") " pod="openshift-ovn-kubernetes/ovnkube-node-vt6d6" Jul 06 22:08:43.041240 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.971635 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a7a62305-2195-4227-b5b7-5dec03864283-ovnkube-config\") pod \"ovnkube-node-vt6d6\" (UID: \"a7a62305-2195-4227-b5b7-5dec03864283\") " pod="openshift-ovn-kubernetes/ovnkube-node-vt6d6" Jul 06 22:08:43.041240 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.971663 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/8ef94701-6fd1-4b4d-aa83-e6effd1023bf-registration-dir\") pod \"aws-ebs-csi-driver-node-st84l\" (UID: \"8ef94701-6fd1-4b4d-aa83-e6effd1023bf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-st84l" Jul 06 22:08:43.041240 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.971721 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f2e37c39-7aed-412b-8a5d-2fc1481f707e-system-cni-dir\") pod \"multus-gdwzl\" (UID: \"f2e37c39-7aed-412b-8a5d-2fc1481f707e\") " pod="openshift-multus/multus-gdwzl" Jul 06 22:08:43.041240 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.971757 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/314ffdbc-f9e6-45c0-b37a-d216700a1de5-hosts-file\") pod \"node-resolver-4b7pr\" (UID: \"314ffdbc-f9e6-45c0-b37a-d216700a1de5\") " pod="openshift-dns/node-resolver-4b7pr" Jul 06 22:08:43.052567 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.971785 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b42224bc-b677-43bf-94fb-fc75223b963a-original-pull-secret\") pod \"global-pull-secret-syncer-klc48\" (UID: \"b42224bc-b677-43bf-94fb-fc75223b963a\") " pod="kube-system/global-pull-secret-syncer-klc48" Jul 06 22:08:43.052567 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.971807 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/0938d2ec-6cb6-4f74-882c-06081dbbfb88-etc-systemd\") pod \"tuned-w7z7d\" (UID: \"0938d2ec-6cb6-4f74-882c-06081dbbfb88\") " pod="openshift-cluster-node-tuning-operator/tuned-w7z7d" Jul 06 22:08:43.052567 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.971830 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nvnzj\" (UniqueName: \"kubernetes.io/projected/f1de085f-40f2-4f95-a5bb-78aba60de69e-kube-api-access-nvnzj\") pod \"multus-additional-cni-plugins-5nrhc\" (UID: \"f1de085f-40f2-4f95-a5bb-78aba60de69e\") " pod="openshift-multus/multus-additional-cni-plugins-5nrhc" Jul 06 22:08:43.052567 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.971856 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a7a62305-2195-4227-b5b7-5dec03864283-host-cni-netd\") pod \"ovnkube-node-vt6d6\" (UID: \"a7a62305-2195-4227-b5b7-5dec03864283\") " pod="openshift-ovn-kubernetes/ovnkube-node-vt6d6" Jul 06 22:08:43.052567 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.971871 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f2e37c39-7aed-412b-8a5d-2fc1481f707e-system-cni-dir\") pod \"multus-gdwzl\" (UID: \"f2e37c39-7aed-412b-8a5d-2fc1481f707e\") " pod="openshift-multus/multus-gdwzl" Jul 06 22:08:43.052567 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.971881 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/314ffdbc-f9e6-45c0-b37a-d216700a1de5-hosts-file\") pod \"node-resolver-4b7pr\" (UID: \"314ffdbc-f9e6-45c0-b37a-d216700a1de5\") " pod="openshift-dns/node-resolver-4b7pr" Jul 06 22:08:43.052567 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.971884 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f2e37c39-7aed-412b-8a5d-2fc1481f707e-etc-kubernetes\") pod \"multus-gdwzl\" (UID: \"f2e37c39-7aed-412b-8a5d-2fc1481f707e\") " pod="openshift-multus/multus-gdwzl" Jul 06 22:08:43.052567 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.971916 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a7a62305-2195-4227-b5b7-5dec03864283-run-systemd\") pod \"ovnkube-node-vt6d6\" (UID: \"a7a62305-2195-4227-b5b7-5dec03864283\") " pod="openshift-ovn-kubernetes/ovnkube-node-vt6d6" Jul 06 22:08:43.052567 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.971924 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f2e37c39-7aed-412b-8a5d-2fc1481f707e-etc-kubernetes\") pod \"multus-gdwzl\" (UID: \"f2e37c39-7aed-412b-8a5d-2fc1481f707e\") " pod="openshift-multus/multus-gdwzl" Jul 06 22:08:43.052567 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.971937 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/8ef94701-6fd1-4b4d-aa83-e6effd1023bf-sys-fs\") pod \"aws-ebs-csi-driver-node-st84l\" (UID: \"8ef94701-6fd1-4b4d-aa83-e6effd1023bf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-st84l" Jul 06 22:08:43.052567 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.971955 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f1de085f-40f2-4f95-a5bb-78aba60de69e-cni-binary-copy\") pod \"multus-additional-cni-plugins-5nrhc\" (UID: \"f1de085f-40f2-4f95-a5bb-78aba60de69e\") " pod="openshift-multus/multus-additional-cni-plugins-5nrhc" Jul 06 22:08:43.052567 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.971973 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f1de085f-40f2-4f95-a5bb-78aba60de69e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-5nrhc\" (UID: \"f1de085f-40f2-4f95-a5bb-78aba60de69e\") " pod="openshift-multus/multus-additional-cni-plugins-5nrhc" Jul 06 22:08:43.052567 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.971992 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f2e37c39-7aed-412b-8a5d-2fc1481f707e-host-var-lib-cni-bin\") pod \"multus-gdwzl\" (UID: \"f2e37c39-7aed-412b-8a5d-2fc1481f707e\") " pod="openshift-multus/multus-gdwzl" Jul 06 22:08:43.052567 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.972032 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f2e37c39-7aed-412b-8a5d-2fc1481f707e-host-var-lib-cni-bin\") pod \"multus-gdwzl\" (UID: \"f2e37c39-7aed-412b-8a5d-2fc1481f707e\") " pod="openshift-multus/multus-gdwzl" Jul 06 22:08:43.052567 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.972034 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f2e37c39-7aed-412b-8a5d-2fc1481f707e-multus-conf-dir\") pod \"multus-gdwzl\" (UID: \"f2e37c39-7aed-412b-8a5d-2fc1481f707e\") " pod="openshift-multus/multus-gdwzl" Jul 06 22:08:43.052567 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.972060 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f2e37c39-7aed-412b-8a5d-2fc1481f707e-multus-conf-dir\") pod \"multus-gdwzl\" (UID: \"f2e37c39-7aed-412b-8a5d-2fc1481f707e\") " pod="openshift-multus/multus-gdwzl" Jul 06 22:08:43.052567 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.972087 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tlvkd\" (UniqueName: \"kubernetes.io/projected/9ade138e-0e88-4087-8042-92fe2db1e1d8-kube-api-access-tlvkd\") pod \"network-check-target-l5rc4\" (UID: \"9ade138e-0e88-4087-8042-92fe2db1e1d8\") " pod="openshift-network-diagnostics/network-check-target-l5rc4" Jul 06 22:08:43.053892 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.972117 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2041467a-e1c5-4702-96c6-8be98208f3eb-agent-certs\") pod \"konnectivity-agent-66tt7\" (UID: \"2041467a-e1c5-4702-96c6-8be98208f3eb\") " pod="kube-system/konnectivity-agent-66tt7" Jul 06 22:08:43.053892 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.972141 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a7a62305-2195-4227-b5b7-5dec03864283-var-lib-openvswitch\") pod \"ovnkube-node-vt6d6\" (UID: \"a7a62305-2195-4227-b5b7-5dec03864283\") " pod="openshift-ovn-kubernetes/ovnkube-node-vt6d6" Jul 06 22:08:43.053892 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.972200 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a7a62305-2195-4227-b5b7-5dec03864283-ovn-node-metrics-cert\") pod \"ovnkube-node-vt6d6\" (UID: \"a7a62305-2195-4227-b5b7-5dec03864283\") " pod="openshift-ovn-kubernetes/ovnkube-node-vt6d6" Jul 06 22:08:43.053892 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.972227 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f2e37c39-7aed-412b-8a5d-2fc1481f707e-hostroot\") pod \"multus-gdwzl\" (UID: \"f2e37c39-7aed-412b-8a5d-2fc1481f707e\") " pod="openshift-multus/multus-gdwzl" Jul 06 22:08:43.053892 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.972253 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wlnmw\" (UniqueName: \"kubernetes.io/projected/f2e37c39-7aed-412b-8a5d-2fc1481f707e-kube-api-access-wlnmw\") pod \"multus-gdwzl\" (UID: \"f2e37c39-7aed-412b-8a5d-2fc1481f707e\") " pod="openshift-multus/multus-gdwzl" Jul 06 22:08:43.053892 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.972269 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0938d2ec-6cb6-4f74-882c-06081dbbfb88-var-lib-kubelet\") pod \"tuned-w7z7d\" (UID: \"0938d2ec-6cb6-4f74-882c-06081dbbfb88\") " pod="openshift-cluster-node-tuning-operator/tuned-w7z7d" Jul 06 22:08:43.053892 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.972284 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0938d2ec-6cb6-4f74-882c-06081dbbfb88-host\") pod \"tuned-w7z7d\" (UID: \"0938d2ec-6cb6-4f74-882c-06081dbbfb88\") " pod="openshift-cluster-node-tuning-operator/tuned-w7z7d" Jul 06 22:08:43.053892 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.972304 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0938d2ec-6cb6-4f74-882c-06081dbbfb88-tmp\") pod \"tuned-w7z7d\" (UID: \"0938d2ec-6cb6-4f74-882c-06081dbbfb88\") " pod="openshift-cluster-node-tuning-operator/tuned-w7z7d" Jul 06 22:08:43.053892 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.972313 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f2e37c39-7aed-412b-8a5d-2fc1481f707e-hostroot\") pod \"multus-gdwzl\" (UID: \"f2e37c39-7aed-412b-8a5d-2fc1481f707e\") " pod="openshift-multus/multus-gdwzl" Jul 06 22:08:43.053892 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.972330 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a7a62305-2195-4227-b5b7-5dec03864283-host-run-ovn-kubernetes\") pod \"ovnkube-node-vt6d6\" (UID: \"a7a62305-2195-4227-b5b7-5dec03864283\") " pod="openshift-ovn-kubernetes/ovnkube-node-vt6d6" Jul 06 22:08:43.053892 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.972356 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/8ef94701-6fd1-4b4d-aa83-e6effd1023bf-device-dir\") pod \"aws-ebs-csi-driver-node-st84l\" (UID: \"8ef94701-6fd1-4b4d-aa83-e6effd1023bf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-st84l" Jul 06 22:08:43.053892 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.972383 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qj46v\" (UniqueName: \"kubernetes.io/projected/314ffdbc-f9e6-45c0-b37a-d216700a1de5-kube-api-access-qj46v\") pod \"node-resolver-4b7pr\" (UID: \"314ffdbc-f9e6-45c0-b37a-d216700a1de5\") " pod="openshift-dns/node-resolver-4b7pr" Jul 06 22:08:43.053892 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.972408 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f1de085f-40f2-4f95-a5bb-78aba60de69e-system-cni-dir\") pod \"multus-additional-cni-plugins-5nrhc\" (UID: \"f1de085f-40f2-4f95-a5bb-78aba60de69e\") " pod="openshift-multus/multus-additional-cni-plugins-5nrhc" Jul 06 22:08:43.053892 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.972433 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a7a62305-2195-4227-b5b7-5dec03864283-ovnkube-script-lib\") pod \"ovnkube-node-vt6d6\" (UID: \"a7a62305-2195-4227-b5b7-5dec03864283\") " pod="openshift-ovn-kubernetes/ovnkube-node-vt6d6" Jul 06 22:08:43.053892 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.972459 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8a83eb6f-f275-4562-b014-0b48c710b624-metrics-certs\") pod \"network-metrics-daemon-n9hgj\" (UID: \"8a83eb6f-f275-4562-b014-0b48c710b624\") " pod="openshift-multus/network-metrics-daemon-n9hgj" Jul 06 22:08:43.053892 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.972483 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a7a62305-2195-4227-b5b7-5dec03864283-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vt6d6\" (UID: \"a7a62305-2195-4227-b5b7-5dec03864283\") " pod="openshift-ovn-kubernetes/ovnkube-node-vt6d6" Jul 06 22:08:43.053892 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.972523 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e5b109e4-bcab-4c91-8791-3c7fcd077f27-host-slash\") pod \"iptables-alerter-fk58k\" (UID: \"e5b109e4-bcab-4c91-8791-3c7fcd077f27\") " pod="openshift-network-operator/iptables-alerter-fk58k" Jul 06 22:08:43.054933 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.972540 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/0938d2ec-6cb6-4f74-882c-06081dbbfb88-etc-sysconfig\") pod \"tuned-w7z7d\" (UID: \"0938d2ec-6cb6-4f74-882c-06081dbbfb88\") " pod="openshift-cluster-node-tuning-operator/tuned-w7z7d" Jul 06 22:08:43.054933 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:08:42.972545 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 22:08:43.054933 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.972560 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0938d2ec-6cb6-4f74-882c-06081dbbfb88-etc-kubernetes\") pod \"tuned-w7z7d\" (UID: \"0938d2ec-6cb6-4f74-882c-06081dbbfb88\") " pod="openshift-cluster-node-tuning-operator/tuned-w7z7d" Jul 06 22:08:43.054933 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.972591 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e5b109e4-bcab-4c91-8791-3c7fcd077f27-host-slash\") pod \"iptables-alerter-fk58k\" (UID: \"e5b109e4-bcab-4c91-8791-3c7fcd077f27\") " pod="openshift-network-operator/iptables-alerter-fk58k" Jul 06 22:08:43.054933 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:08:42.972629 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8a83eb6f-f275-4562-b014-0b48c710b624-metrics-certs podName:8a83eb6f-f275-4562-b014-0b48c710b624 nodeName:}" failed. No retries permitted until 2026-07-06 22:08:43.472608819 +0000 UTC m=+3.031404136 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8a83eb6f-f275-4562-b014-0b48c710b624-metrics-certs") pod "network-metrics-daemon-n9hgj" (UID: "8a83eb6f-f275-4562-b014-0b48c710b624") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 22:08:43.054933 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.972653 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a7a62305-2195-4227-b5b7-5dec03864283-systemd-units\") pod \"ovnkube-node-vt6d6\" (UID: \"a7a62305-2195-4227-b5b7-5dec03864283\") " pod="openshift-ovn-kubernetes/ovnkube-node-vt6d6" Jul 06 22:08:43.054933 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.972676 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f2e37c39-7aed-412b-8a5d-2fc1481f707e-multus-cni-dir\") pod \"multus-gdwzl\" (UID: \"f2e37c39-7aed-412b-8a5d-2fc1481f707e\") " pod="openshift-multus/multus-gdwzl" Jul 06 22:08:43.054933 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.972693 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f2e37c39-7aed-412b-8a5d-2fc1481f707e-os-release\") pod \"multus-gdwzl\" (UID: \"f2e37c39-7aed-412b-8a5d-2fc1481f707e\") " pod="openshift-multus/multus-gdwzl" Jul 06 22:08:43.054933 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.972715 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f2e37c39-7aed-412b-8a5d-2fc1481f707e-cni-binary-copy\") pod \"multus-gdwzl\" (UID: \"f2e37c39-7aed-412b-8a5d-2fc1481f707e\") " pod="openshift-multus/multus-gdwzl" Jul 06 22:08:43.054933 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.972731 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f2e37c39-7aed-412b-8a5d-2fc1481f707e-multus-daemon-config\") pod \"multus-gdwzl\" (UID: \"f2e37c39-7aed-412b-8a5d-2fc1481f707e\") " pod="openshift-multus/multus-gdwzl" Jul 06 22:08:43.054933 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.972768 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f1de085f-40f2-4f95-a5bb-78aba60de69e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-5nrhc\" (UID: \"f1de085f-40f2-4f95-a5bb-78aba60de69e\") " pod="openshift-multus/multus-additional-cni-plugins-5nrhc" Jul 06 22:08:43.054933 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.972796 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f2e37c39-7aed-412b-8a5d-2fc1481f707e-host-run-netns\") pod \"multus-gdwzl\" (UID: \"f2e37c39-7aed-412b-8a5d-2fc1481f707e\") " pod="openshift-multus/multus-gdwzl" Jul 06 22:08:43.054933 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.972813 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f2e37c39-7aed-412b-8a5d-2fc1481f707e-multus-socket-dir-parent\") pod \"multus-gdwzl\" (UID: \"f2e37c39-7aed-412b-8a5d-2fc1481f707e\") " pod="openshift-multus/multus-gdwzl" Jul 06 22:08:43.054933 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.972828 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2041467a-e1c5-4702-96c6-8be98208f3eb-konnectivity-ca\") pod \"konnectivity-agent-66tt7\" (UID: \"2041467a-e1c5-4702-96c6-8be98208f3eb\") " pod="kube-system/konnectivity-agent-66tt7" Jul 06 22:08:43.054933 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.972846 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f2e37c39-7aed-412b-8a5d-2fc1481f707e-os-release\") pod \"multus-gdwzl\" (UID: \"f2e37c39-7aed-412b-8a5d-2fc1481f707e\") " pod="openshift-multus/multus-gdwzl" Jul 06 22:08:43.054933 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.972860 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0938d2ec-6cb6-4f74-882c-06081dbbfb88-lib-modules\") pod \"tuned-w7z7d\" (UID: \"0938d2ec-6cb6-4f74-882c-06081dbbfb88\") " pod="openshift-cluster-node-tuning-operator/tuned-w7z7d" Jul 06 22:08:43.054933 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.972860 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f2e37c39-7aed-412b-8a5d-2fc1481f707e-host-run-netns\") pod \"multus-gdwzl\" (UID: \"f2e37c39-7aed-412b-8a5d-2fc1481f707e\") " pod="openshift-multus/multus-gdwzl" Jul 06 22:08:43.056731 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.972879 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f2e37c39-7aed-412b-8a5d-2fc1481f707e-multus-cni-dir\") pod \"multus-gdwzl\" (UID: \"f2e37c39-7aed-412b-8a5d-2fc1481f707e\") " pod="openshift-multus/multus-gdwzl" Jul 06 22:08:43.056731 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.972884 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/0938d2ec-6cb6-4f74-882c-06081dbbfb88-etc-tuned\") pod \"tuned-w7z7d\" (UID: \"0938d2ec-6cb6-4f74-882c-06081dbbfb88\") " pod="openshift-cluster-node-tuning-operator/tuned-w7z7d" Jul 06 22:08:43.056731 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.972919 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f2e37c39-7aed-412b-8a5d-2fc1481f707e-multus-socket-dir-parent\") pod \"multus-gdwzl\" (UID: \"f2e37c39-7aed-412b-8a5d-2fc1481f707e\") " pod="openshift-multus/multus-gdwzl" Jul 06 22:08:43.056731 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.972955 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a7a62305-2195-4227-b5b7-5dec03864283-etc-openvswitch\") pod \"ovnkube-node-vt6d6\" (UID: \"a7a62305-2195-4227-b5b7-5dec03864283\") " pod="openshift-ovn-kubernetes/ovnkube-node-vt6d6" Jul 06 22:08:43.056731 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.972982 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a7a62305-2195-4227-b5b7-5dec03864283-host-cni-bin\") pod \"ovnkube-node-vt6d6\" (UID: \"a7a62305-2195-4227-b5b7-5dec03864283\") " pod="openshift-ovn-kubernetes/ovnkube-node-vt6d6" Jul 06 22:08:43.056731 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.973016 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/8ef94701-6fd1-4b4d-aa83-e6effd1023bf-socket-dir\") pod \"aws-ebs-csi-driver-node-st84l\" (UID: \"8ef94701-6fd1-4b4d-aa83-e6effd1023bf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-st84l" Jul 06 22:08:43.056731 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.973043 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kltv7\" (UniqueName: \"kubernetes.io/projected/8a83eb6f-f275-4562-b014-0b48c710b624-kube-api-access-kltv7\") pod \"network-metrics-daemon-n9hgj\" (UID: \"8a83eb6f-f275-4562-b014-0b48c710b624\") " pod="openshift-multus/network-metrics-daemon-n9hgj" Jul 06 22:08:43.056731 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.973059 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/0938d2ec-6cb6-4f74-882c-06081dbbfb88-etc-sysctl-d\") pod \"tuned-w7z7d\" (UID: \"0938d2ec-6cb6-4f74-882c-06081dbbfb88\") " pod="openshift-cluster-node-tuning-operator/tuned-w7z7d" Jul 06 22:08:43.056731 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.973097 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/0938d2ec-6cb6-4f74-882c-06081dbbfb88-run\") pod \"tuned-w7z7d\" (UID: \"0938d2ec-6cb6-4f74-882c-06081dbbfb88\") " pod="openshift-cluster-node-tuning-operator/tuned-w7z7d" Jul 06 22:08:43.056731 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.973160 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f1de085f-40f2-4f95-a5bb-78aba60de69e-cnibin\") pod \"multus-additional-cni-plugins-5nrhc\" (UID: \"f1de085f-40f2-4f95-a5bb-78aba60de69e\") " pod="openshift-multus/multus-additional-cni-plugins-5nrhc" Jul 06 22:08:43.056731 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.973183 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/8ef94701-6fd1-4b4d-aa83-e6effd1023bf-etc-selinux\") pod \"aws-ebs-csi-driver-node-st84l\" (UID: \"8ef94701-6fd1-4b4d-aa83-e6effd1023bf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-st84l" Jul 06 22:08:43.056731 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.973214 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9jchz\" (UniqueName: \"kubernetes.io/projected/8ef94701-6fd1-4b4d-aa83-e6effd1023bf-kube-api-access-9jchz\") pod \"aws-ebs-csi-driver-node-st84l\" (UID: \"8ef94701-6fd1-4b4d-aa83-e6effd1023bf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-st84l" Jul 06 22:08:43.056731 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.973244 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0938d2ec-6cb6-4f74-882c-06081dbbfb88-sys\") pod \"tuned-w7z7d\" (UID: \"0938d2ec-6cb6-4f74-882c-06081dbbfb88\") " pod="openshift-cluster-node-tuning-operator/tuned-w7z7d" Jul 06 22:08:43.056731 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.973267 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9a1fd480-e89a-4522-b82a-87b8c72124d5-host\") pod \"node-ca-cmtz2\" (UID: \"9a1fd480-e89a-4522-b82a-87b8c72124d5\") " pod="openshift-image-registry/node-ca-cmtz2" Jul 06 22:08:43.056731 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.973284 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/9a1fd480-e89a-4522-b82a-87b8c72124d5-serviceca\") pod \"node-ca-cmtz2\" (UID: \"9a1fd480-e89a-4522-b82a-87b8c72124d5\") " pod="openshift-image-registry/node-ca-cmtz2" Jul 06 22:08:43.056731 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.973306 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e5b109e4-bcab-4c91-8791-3c7fcd077f27-iptables-alerter-script\") pod \"iptables-alerter-fk58k\" (UID: \"e5b109e4-bcab-4c91-8791-3c7fcd077f27\") " pod="openshift-network-operator/iptables-alerter-fk58k" Jul 06 22:08:43.056731 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.973320 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9a1fd480-e89a-4522-b82a-87b8c72124d5-host\") pod \"node-ca-cmtz2\" (UID: \"9a1fd480-e89a-4522-b82a-87b8c72124d5\") " pod="openshift-image-registry/node-ca-cmtz2" Jul 06 22:08:43.058080 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.973325 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/314ffdbc-f9e6-45c0-b37a-d216700a1de5-tmp-dir\") pod \"node-resolver-4b7pr\" (UID: \"314ffdbc-f9e6-45c0-b37a-d216700a1de5\") " pod="openshift-dns/node-resolver-4b7pr" Jul 06 22:08:43.058080 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.973348 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a7a62305-2195-4227-b5b7-5dec03864283-run-openvswitch\") pod \"ovnkube-node-vt6d6\" (UID: \"a7a62305-2195-4227-b5b7-5dec03864283\") " pod="openshift-ovn-kubernetes/ovnkube-node-vt6d6" Jul 06 22:08:43.058080 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.973366 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tvhkv\" (UniqueName: \"kubernetes.io/projected/a7a62305-2195-4227-b5b7-5dec03864283-kube-api-access-tvhkv\") pod \"ovnkube-node-vt6d6\" (UID: \"a7a62305-2195-4227-b5b7-5dec03864283\") " pod="openshift-ovn-kubernetes/ovnkube-node-vt6d6" Jul 06 22:08:43.058080 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.973395 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/8ef94701-6fd1-4b4d-aa83-e6effd1023bf-kubelet-dir\") pod \"aws-ebs-csi-driver-node-st84l\" (UID: \"8ef94701-6fd1-4b4d-aa83-e6effd1023bf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-st84l" Jul 06 22:08:43.058080 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.973413 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f2e37c39-7aed-412b-8a5d-2fc1481f707e-host-var-lib-kubelet\") pod \"multus-gdwzl\" (UID: \"f2e37c39-7aed-412b-8a5d-2fc1481f707e\") " pod="openshift-multus/multus-gdwzl" Jul 06 22:08:43.058080 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.973432 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dgtg9\" (UniqueName: \"kubernetes.io/projected/0938d2ec-6cb6-4f74-882c-06081dbbfb88-kube-api-access-dgtg9\") pod \"tuned-w7z7d\" (UID: \"0938d2ec-6cb6-4f74-882c-06081dbbfb88\") " pod="openshift-cluster-node-tuning-operator/tuned-w7z7d" Jul 06 22:08:43.058080 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.973469 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f2e37c39-7aed-412b-8a5d-2fc1481f707e-host-run-multus-certs\") pod \"multus-gdwzl\" (UID: \"f2e37c39-7aed-412b-8a5d-2fc1481f707e\") " pod="openshift-multus/multus-gdwzl" Jul 06 22:08:43.058080 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.973479 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f2e37c39-7aed-412b-8a5d-2fc1481f707e-host-var-lib-kubelet\") pod \"multus-gdwzl\" (UID: \"f2e37c39-7aed-412b-8a5d-2fc1481f707e\") " pod="openshift-multus/multus-gdwzl" Jul 06 22:08:43.058080 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.973488 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b42224bc-b677-43bf-94fb-fc75223b963a-kubelet-config\") pod \"global-pull-secret-syncer-klc48\" (UID: \"b42224bc-b677-43bf-94fb-fc75223b963a\") " pod="kube-system/global-pull-secret-syncer-klc48" Jul 06 22:08:43.058080 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.973504 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f1de085f-40f2-4f95-a5bb-78aba60de69e-os-release\") pod \"multus-additional-cni-plugins-5nrhc\" (UID: \"f1de085f-40f2-4f95-a5bb-78aba60de69e\") " pod="openshift-multus/multus-additional-cni-plugins-5nrhc" Jul 06 22:08:43.058080 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.973513 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f2e37c39-7aed-412b-8a5d-2fc1481f707e-host-run-multus-certs\") pod \"multus-gdwzl\" (UID: \"f2e37c39-7aed-412b-8a5d-2fc1481f707e\") " pod="openshift-multus/multus-gdwzl" Jul 06 22:08:43.058080 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.973519 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a7a62305-2195-4227-b5b7-5dec03864283-host-kubelet\") pod \"ovnkube-node-vt6d6\" (UID: \"a7a62305-2195-4227-b5b7-5dec03864283\") " pod="openshift-ovn-kubernetes/ovnkube-node-vt6d6" Jul 06 22:08:43.058080 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.973534 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a7a62305-2195-4227-b5b7-5dec03864283-node-log\") pod \"ovnkube-node-vt6d6\" (UID: \"a7a62305-2195-4227-b5b7-5dec03864283\") " pod="openshift-ovn-kubernetes/ovnkube-node-vt6d6" Jul 06 22:08:43.058080 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.973565 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a7a62305-2195-4227-b5b7-5dec03864283-log-socket\") pod \"ovnkube-node-vt6d6\" (UID: \"a7a62305-2195-4227-b5b7-5dec03864283\") " pod="openshift-ovn-kubernetes/ovnkube-node-vt6d6" Jul 06 22:08:43.058080 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.973582 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f2e37c39-7aed-412b-8a5d-2fc1481f707e-host-run-k8s-cni-cncf-io\") pod \"multus-gdwzl\" (UID: \"f2e37c39-7aed-412b-8a5d-2fc1481f707e\") " pod="openshift-multus/multus-gdwzl" Jul 06 22:08:43.058080 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.973617 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b42224bc-b677-43bf-94fb-fc75223b963a-dbus\") pod \"global-pull-secret-syncer-klc48\" (UID: \"b42224bc-b677-43bf-94fb-fc75223b963a\") " pod="kube-system/global-pull-secret-syncer-klc48" Jul 06 22:08:43.058080 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.973637 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f2e37c39-7aed-412b-8a5d-2fc1481f707e-host-run-k8s-cni-cncf-io\") pod \"multus-gdwzl\" (UID: \"f2e37c39-7aed-412b-8a5d-2fc1481f707e\") " pod="openshift-multus/multus-gdwzl" Jul 06 22:08:43.059827 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:08:42.977680 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 22:08:43.059827 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:08:42.977701 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 22:08:43.059827 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:08:42.977712 2581 projected.go:196] Error preparing data for projected volume kube-api-access-tlvkd for pod openshift-network-diagnostics/network-check-target-l5rc4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 22:08:43.059827 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:08:42.977802 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9ade138e-0e88-4087-8042-92fe2db1e1d8-kube-api-access-tlvkd podName:9ade138e-0e88-4087-8042-92fe2db1e1d8 nodeName:}" failed. No retries permitted until 2026-07-06 22:08:43.477790295 +0000 UTC m=+3.036585596 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-tlvkd" (UniqueName: "kubernetes.io/projected/9ade138e-0e88-4087-8042-92fe2db1e1d8-kube-api-access-tlvkd") pod "network-check-target-l5rc4" (UID: "9ade138e-0e88-4087-8042-92fe2db1e1d8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 22:08:43.059827 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.978975 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/314ffdbc-f9e6-45c0-b37a-d216700a1de5-tmp-dir\") pod \"node-resolver-4b7pr\" (UID: \"314ffdbc-f9e6-45c0-b37a-d216700a1de5\") " pod="openshift-dns/node-resolver-4b7pr" Jul 06 22:08:43.059827 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.979009 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/9a1fd480-e89a-4522-b82a-87b8c72124d5-serviceca\") pod \"node-ca-cmtz2\" (UID: \"9a1fd480-e89a-4522-b82a-87b8c72124d5\") " pod="openshift-image-registry/node-ca-cmtz2" Jul 06 22:08:43.059827 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.979055 2581 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 06 22:08:43.059827 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.979092 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e5b109e4-bcab-4c91-8791-3c7fcd077f27-iptables-alerter-script\") pod \"iptables-alerter-fk58k\" (UID: \"e5b109e4-bcab-4c91-8791-3c7fcd077f27\") " pod="openshift-network-operator/iptables-alerter-fk58k" Jul 06 22:08:43.059827 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.979939 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f2e37c39-7aed-412b-8a5d-2fc1481f707e-multus-daemon-config\") pod \"multus-gdwzl\" (UID: \"f2e37c39-7aed-412b-8a5d-2fc1481f707e\") " pod="openshift-multus/multus-gdwzl" Jul 06 22:08:43.059827 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.979983 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f2e37c39-7aed-412b-8a5d-2fc1481f707e-cni-binary-copy\") pod \"multus-gdwzl\" (UID: \"f2e37c39-7aed-412b-8a5d-2fc1481f707e\") " pod="openshift-multus/multus-gdwzl" Jul 06 22:08:43.059827 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.982583 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fdhw6\" (UniqueName: \"kubernetes.io/projected/e5b109e4-bcab-4c91-8791-3c7fcd077f27-kube-api-access-fdhw6\") pod \"iptables-alerter-fk58k\" (UID: \"e5b109e4-bcab-4c91-8791-3c7fcd077f27\") " pod="openshift-network-operator/iptables-alerter-fk58k" Jul 06 22:08:43.059827 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.982606 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w8xnp\" (UniqueName: \"kubernetes.io/projected/9a1fd480-e89a-4522-b82a-87b8c72124d5-kube-api-access-w8xnp\") pod \"node-ca-cmtz2\" (UID: \"9a1fd480-e89a-4522-b82a-87b8c72124d5\") " pod="openshift-image-registry/node-ca-cmtz2" Jul 06 22:08:43.059827 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.982621 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qj46v\" (UniqueName: \"kubernetes.io/projected/314ffdbc-f9e6-45c0-b37a-d216700a1de5-kube-api-access-qj46v\") pod \"node-resolver-4b7pr\" (UID: \"314ffdbc-f9e6-45c0-b37a-d216700a1de5\") " pod="openshift-dns/node-resolver-4b7pr" Jul 06 22:08:43.059827 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.982581 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wlnmw\" (UniqueName: \"kubernetes.io/projected/f2e37c39-7aed-412b-8a5d-2fc1481f707e-kube-api-access-wlnmw\") pod \"multus-gdwzl\" (UID: \"f2e37c39-7aed-412b-8a5d-2fc1481f707e\") " pod="openshift-multus/multus-gdwzl" Jul 06 22:08:43.059827 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:42.982927 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kltv7\" (UniqueName: \"kubernetes.io/projected/8a83eb6f-f275-4562-b014-0b48c710b624-kube-api-access-kltv7\") pod \"network-metrics-daemon-n9hgj\" (UID: \"8a83eb6f-f275-4562-b014-0b48c710b624\") " pod="openshift-multus/network-metrics-daemon-n9hgj" Jul 06 22:08:43.074678 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:43.074652 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f1de085f-40f2-4f95-a5bb-78aba60de69e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-5nrhc\" (UID: \"f1de085f-40f2-4f95-a5bb-78aba60de69e\") " pod="openshift-multus/multus-additional-cni-plugins-5nrhc" Jul 06 22:08:43.074751 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:43.074689 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2041467a-e1c5-4702-96c6-8be98208f3eb-konnectivity-ca\") pod \"konnectivity-agent-66tt7\" (UID: \"2041467a-e1c5-4702-96c6-8be98208f3eb\") " pod="kube-system/konnectivity-agent-66tt7" Jul 06 22:08:43.074751 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:43.074705 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0938d2ec-6cb6-4f74-882c-06081dbbfb88-lib-modules\") pod \"tuned-w7z7d\" (UID: \"0938d2ec-6cb6-4f74-882c-06081dbbfb88\") " pod="openshift-cluster-node-tuning-operator/tuned-w7z7d" Jul 06 22:08:43.074751 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:43.074722 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/0938d2ec-6cb6-4f74-882c-06081dbbfb88-etc-tuned\") pod \"tuned-w7z7d\" (UID: \"0938d2ec-6cb6-4f74-882c-06081dbbfb88\") " pod="openshift-cluster-node-tuning-operator/tuned-w7z7d" Jul 06 22:08:43.074751 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:43.074744 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a7a62305-2195-4227-b5b7-5dec03864283-etc-openvswitch\") pod \"ovnkube-node-vt6d6\" (UID: \"a7a62305-2195-4227-b5b7-5dec03864283\") " pod="openshift-ovn-kubernetes/ovnkube-node-vt6d6" Jul 06 22:08:43.074947 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:43.074822 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a7a62305-2195-4227-b5b7-5dec03864283-etc-openvswitch\") pod \"ovnkube-node-vt6d6\" (UID: \"a7a62305-2195-4227-b5b7-5dec03864283\") " pod="openshift-ovn-kubernetes/ovnkube-node-vt6d6" Jul 06 22:08:43.074947 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:43.074845 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a7a62305-2195-4227-b5b7-5dec03864283-host-cni-bin\") pod \"ovnkube-node-vt6d6\" (UID: \"a7a62305-2195-4227-b5b7-5dec03864283\") " pod="openshift-ovn-kubernetes/ovnkube-node-vt6d6" Jul 06 22:08:43.074947 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:43.074888 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/8ef94701-6fd1-4b4d-aa83-e6effd1023bf-socket-dir\") pod \"aws-ebs-csi-driver-node-st84l\" (UID: \"8ef94701-6fd1-4b4d-aa83-e6effd1023bf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-st84l" Jul 06 22:08:43.074947 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:43.074909 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/a7a62305-2195-4227-b5b7-5dec03864283-host-cni-bin\") pod \"ovnkube-node-vt6d6\" (UID: \"a7a62305-2195-4227-b5b7-5dec03864283\") " pod="openshift-ovn-kubernetes/ovnkube-node-vt6d6" Jul 06 22:08:43.074947 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:43.074939 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0938d2ec-6cb6-4f74-882c-06081dbbfb88-lib-modules\") pod \"tuned-w7z7d\" (UID: \"0938d2ec-6cb6-4f74-882c-06081dbbfb88\") " pod="openshift-cluster-node-tuning-operator/tuned-w7z7d" Jul 06 22:08:43.075203 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:43.074941 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/0938d2ec-6cb6-4f74-882c-06081dbbfb88-etc-sysctl-d\") pod \"tuned-w7z7d\" (UID: \"0938d2ec-6cb6-4f74-882c-06081dbbfb88\") " pod="openshift-cluster-node-tuning-operator/tuned-w7z7d" Jul 06 22:08:43.075203 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:43.074982 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/0938d2ec-6cb6-4f74-882c-06081dbbfb88-run\") pod \"tuned-w7z7d\" (UID: \"0938d2ec-6cb6-4f74-882c-06081dbbfb88\") " pod="openshift-cluster-node-tuning-operator/tuned-w7z7d" Jul 06 22:08:43.075203 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:43.075028 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f1de085f-40f2-4f95-a5bb-78aba60de69e-cnibin\") pod \"multus-additional-cni-plugins-5nrhc\" (UID: \"f1de085f-40f2-4f95-a5bb-78aba60de69e\") " pod="openshift-multus/multus-additional-cni-plugins-5nrhc" Jul 06 22:08:43.075203 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:43.075055 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/8ef94701-6fd1-4b4d-aa83-e6effd1023bf-etc-selinux\") pod \"aws-ebs-csi-driver-node-st84l\" (UID: \"8ef94701-6fd1-4b4d-aa83-e6effd1023bf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-st84l" Jul 06 22:08:43.075203 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:43.075081 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9jchz\" (UniqueName: \"kubernetes.io/projected/8ef94701-6fd1-4b4d-aa83-e6effd1023bf-kube-api-access-9jchz\") pod \"aws-ebs-csi-driver-node-st84l\" (UID: \"8ef94701-6fd1-4b4d-aa83-e6effd1023bf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-st84l" Jul 06 22:08:43.075203 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:43.075101 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/0938d2ec-6cb6-4f74-882c-06081dbbfb88-etc-sysctl-d\") pod \"tuned-w7z7d\" (UID: \"0938d2ec-6cb6-4f74-882c-06081dbbfb88\") " pod="openshift-cluster-node-tuning-operator/tuned-w7z7d" Jul 06 22:08:43.075203 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:43.075123 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0938d2ec-6cb6-4f74-882c-06081dbbfb88-sys\") pod \"tuned-w7z7d\" (UID: \"0938d2ec-6cb6-4f74-882c-06081dbbfb88\") " pod="openshift-cluster-node-tuning-operator/tuned-w7z7d" Jul 06 22:08:43.075203 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:43.075127 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/0938d2ec-6cb6-4f74-882c-06081dbbfb88-run\") pod \"tuned-w7z7d\" (UID: \"0938d2ec-6cb6-4f74-882c-06081dbbfb88\") " pod="openshift-cluster-node-tuning-operator/tuned-w7z7d" Jul 06 22:08:43.075203 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:43.075155 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a7a62305-2195-4227-b5b7-5dec03864283-run-openvswitch\") pod \"ovnkube-node-vt6d6\" (UID: \"a7a62305-2195-4227-b5b7-5dec03864283\") " pod="openshift-ovn-kubernetes/ovnkube-node-vt6d6" Jul 06 22:08:43.075203 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:43.075187 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tvhkv\" (UniqueName: \"kubernetes.io/projected/a7a62305-2195-4227-b5b7-5dec03864283-kube-api-access-tvhkv\") pod \"ovnkube-node-vt6d6\" (UID: \"a7a62305-2195-4227-b5b7-5dec03864283\") " pod="openshift-ovn-kubernetes/ovnkube-node-vt6d6" Jul 06 22:08:43.075736 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:43.075219 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/8ef94701-6fd1-4b4d-aa83-e6effd1023bf-kubelet-dir\") pod \"aws-ebs-csi-driver-node-st84l\" (UID: \"8ef94701-6fd1-4b4d-aa83-e6effd1023bf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-st84l" Jul 06 22:08:43.075736 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:43.075223 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/8ef94701-6fd1-4b4d-aa83-e6effd1023bf-etc-selinux\") pod \"aws-ebs-csi-driver-node-st84l\" (UID: \"8ef94701-6fd1-4b4d-aa83-e6effd1023bf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-st84l" Jul 06 22:08:43.075736 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:43.075250 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dgtg9\" (UniqueName: \"kubernetes.io/projected/0938d2ec-6cb6-4f74-882c-06081dbbfb88-kube-api-access-dgtg9\") pod \"tuned-w7z7d\" (UID: \"0938d2ec-6cb6-4f74-882c-06081dbbfb88\") " pod="openshift-cluster-node-tuning-operator/tuned-w7z7d" Jul 06 22:08:43.075736 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:43.075275 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b42224bc-b677-43bf-94fb-fc75223b963a-kubelet-config\") pod \"global-pull-secret-syncer-klc48\" (UID: \"b42224bc-b677-43bf-94fb-fc75223b963a\") " pod="kube-system/global-pull-secret-syncer-klc48" Jul 06 22:08:43.075736 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:43.075283 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a7a62305-2195-4227-b5b7-5dec03864283-run-openvswitch\") pod \"ovnkube-node-vt6d6\" (UID: \"a7a62305-2195-4227-b5b7-5dec03864283\") " pod="openshift-ovn-kubernetes/ovnkube-node-vt6d6" Jul 06 22:08:43.075736 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:43.075321 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f1de085f-40f2-4f95-a5bb-78aba60de69e-os-release\") pod \"multus-additional-cni-plugins-5nrhc\" (UID: \"f1de085f-40f2-4f95-a5bb-78aba60de69e\") " pod="openshift-multus/multus-additional-cni-plugins-5nrhc" Jul 06 22:08:43.075736 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:43.075346 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0938d2ec-6cb6-4f74-882c-06081dbbfb88-sys\") pod \"tuned-w7z7d\" (UID: \"0938d2ec-6cb6-4f74-882c-06081dbbfb88\") " pod="openshift-cluster-node-tuning-operator/tuned-w7z7d" Jul 06 22:08:43.075736 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:43.075349 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a7a62305-2195-4227-b5b7-5dec03864283-host-kubelet\") pod \"ovnkube-node-vt6d6\" (UID: \"a7a62305-2195-4227-b5b7-5dec03864283\") " pod="openshift-ovn-kubernetes/ovnkube-node-vt6d6" Jul 06 22:08:43.075736 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:43.075392 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a7a62305-2195-4227-b5b7-5dec03864283-node-log\") pod \"ovnkube-node-vt6d6\" (UID: \"a7a62305-2195-4227-b5b7-5dec03864283\") " pod="openshift-ovn-kubernetes/ovnkube-node-vt6d6" Jul 06 22:08:43.075736 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:43.075400 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/8ef94701-6fd1-4b4d-aa83-e6effd1023bf-kubelet-dir\") pod \"aws-ebs-csi-driver-node-st84l\" (UID: \"8ef94701-6fd1-4b4d-aa83-e6effd1023bf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-st84l" Jul 06 22:08:43.075736 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:43.075434 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a7a62305-2195-4227-b5b7-5dec03864283-log-socket\") pod \"ovnkube-node-vt6d6\" (UID: \"a7a62305-2195-4227-b5b7-5dec03864283\") " pod="openshift-ovn-kubernetes/ovnkube-node-vt6d6" Jul 06 22:08:43.075736 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:43.075460 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b42224bc-b677-43bf-94fb-fc75223b963a-kubelet-config\") pod \"global-pull-secret-syncer-klc48\" (UID: \"b42224bc-b677-43bf-94fb-fc75223b963a\") " pod="kube-system/global-pull-secret-syncer-klc48" Jul 06 22:08:43.075736 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:43.075507 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f1de085f-40f2-4f95-a5bb-78aba60de69e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-5nrhc\" (UID: \"f1de085f-40f2-4f95-a5bb-78aba60de69e\") " pod="openshift-multus/multus-additional-cni-plugins-5nrhc" Jul 06 22:08:43.075736 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:43.075528 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2041467a-e1c5-4702-96c6-8be98208f3eb-konnectivity-ca\") pod \"konnectivity-agent-66tt7\" (UID: \"2041467a-e1c5-4702-96c6-8be98208f3eb\") " pod="kube-system/konnectivity-agent-66tt7" Jul 06 22:08:43.075736 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:43.075591 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/a7a62305-2195-4227-b5b7-5dec03864283-node-log\") pod \"ovnkube-node-vt6d6\" (UID: \"a7a62305-2195-4227-b5b7-5dec03864283\") " pod="openshift-ovn-kubernetes/ovnkube-node-vt6d6" Jul 06 22:08:43.075736 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:43.075464 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b42224bc-b677-43bf-94fb-fc75223b963a-dbus\") pod \"global-pull-secret-syncer-klc48\" (UID: \"b42224bc-b677-43bf-94fb-fc75223b963a\") " pod="kube-system/global-pull-secret-syncer-klc48" Jul 06 22:08:43.075736 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:43.075639 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f1de085f-40f2-4f95-a5bb-78aba60de69e-os-release\") pod \"multus-additional-cni-plugins-5nrhc\" (UID: \"f1de085f-40f2-4f95-a5bb-78aba60de69e\") " pod="openshift-multus/multus-additional-cni-plugins-5nrhc" Jul 06 22:08:43.076542 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:43.075655 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/a7a62305-2195-4227-b5b7-5dec03864283-host-kubelet\") pod \"ovnkube-node-vt6d6\" (UID: \"a7a62305-2195-4227-b5b7-5dec03864283\") " pod="openshift-ovn-kubernetes/ovnkube-node-vt6d6" Jul 06 22:08:43.076542 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:43.075608 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b42224bc-b677-43bf-94fb-fc75223b963a-dbus\") pod \"global-pull-secret-syncer-klc48\" (UID: \"b42224bc-b677-43bf-94fb-fc75223b963a\") " pod="kube-system/global-pull-secret-syncer-klc48" Jul 06 22:08:43.076542 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:43.075657 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/a7a62305-2195-4227-b5b7-5dec03864283-log-socket\") pod \"ovnkube-node-vt6d6\" (UID: \"a7a62305-2195-4227-b5b7-5dec03864283\") " pod="openshift-ovn-kubernetes/ovnkube-node-vt6d6" Jul 06 22:08:43.076542 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:43.075698 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/8ef94701-6fd1-4b4d-aa83-e6effd1023bf-socket-dir\") pod \"aws-ebs-csi-driver-node-st84l\" (UID: \"8ef94701-6fd1-4b4d-aa83-e6effd1023bf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-st84l" Jul 06 22:08:43.076542 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:43.075717 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f1de085f-40f2-4f95-a5bb-78aba60de69e-cnibin\") pod \"multus-additional-cni-plugins-5nrhc\" (UID: \"f1de085f-40f2-4f95-a5bb-78aba60de69e\") " pod="openshift-multus/multus-additional-cni-plugins-5nrhc" Jul 06 22:08:43.076542 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:43.075670 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f1de085f-40f2-4f95-a5bb-78aba60de69e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-5nrhc\" (UID: \"f1de085f-40f2-4f95-a5bb-78aba60de69e\") " pod="openshift-multus/multus-additional-cni-plugins-5nrhc" Jul 06 22:08:43.076542 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:43.075781 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a7a62305-2195-4227-b5b7-5dec03864283-run-ovn\") pod \"ovnkube-node-vt6d6\" (UID: \"a7a62305-2195-4227-b5b7-5dec03864283\") " pod="openshift-ovn-kubernetes/ovnkube-node-vt6d6" Jul 06 22:08:43.076542 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:43.075818 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a7a62305-2195-4227-b5b7-5dec03864283-env-overrides\") pod \"ovnkube-node-vt6d6\" (UID: \"a7a62305-2195-4227-b5b7-5dec03864283\") " pod="openshift-ovn-kubernetes/ovnkube-node-vt6d6" Jul 06 22:08:43.076542 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:43.075847 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/0938d2ec-6cb6-4f74-882c-06081dbbfb88-etc-modprobe-d\") pod \"tuned-w7z7d\" (UID: \"0938d2ec-6cb6-4f74-882c-06081dbbfb88\") " pod="openshift-cluster-node-tuning-operator/tuned-w7z7d" Jul 06 22:08:43.076542 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:43.075873 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/0938d2ec-6cb6-4f74-882c-06081dbbfb88-etc-sysctl-conf\") pod \"tuned-w7z7d\" (UID: \"0938d2ec-6cb6-4f74-882c-06081dbbfb88\") " pod="openshift-cluster-node-tuning-operator/tuned-w7z7d" Jul 06 22:08:43.076542 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:43.075878 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/a7a62305-2195-4227-b5b7-5dec03864283-run-ovn\") pod \"ovnkube-node-vt6d6\" (UID: \"a7a62305-2195-4227-b5b7-5dec03864283\") " pod="openshift-ovn-kubernetes/ovnkube-node-vt6d6" Jul 06 22:08:43.076542 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:43.075904 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a7a62305-2195-4227-b5b7-5dec03864283-host-slash\") pod \"ovnkube-node-vt6d6\" (UID: \"a7a62305-2195-4227-b5b7-5dec03864283\") " pod="openshift-ovn-kubernetes/ovnkube-node-vt6d6" Jul 06 22:08:43.076542 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:43.075949 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a7a62305-2195-4227-b5b7-5dec03864283-host-slash\") pod \"ovnkube-node-vt6d6\" (UID: \"a7a62305-2195-4227-b5b7-5dec03864283\") " pod="openshift-ovn-kubernetes/ovnkube-node-vt6d6" Jul 06 22:08:43.076542 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:43.075963 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/0938d2ec-6cb6-4f74-882c-06081dbbfb88-etc-modprobe-d\") pod \"tuned-w7z7d\" (UID: \"0938d2ec-6cb6-4f74-882c-06081dbbfb88\") " pod="openshift-cluster-node-tuning-operator/tuned-w7z7d" Jul 06 22:08:43.076542 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:43.075981 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a7a62305-2195-4227-b5b7-5dec03864283-host-run-netns\") pod \"ovnkube-node-vt6d6\" (UID: \"a7a62305-2195-4227-b5b7-5dec03864283\") " pod="openshift-ovn-kubernetes/ovnkube-node-vt6d6" Jul 06 22:08:43.076542 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:43.076006 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a7a62305-2195-4227-b5b7-5dec03864283-ovnkube-config\") pod \"ovnkube-node-vt6d6\" (UID: \"a7a62305-2195-4227-b5b7-5dec03864283\") " pod="openshift-ovn-kubernetes/ovnkube-node-vt6d6" Jul 06 22:08:43.076542 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:43.076041 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/0938d2ec-6cb6-4f74-882c-06081dbbfb88-etc-sysctl-conf\") pod \"tuned-w7z7d\" (UID: \"0938d2ec-6cb6-4f74-882c-06081dbbfb88\") " pod="openshift-cluster-node-tuning-operator/tuned-w7z7d" Jul 06 22:08:43.077367 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:43.076079 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/8ef94701-6fd1-4b4d-aa83-e6effd1023bf-registration-dir\") pod \"aws-ebs-csi-driver-node-st84l\" (UID: \"8ef94701-6fd1-4b4d-aa83-e6effd1023bf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-st84l" Jul 06 22:08:43.077367 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:43.076109 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/a7a62305-2195-4227-b5b7-5dec03864283-host-run-netns\") pod \"ovnkube-node-vt6d6\" (UID: \"a7a62305-2195-4227-b5b7-5dec03864283\") " pod="openshift-ovn-kubernetes/ovnkube-node-vt6d6" Jul 06 22:08:43.077367 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:43.076176 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b42224bc-b677-43bf-94fb-fc75223b963a-original-pull-secret\") pod \"global-pull-secret-syncer-klc48\" (UID: \"b42224bc-b677-43bf-94fb-fc75223b963a\") " pod="kube-system/global-pull-secret-syncer-klc48" Jul 06 22:08:43.077367 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:43.076247 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/0938d2ec-6cb6-4f74-882c-06081dbbfb88-etc-systemd\") pod \"tuned-w7z7d\" (UID: \"0938d2ec-6cb6-4f74-882c-06081dbbfb88\") " pod="openshift-cluster-node-tuning-operator/tuned-w7z7d" Jul 06 22:08:43.077367 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:43.076667 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f1de085f-40f2-4f95-a5bb-78aba60de69e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-5nrhc\" (UID: \"f1de085f-40f2-4f95-a5bb-78aba60de69e\") " pod="openshift-multus/multus-additional-cni-plugins-5nrhc" Jul 06 22:08:43.077367 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:43.076835 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nvnzj\" (UniqueName: \"kubernetes.io/projected/f1de085f-40f2-4f95-a5bb-78aba60de69e-kube-api-access-nvnzj\") pod \"multus-additional-cni-plugins-5nrhc\" (UID: \"f1de085f-40f2-4f95-a5bb-78aba60de69e\") " pod="openshift-multus/multus-additional-cni-plugins-5nrhc" Jul 06 22:08:43.077367 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:43.076915 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a7a62305-2195-4227-b5b7-5dec03864283-host-cni-netd\") pod \"ovnkube-node-vt6d6\" (UID: \"a7a62305-2195-4227-b5b7-5dec03864283\") " pod="openshift-ovn-kubernetes/ovnkube-node-vt6d6" Jul 06 22:08:43.077367 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:43.076968 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a7a62305-2195-4227-b5b7-5dec03864283-run-systemd\") pod \"ovnkube-node-vt6d6\" (UID: \"a7a62305-2195-4227-b5b7-5dec03864283\") " pod="openshift-ovn-kubernetes/ovnkube-node-vt6d6" Jul 06 22:08:43.077367 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:43.077007 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/8ef94701-6fd1-4b4d-aa83-e6effd1023bf-sys-fs\") pod \"aws-ebs-csi-driver-node-st84l\" (UID: \"8ef94701-6fd1-4b4d-aa83-e6effd1023bf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-st84l" Jul 06 22:08:43.077367 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:43.077044 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f1de085f-40f2-4f95-a5bb-78aba60de69e-cni-binary-copy\") pod \"multus-additional-cni-plugins-5nrhc\" (UID: \"f1de085f-40f2-4f95-a5bb-78aba60de69e\") " pod="openshift-multus/multus-additional-cni-plugins-5nrhc" Jul 06 22:08:43.077367 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:43.077077 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f1de085f-40f2-4f95-a5bb-78aba60de69e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-5nrhc\" (UID: \"f1de085f-40f2-4f95-a5bb-78aba60de69e\") " pod="openshift-multus/multus-additional-cni-plugins-5nrhc" Jul 06 22:08:43.077367 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:43.077162 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2041467a-e1c5-4702-96c6-8be98208f3eb-agent-certs\") pod \"konnectivity-agent-66tt7\" (UID: \"2041467a-e1c5-4702-96c6-8be98208f3eb\") " pod="kube-system/konnectivity-agent-66tt7" Jul 06 22:08:43.077367 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:43.077220 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a7a62305-2195-4227-b5b7-5dec03864283-var-lib-openvswitch\") pod \"ovnkube-node-vt6d6\" (UID: \"a7a62305-2195-4227-b5b7-5dec03864283\") " pod="openshift-ovn-kubernetes/ovnkube-node-vt6d6" Jul 06 22:08:43.077367 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:43.077268 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a7a62305-2195-4227-b5b7-5dec03864283-ovn-node-metrics-cert\") pod \"ovnkube-node-vt6d6\" (UID: \"a7a62305-2195-4227-b5b7-5dec03864283\") " pod="openshift-ovn-kubernetes/ovnkube-node-vt6d6" Jul 06 22:08:43.077367 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:43.077307 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0938d2ec-6cb6-4f74-882c-06081dbbfb88-var-lib-kubelet\") pod \"tuned-w7z7d\" (UID: \"0938d2ec-6cb6-4f74-882c-06081dbbfb88\") " pod="openshift-cluster-node-tuning-operator/tuned-w7z7d" Jul 06 22:08:43.077367 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:43.077334 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0938d2ec-6cb6-4f74-882c-06081dbbfb88-host\") pod \"tuned-w7z7d\" (UID: \"0938d2ec-6cb6-4f74-882c-06081dbbfb88\") " pod="openshift-cluster-node-tuning-operator/tuned-w7z7d" Jul 06 22:08:43.078063 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:43.077412 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0938d2ec-6cb6-4f74-882c-06081dbbfb88-host\") pod \"tuned-w7z7d\" (UID: \"0938d2ec-6cb6-4f74-882c-06081dbbfb88\") " pod="openshift-cluster-node-tuning-operator/tuned-w7z7d" Jul 06 22:08:43.078063 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:43.077411 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/8ef94701-6fd1-4b4d-aa83-e6effd1023bf-registration-dir\") pod \"aws-ebs-csi-driver-node-st84l\" (UID: \"8ef94701-6fd1-4b4d-aa83-e6effd1023bf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-st84l" Jul 06 22:08:43.078063 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:43.077517 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/a7a62305-2195-4227-b5b7-5dec03864283-host-cni-netd\") pod \"ovnkube-node-vt6d6\" (UID: \"a7a62305-2195-4227-b5b7-5dec03864283\") " pod="openshift-ovn-kubernetes/ovnkube-node-vt6d6" Jul 06 22:08:43.078063 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:43.077596 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/a7a62305-2195-4227-b5b7-5dec03864283-run-systemd\") pod \"ovnkube-node-vt6d6\" (UID: \"a7a62305-2195-4227-b5b7-5dec03864283\") " pod="openshift-ovn-kubernetes/ovnkube-node-vt6d6" Jul 06 22:08:43.078063 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:43.077656 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/8ef94701-6fd1-4b4d-aa83-e6effd1023bf-sys-fs\") pod \"aws-ebs-csi-driver-node-st84l\" (UID: \"8ef94701-6fd1-4b4d-aa83-e6effd1023bf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-st84l" Jul 06 22:08:43.078063 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:43.077802 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/a7a62305-2195-4227-b5b7-5dec03864283-env-overrides\") pod \"ovnkube-node-vt6d6\" (UID: \"a7a62305-2195-4227-b5b7-5dec03864283\") " pod="openshift-ovn-kubernetes/ovnkube-node-vt6d6" Jul 06 22:08:43.078063 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:43.077890 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/a7a62305-2195-4227-b5b7-5dec03864283-var-lib-openvswitch\") pod \"ovnkube-node-vt6d6\" (UID: \"a7a62305-2195-4227-b5b7-5dec03864283\") " pod="openshift-ovn-kubernetes/ovnkube-node-vt6d6" Jul 06 22:08:43.078063 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:43.077940 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/a7a62305-2195-4227-b5b7-5dec03864283-ovnkube-config\") pod \"ovnkube-node-vt6d6\" (UID: \"a7a62305-2195-4227-b5b7-5dec03864283\") " pod="openshift-ovn-kubernetes/ovnkube-node-vt6d6" Jul 06 22:08:43.078454 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:43.078064 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f1de085f-40f2-4f95-a5bb-78aba60de69e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-5nrhc\" (UID: \"f1de085f-40f2-4f95-a5bb-78aba60de69e\") " pod="openshift-multus/multus-additional-cni-plugins-5nrhc" Jul 06 22:08:43.078454 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:43.078135 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/0938d2ec-6cb6-4f74-882c-06081dbbfb88-etc-systemd\") pod \"tuned-w7z7d\" (UID: \"0938d2ec-6cb6-4f74-882c-06081dbbfb88\") " pod="openshift-cluster-node-tuning-operator/tuned-w7z7d" Jul 06 22:08:43.078454 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:43.078286 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f1de085f-40f2-4f95-a5bb-78aba60de69e-cni-binary-copy\") pod \"multus-additional-cni-plugins-5nrhc\" (UID: \"f1de085f-40f2-4f95-a5bb-78aba60de69e\") " pod="openshift-multus/multus-additional-cni-plugins-5nrhc" Jul 06 22:08:43.078454 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:43.078340 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0938d2ec-6cb6-4f74-882c-06081dbbfb88-var-lib-kubelet\") pod \"tuned-w7z7d\" (UID: \"0938d2ec-6cb6-4f74-882c-06081dbbfb88\") " pod="openshift-cluster-node-tuning-operator/tuned-w7z7d" Jul 06 22:08:43.078642 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:43.078561 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/0938d2ec-6cb6-4f74-882c-06081dbbfb88-etc-tuned\") pod \"tuned-w7z7d\" (UID: \"0938d2ec-6cb6-4f74-882c-06081dbbfb88\") " pod="openshift-cluster-node-tuning-operator/tuned-w7z7d" Jul 06 22:08:43.078642 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:08:43.078589 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 22:08:43.078642 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:43.078608 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0938d2ec-6cb6-4f74-882c-06081dbbfb88-tmp\") pod \"tuned-w7z7d\" (UID: \"0938d2ec-6cb6-4f74-882c-06081dbbfb88\") " pod="openshift-cluster-node-tuning-operator/tuned-w7z7d" Jul 06 22:08:43.078851 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:43.078649 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a7a62305-2195-4227-b5b7-5dec03864283-host-run-ovn-kubernetes\") pod \"ovnkube-node-vt6d6\" (UID: \"a7a62305-2195-4227-b5b7-5dec03864283\") " pod="openshift-ovn-kubernetes/ovnkube-node-vt6d6" Jul 06 22:08:43.078851 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:43.078691 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/8ef94701-6fd1-4b4d-aa83-e6effd1023bf-device-dir\") pod \"aws-ebs-csi-driver-node-st84l\" (UID: \"8ef94701-6fd1-4b4d-aa83-e6effd1023bf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-st84l" Jul 06 22:08:43.078851 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:43.078728 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f1de085f-40f2-4f95-a5bb-78aba60de69e-system-cni-dir\") pod \"multus-additional-cni-plugins-5nrhc\" (UID: \"f1de085f-40f2-4f95-a5bb-78aba60de69e\") " pod="openshift-multus/multus-additional-cni-plugins-5nrhc" Jul 06 22:08:43.078851 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:43.078769 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a7a62305-2195-4227-b5b7-5dec03864283-ovnkube-script-lib\") pod \"ovnkube-node-vt6d6\" (UID: \"a7a62305-2195-4227-b5b7-5dec03864283\") " pod="openshift-ovn-kubernetes/ovnkube-node-vt6d6" Jul 06 22:08:43.078851 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:43.078834 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a7a62305-2195-4227-b5b7-5dec03864283-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vt6d6\" (UID: \"a7a62305-2195-4227-b5b7-5dec03864283\") " pod="openshift-ovn-kubernetes/ovnkube-node-vt6d6" Jul 06 22:08:43.079089 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:43.078945 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a7a62305-2195-4227-b5b7-5dec03864283-host-run-ovn-kubernetes\") pod \"ovnkube-node-vt6d6\" (UID: \"a7a62305-2195-4227-b5b7-5dec03864283\") " pod="openshift-ovn-kubernetes/ovnkube-node-vt6d6" Jul 06 22:08:43.079089 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:43.078862 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/0938d2ec-6cb6-4f74-882c-06081dbbfb88-etc-sysconfig\") pod \"tuned-w7z7d\" (UID: \"0938d2ec-6cb6-4f74-882c-06081dbbfb88\") " pod="openshift-cluster-node-tuning-operator/tuned-w7z7d" Jul 06 22:08:43.079089 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:08:43.079032 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b42224bc-b677-43bf-94fb-fc75223b963a-original-pull-secret podName:b42224bc-b677-43bf-94fb-fc75223b963a nodeName:}" failed. No retries permitted until 2026-07-06 22:08:43.578981355 +0000 UTC m=+3.137776675 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b42224bc-b677-43bf-94fb-fc75223b963a-original-pull-secret") pod "global-pull-secret-syncer-klc48" (UID: "b42224bc-b677-43bf-94fb-fc75223b963a") : object "kube-system"/"original-pull-secret" not registered Jul 06 22:08:43.079089 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:43.079063 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f1de085f-40f2-4f95-a5bb-78aba60de69e-system-cni-dir\") pod \"multus-additional-cni-plugins-5nrhc\" (UID: \"f1de085f-40f2-4f95-a5bb-78aba60de69e\") " pod="openshift-multus/multus-additional-cni-plugins-5nrhc" Jul 06 22:08:43.079089 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:43.079077 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0938d2ec-6cb6-4f74-882c-06081dbbfb88-etc-kubernetes\") pod \"tuned-w7z7d\" (UID: \"0938d2ec-6cb6-4f74-882c-06081dbbfb88\") " pod="openshift-cluster-node-tuning-operator/tuned-w7z7d" Jul 06 22:08:43.079334 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:43.079114 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a7a62305-2195-4227-b5b7-5dec03864283-systemd-units\") pod \"ovnkube-node-vt6d6\" (UID: \"a7a62305-2195-4227-b5b7-5dec03864283\") " pod="openshift-ovn-kubernetes/ovnkube-node-vt6d6" Jul 06 22:08:43.079334 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:43.079165 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/8ef94701-6fd1-4b4d-aa83-e6effd1023bf-device-dir\") pod \"aws-ebs-csi-driver-node-st84l\" (UID: \"8ef94701-6fd1-4b4d-aa83-e6effd1023bf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-st84l" Jul 06 22:08:43.079334 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:43.079243 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/0938d2ec-6cb6-4f74-882c-06081dbbfb88-etc-sysconfig\") pod \"tuned-w7z7d\" (UID: \"0938d2ec-6cb6-4f74-882c-06081dbbfb88\") " pod="openshift-cluster-node-tuning-operator/tuned-w7z7d" Jul 06 22:08:43.079474 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:43.079349 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/a7a62305-2195-4227-b5b7-5dec03864283-systemd-units\") pod \"ovnkube-node-vt6d6\" (UID: \"a7a62305-2195-4227-b5b7-5dec03864283\") " pod="openshift-ovn-kubernetes/ovnkube-node-vt6d6" Jul 06 22:08:43.079524 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:43.079460 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0938d2ec-6cb6-4f74-882c-06081dbbfb88-etc-kubernetes\") pod \"tuned-w7z7d\" (UID: \"0938d2ec-6cb6-4f74-882c-06081dbbfb88\") " pod="openshift-cluster-node-tuning-operator/tuned-w7z7d" Jul 06 22:08:43.079571 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:43.079532 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a7a62305-2195-4227-b5b7-5dec03864283-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vt6d6\" (UID: \"a7a62305-2195-4227-b5b7-5dec03864283\") " pod="openshift-ovn-kubernetes/ovnkube-node-vt6d6" Jul 06 22:08:43.081688 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:43.079691 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/a7a62305-2195-4227-b5b7-5dec03864283-ovnkube-script-lib\") pod \"ovnkube-node-vt6d6\" (UID: \"a7a62305-2195-4227-b5b7-5dec03864283\") " pod="openshift-ovn-kubernetes/ovnkube-node-vt6d6" Jul 06 22:08:43.082517 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:43.082491 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2041467a-e1c5-4702-96c6-8be98208f3eb-agent-certs\") pod \"konnectivity-agent-66tt7\" (UID: \"2041467a-e1c5-4702-96c6-8be98208f3eb\") " pod="kube-system/konnectivity-agent-66tt7" Jul 06 22:08:43.082630 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:43.082490 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/a7a62305-2195-4227-b5b7-5dec03864283-ovn-node-metrics-cert\") pod \"ovnkube-node-vt6d6\" (UID: \"a7a62305-2195-4227-b5b7-5dec03864283\") " pod="openshift-ovn-kubernetes/ovnkube-node-vt6d6" Jul 06 22:08:43.083380 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:43.083352 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0938d2ec-6cb6-4f74-882c-06081dbbfb88-tmp\") pod \"tuned-w7z7d\" (UID: \"0938d2ec-6cb6-4f74-882c-06081dbbfb88\") " pod="openshift-cluster-node-tuning-operator/tuned-w7z7d" Jul 06 22:08:43.084823 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:43.084793 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tvhkv\" (UniqueName: \"kubernetes.io/projected/a7a62305-2195-4227-b5b7-5dec03864283-kube-api-access-tvhkv\") pod \"ovnkube-node-vt6d6\" (UID: \"a7a62305-2195-4227-b5b7-5dec03864283\") " pod="openshift-ovn-kubernetes/ovnkube-node-vt6d6" Jul 06 22:08:43.085007 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:43.084985 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9jchz\" (UniqueName: \"kubernetes.io/projected/8ef94701-6fd1-4b4d-aa83-e6effd1023bf-kube-api-access-9jchz\") pod \"aws-ebs-csi-driver-node-st84l\" (UID: \"8ef94701-6fd1-4b4d-aa83-e6effd1023bf\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-st84l" Jul 06 22:08:43.085220 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:43.085205 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dgtg9\" (UniqueName: \"kubernetes.io/projected/0938d2ec-6cb6-4f74-882c-06081dbbfb88-kube-api-access-dgtg9\") pod \"tuned-w7z7d\" (UID: \"0938d2ec-6cb6-4f74-882c-06081dbbfb88\") " pod="openshift-cluster-node-tuning-operator/tuned-w7z7d" Jul 06 22:08:43.086573 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:43.086556 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nvnzj\" (UniqueName: \"kubernetes.io/projected/f1de085f-40f2-4f95-a5bb-78aba60de69e-kube-api-access-nvnzj\") pod \"multus-additional-cni-plugins-5nrhc\" (UID: \"f1de085f-40f2-4f95-a5bb-78aba60de69e\") " pod="openshift-multus/multus-additional-cni-plugins-5nrhc" Jul 06 22:08:43.216426 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:43.216399 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-fk58k" Jul 06 22:08:43.233992 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:43.233966 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-4b7pr" Jul 06 22:08:43.255853 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:43.255829 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-cmtz2" Jul 06 22:08:43.261908 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:43.261879 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-gdwzl" Jul 06 22:08:43.293809 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:43.293787 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-66tt7" Jul 06 22:08:43.309847 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:43.309825 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-5nrhc" Jul 06 22:08:43.316109 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:43.316076 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-vt6d6" Jul 06 22:08:43.331938 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:43.331921 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-st84l" Jul 06 22:08:43.336997 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:43.336980 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-w7z7d" Jul 06 22:08:43.483138 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:43.483051 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tlvkd\" (UniqueName: \"kubernetes.io/projected/9ade138e-0e88-4087-8042-92fe2db1e1d8-kube-api-access-tlvkd\") pod \"network-check-target-l5rc4\" (UID: \"9ade138e-0e88-4087-8042-92fe2db1e1d8\") " pod="openshift-network-diagnostics/network-check-target-l5rc4" Jul 06 22:08:43.483278 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:08:43.483205 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 22:08:43.483278 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:08:43.483228 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 22:08:43.483278 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:08:43.483241 2581 projected.go:196] Error preparing data for projected volume kube-api-access-tlvkd for pod openshift-network-diagnostics/network-check-target-l5rc4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 22:08:43.483278 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:08:43.483274 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 22:08:43.483419 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:08:43.483298 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9ade138e-0e88-4087-8042-92fe2db1e1d8-kube-api-access-tlvkd podName:9ade138e-0e88-4087-8042-92fe2db1e1d8 nodeName:}" failed. No retries permitted until 2026-07-06 22:08:44.483277989 +0000 UTC m=+4.042073313 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-tlvkd" (UniqueName: "kubernetes.io/projected/9ade138e-0e88-4087-8042-92fe2db1e1d8-kube-api-access-tlvkd") pod "network-check-target-l5rc4" (UID: "9ade138e-0e88-4087-8042-92fe2db1e1d8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 22:08:43.483419 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:43.483204 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8a83eb6f-f275-4562-b014-0b48c710b624-metrics-certs\") pod \"network-metrics-daemon-n9hgj\" (UID: \"8a83eb6f-f275-4562-b014-0b48c710b624\") " pod="openshift-multus/network-metrics-daemon-n9hgj" Jul 06 22:08:43.483419 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:08:43.483319 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8a83eb6f-f275-4562-b014-0b48c710b624-metrics-certs podName:8a83eb6f-f275-4562-b014-0b48c710b624 nodeName:}" failed. No retries permitted until 2026-07-06 22:08:44.483308356 +0000 UTC m=+4.042103657 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8a83eb6f-f275-4562-b014-0b48c710b624-metrics-certs") pod "network-metrics-daemon-n9hgj" (UID: "8a83eb6f-f275-4562-b014-0b48c710b624") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 22:08:43.583789 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:43.583755 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b42224bc-b677-43bf-94fb-fc75223b963a-original-pull-secret\") pod \"global-pull-secret-syncer-klc48\" (UID: \"b42224bc-b677-43bf-94fb-fc75223b963a\") " pod="kube-system/global-pull-secret-syncer-klc48" Jul 06 22:08:43.583924 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:08:43.583877 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 22:08:43.583960 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:08:43.583928 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b42224bc-b677-43bf-94fb-fc75223b963a-original-pull-secret podName:b42224bc-b677-43bf-94fb-fc75223b963a nodeName:}" failed. No retries permitted until 2026-07-06 22:08:44.583911873 +0000 UTC m=+4.142707177 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b42224bc-b677-43bf-94fb-fc75223b963a-original-pull-secret") pod "global-pull-secret-syncer-klc48" (UID: "b42224bc-b677-43bf-94fb-fc75223b963a") : object "kube-system"/"original-pull-secret" not registered Jul 06 22:08:43.914314 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:43.914275 2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-05 22:03:43 +0000 UTC" deadline="2028-04-16 07:45:53.375001894 +0000 UTC" Jul 06 22:08:43.914314 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:43.914309 2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15585h37m9.460696618s" Jul 06 22:08:43.947958 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:43.947927 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42603ae9f82038ef3f0cca72faabcc35.slice/crio-74c0303721e65417886b3921f9c236786c091941052d5358934dea60998810a4 WatchSource:0}: Error finding container 74c0303721e65417886b3921f9c236786c091941052d5358934dea60998810a4: Status 404 returned error can't find the container with id 74c0303721e65417886b3921f9c236786c091941052d5358934dea60998810a4 Jul 06 22:08:43.949182 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:43.949157 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5b109e4_bcab_4c91_8791_3c7fcd077f27.slice/crio-23b410fbf0c19d8b454931ae496fa1240d06929a9fbf293c27c67ec8795b7129 WatchSource:0}: Error finding container 23b410fbf0c19d8b454931ae496fa1240d06929a9fbf293c27c67ec8795b7129: Status 404 returned error can't find the container with id 23b410fbf0c19d8b454931ae496fa1240d06929a9fbf293c27c67ec8795b7129 Jul 06 22:08:43.950822 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:43.950793 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0938d2ec_6cb6_4f74_882c_06081dbbfb88.slice/crio-9cfe935aa3992c87dad294c3be195b0312ac0b990738c5c90d324f4d23353af5 WatchSource:0}: Error finding container 9cfe935aa3992c87dad294c3be195b0312ac0b990738c5c90d324f4d23353af5: Status 404 returned error can't find the container with id 9cfe935aa3992c87dad294c3be195b0312ac0b990738c5c90d324f4d23353af5 Jul 06 22:08:43.954382 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:43.954342 2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 22:08:43.957421 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:43.957402 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ef94701_6fd1_4b4d_aa83_e6effd1023bf.slice/crio-faa0a4393f631d5839175608e5a091b88022a105a7c4cb7762e8a4c7cdb02d96 WatchSource:0}: Error finding container faa0a4393f631d5839175608e5a091b88022a105a7c4cb7762e8a4c7cdb02d96: Status 404 returned error can't find the container with id faa0a4393f631d5839175608e5a091b88022a105a7c4cb7762e8a4c7cdb02d96 Jul 06 22:08:43.958369 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:43.958348 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1de085f_40f2_4f95_a5bb_78aba60de69e.slice/crio-cc8c1596ef52208084a94c23b9e6bc64222a21594bff66337a3f8ccbe6588f41 WatchSource:0}: Error finding container cc8c1596ef52208084a94c23b9e6bc64222a21594bff66337a3f8ccbe6588f41: Status 404 returned error can't find the container with id cc8c1596ef52208084a94c23b9e6bc64222a21594bff66337a3f8ccbe6588f41 Jul 06 22:08:44.007974 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:44.007238 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod314ffdbc_f9e6_45c0_b37a_d216700a1de5.slice/crio-137b5c752bf58eaa40e551a80a8d4199faa18ba4a492c96f5530d9e029560d4e WatchSource:0}: Error finding container 137b5c752bf58eaa40e551a80a8d4199faa18ba4a492c96f5530d9e029560d4e: Status 404 returned error can't find the container with id 137b5c752bf58eaa40e551a80a8d4199faa18ba4a492c96f5530d9e029560d4e Jul 06 22:08:44.054599 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:44.054573 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2573bcaf102b258c7dbacbd9609f526e.slice/crio-326efdf581b1c2991a231c711e32276b2d29f5e6d54c908953273e5b31ea8707 WatchSource:0}: Error finding container 326efdf581b1c2991a231c711e32276b2d29f5e6d54c908953273e5b31ea8707: Status 404 returned error can't find the container with id 326efdf581b1c2991a231c711e32276b2d29f5e6d54c908953273e5b31ea8707 Jul 06 22:08:44.105950 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:44.105923 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2e37c39_7aed_412b_8a5d_2fc1481f707e.slice/crio-5c2b2ed1b06711fcfc6fd2cae278521a33c7600986f2b805d2d8793ec9670361 WatchSource:0}: Error finding container 5c2b2ed1b06711fcfc6fd2cae278521a33c7600986f2b805d2d8793ec9670361: Status 404 returned error can't find the container with id 5c2b2ed1b06711fcfc6fd2cae278521a33c7600986f2b805d2d8793ec9670361 Jul 06 22:08:44.174549 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:44.174494 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-st84l" event={"ID":"8ef94701-6fd1-4b4d-aa83-e6effd1023bf","Type":"ContainerStarted","Data":"faa0a4393f631d5839175608e5a091b88022a105a7c4cb7762e8a4c7cdb02d96"} Jul 06 22:08:44.175478 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:44.175457 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-w7z7d" event={"ID":"0938d2ec-6cb6-4f74-882c-06081dbbfb88","Type":"ContainerStarted","Data":"9cfe935aa3992c87dad294c3be195b0312ac0b990738c5c90d324f4d23353af5"} Jul 06 22:08:44.176420 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:44.176400 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-fk58k" event={"ID":"e5b109e4-bcab-4c91-8791-3c7fcd077f27","Type":"ContainerStarted","Data":"23b410fbf0c19d8b454931ae496fa1240d06929a9fbf293c27c67ec8795b7129"} Jul 06 22:08:44.177317 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:44.177299 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-101.ec2.internal" event={"ID":"42603ae9f82038ef3f0cca72faabcc35","Type":"ContainerStarted","Data":"74c0303721e65417886b3921f9c236786c091941052d5358934dea60998810a4"} Jul 06 22:08:44.178257 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:44.178240 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-gdwzl" event={"ID":"f2e37c39-7aed-412b-8a5d-2fc1481f707e","Type":"ContainerStarted","Data":"5c2b2ed1b06711fcfc6fd2cae278521a33c7600986f2b805d2d8793ec9670361"} Jul 06 22:08:44.179179 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:44.179153 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-101.ec2.internal" event={"ID":"2573bcaf102b258c7dbacbd9609f526e","Type":"ContainerStarted","Data":"326efdf581b1c2991a231c711e32276b2d29f5e6d54c908953273e5b31ea8707"} Jul 06 22:08:44.180035 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:44.180018 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-4b7pr" event={"ID":"314ffdbc-f9e6-45c0-b37a-d216700a1de5","Type":"ContainerStarted","Data":"137b5c752bf58eaa40e551a80a8d4199faa18ba4a492c96f5530d9e029560d4e"} Jul 06 22:08:44.180966 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:44.180949 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5nrhc" event={"ID":"f1de085f-40f2-4f95-a5bb-78aba60de69e","Type":"ContainerStarted","Data":"cc8c1596ef52208084a94c23b9e6bc64222a21594bff66337a3f8ccbe6588f41"} Jul 06 22:08:44.301848 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:44.301815 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7a62305_2195_4227_b5b7_5dec03864283.slice/crio-988b7b4bf063bd9acccb419668c35da4eedfa4448a8eaac9029bb9b31e8d0fe7 WatchSource:0}: Error finding container 988b7b4bf063bd9acccb419668c35da4eedfa4448a8eaac9029bb9b31e8d0fe7: Status 404 returned error can't find the container with id 988b7b4bf063bd9acccb419668c35da4eedfa4448a8eaac9029bb9b31e8d0fe7 Jul 06 22:08:44.331333 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:44.331300 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2041467a_e1c5_4702_96c6_8be98208f3eb.slice/crio-adfbb44a2ed9c040d0ad7b483d1f6b7f329722bc6b66b62b9b842b2e06830be5 WatchSource:0}: Error finding container adfbb44a2ed9c040d0ad7b483d1f6b7f329722bc6b66b62b9b842b2e06830be5: Status 404 returned error can't find the container with id adfbb44a2ed9c040d0ad7b483d1f6b7f329722bc6b66b62b9b842b2e06830be5 Jul 06 22:08:44.381040 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:08:44.381012 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a1fd480_e89a_4522_b82a_87b8c72124d5.slice/crio-1826aa48a01fb5de086c96d11cff5f77152c5b3ffacb9b93a0ae8ad293aa89b4 WatchSource:0}: Error finding container 1826aa48a01fb5de086c96d11cff5f77152c5b3ffacb9b93a0ae8ad293aa89b4: Status 404 returned error can't find the container with id 1826aa48a01fb5de086c96d11cff5f77152c5b3ffacb9b93a0ae8ad293aa89b4 Jul 06 22:08:44.489796 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:44.489762 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8a83eb6f-f275-4562-b014-0b48c710b624-metrics-certs\") pod \"network-metrics-daemon-n9hgj\" (UID: \"8a83eb6f-f275-4562-b014-0b48c710b624\") " pod="openshift-multus/network-metrics-daemon-n9hgj" Jul 06 22:08:44.489986 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:44.489841 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tlvkd\" (UniqueName: \"kubernetes.io/projected/9ade138e-0e88-4087-8042-92fe2db1e1d8-kube-api-access-tlvkd\") pod \"network-check-target-l5rc4\" (UID: \"9ade138e-0e88-4087-8042-92fe2db1e1d8\") " pod="openshift-network-diagnostics/network-check-target-l5rc4" Jul 06 22:08:44.489986 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:08:44.489942 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 22:08:44.489986 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:08:44.489953 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 22:08:44.489986 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:08:44.489971 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 22:08:44.489986 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:08:44.489984 2581 projected.go:196] Error preparing data for projected volume kube-api-access-tlvkd for pod openshift-network-diagnostics/network-check-target-l5rc4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 22:08:44.490175 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:08:44.490021 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8a83eb6f-f275-4562-b014-0b48c710b624-metrics-certs podName:8a83eb6f-f275-4562-b014-0b48c710b624 nodeName:}" failed. No retries permitted until 2026-07-06 22:08:46.490000618 +0000 UTC m=+6.048795937 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8a83eb6f-f275-4562-b014-0b48c710b624-metrics-certs") pod "network-metrics-daemon-n9hgj" (UID: "8a83eb6f-f275-4562-b014-0b48c710b624") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 22:08:44.490175 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:08:44.490042 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9ade138e-0e88-4087-8042-92fe2db1e1d8-kube-api-access-tlvkd podName:9ade138e-0e88-4087-8042-92fe2db1e1d8 nodeName:}" failed. No retries permitted until 2026-07-06 22:08:46.490031164 +0000 UTC m=+6.048826471 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-tlvkd" (UniqueName: "kubernetes.io/projected/9ade138e-0e88-4087-8042-92fe2db1e1d8-kube-api-access-tlvkd") pod "network-check-target-l5rc4" (UID: "9ade138e-0e88-4087-8042-92fe2db1e1d8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 22:08:44.590198 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:44.590100 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b42224bc-b677-43bf-94fb-fc75223b963a-original-pull-secret\") pod \"global-pull-secret-syncer-klc48\" (UID: \"b42224bc-b677-43bf-94fb-fc75223b963a\") " pod="kube-system/global-pull-secret-syncer-klc48" Jul 06 22:08:44.590358 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:08:44.590270 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 22:08:44.590358 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:08:44.590352 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b42224bc-b677-43bf-94fb-fc75223b963a-original-pull-secret podName:b42224bc-b677-43bf-94fb-fc75223b963a nodeName:}" failed. No retries permitted until 2026-07-06 22:08:46.590330919 +0000 UTC m=+6.149126222 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b42224bc-b677-43bf-94fb-fc75223b963a-original-pull-secret") pod "global-pull-secret-syncer-klc48" (UID: "b42224bc-b677-43bf-94fb-fc75223b963a") : object "kube-system"/"original-pull-secret" not registered Jul 06 22:08:44.915640 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:44.915505 2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-05 22:03:43 +0000 UTC" deadline="2027-12-18 06:42:29.277758766 +0000 UTC" Jul 06 22:08:44.915640 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:44.915561 2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12704h33m44.362202s" Jul 06 22:08:44.973314 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:44.972504 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n9hgj" Jul 06 22:08:44.973314 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:08:44.972642 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n9hgj" podUID="8a83eb6f-f275-4562-b014-0b48c710b624" Jul 06 22:08:44.973314 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:44.973018 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-klc48" Jul 06 22:08:44.973314 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:08:44.973133 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-klc48" podUID="b42224bc-b677-43bf-94fb-fc75223b963a" Jul 06 22:08:44.973314 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:44.973191 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l5rc4" Jul 06 22:08:44.973314 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:08:44.973262 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l5rc4" podUID="9ade138e-0e88-4087-8042-92fe2db1e1d8" Jul 06 22:08:45.209778 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:45.209482 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-cmtz2" event={"ID":"9a1fd480-e89a-4522-b82a-87b8c72124d5","Type":"ContainerStarted","Data":"1826aa48a01fb5de086c96d11cff5f77152c5b3ffacb9b93a0ae8ad293aa89b4"} Jul 06 22:08:45.229420 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:45.229378 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vt6d6" event={"ID":"a7a62305-2195-4227-b5b7-5dec03864283","Type":"ContainerStarted","Data":"988b7b4bf063bd9acccb419668c35da4eedfa4448a8eaac9029bb9b31e8d0fe7"} Jul 06 22:08:45.248178 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:45.248135 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-66tt7" event={"ID":"2041467a-e1c5-4702-96c6-8be98208f3eb","Type":"ContainerStarted","Data":"adfbb44a2ed9c040d0ad7b483d1f6b7f329722bc6b66b62b9b842b2e06830be5"} Jul 06 22:08:46.505397 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:46.504611 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tlvkd\" (UniqueName: \"kubernetes.io/projected/9ade138e-0e88-4087-8042-92fe2db1e1d8-kube-api-access-tlvkd\") pod \"network-check-target-l5rc4\" (UID: \"9ade138e-0e88-4087-8042-92fe2db1e1d8\") " pod="openshift-network-diagnostics/network-check-target-l5rc4" Jul 06 22:08:46.505397 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:46.504667 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8a83eb6f-f275-4562-b014-0b48c710b624-metrics-certs\") pod \"network-metrics-daemon-n9hgj\" (UID: \"8a83eb6f-f275-4562-b014-0b48c710b624\") " pod="openshift-multus/network-metrics-daemon-n9hgj" Jul 06 22:08:46.505397 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:08:46.504783 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 22:08:46.505397 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:08:46.504849 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8a83eb6f-f275-4562-b014-0b48c710b624-metrics-certs podName:8a83eb6f-f275-4562-b014-0b48c710b624 nodeName:}" failed. No retries permitted until 2026-07-06 22:08:50.5048291 +0000 UTC m=+10.063624401 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8a83eb6f-f275-4562-b014-0b48c710b624-metrics-certs") pod "network-metrics-daemon-n9hgj" (UID: "8a83eb6f-f275-4562-b014-0b48c710b624") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 22:08:46.505397 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:08:46.505274 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 22:08:46.505397 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:08:46.505294 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 22:08:46.505397 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:08:46.505309 2581 projected.go:196] Error preparing data for projected volume kube-api-access-tlvkd for pod openshift-network-diagnostics/network-check-target-l5rc4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 22:08:46.505397 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:08:46.505356 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9ade138e-0e88-4087-8042-92fe2db1e1d8-kube-api-access-tlvkd podName:9ade138e-0e88-4087-8042-92fe2db1e1d8 nodeName:}" failed. No retries permitted until 2026-07-06 22:08:50.505340539 +0000 UTC m=+10.064135851 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-tlvkd" (UniqueName: "kubernetes.io/projected/9ade138e-0e88-4087-8042-92fe2db1e1d8-kube-api-access-tlvkd") pod "network-check-target-l5rc4" (UID: "9ade138e-0e88-4087-8042-92fe2db1e1d8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 22:08:46.605861 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:46.605822 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b42224bc-b677-43bf-94fb-fc75223b963a-original-pull-secret\") pod \"global-pull-secret-syncer-klc48\" (UID: \"b42224bc-b677-43bf-94fb-fc75223b963a\") " pod="kube-system/global-pull-secret-syncer-klc48" Jul 06 22:08:46.606086 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:08:46.606044 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 22:08:46.606175 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:08:46.606129 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b42224bc-b677-43bf-94fb-fc75223b963a-original-pull-secret podName:b42224bc-b677-43bf-94fb-fc75223b963a nodeName:}" failed. No retries permitted until 2026-07-06 22:08:50.606111803 +0000 UTC m=+10.164907107 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b42224bc-b677-43bf-94fb-fc75223b963a-original-pull-secret") pod "global-pull-secret-syncer-klc48" (UID: "b42224bc-b677-43bf-94fb-fc75223b963a") : object "kube-system"/"original-pull-secret" not registered Jul 06 22:08:46.972354 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:46.972267 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n9hgj" Jul 06 22:08:46.972507 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:08:46.972424 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n9hgj" podUID="8a83eb6f-f275-4562-b014-0b48c710b624" Jul 06 22:08:46.972507 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:46.972492 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-klc48" Jul 06 22:08:46.972620 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:08:46.972563 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-klc48" podUID="b42224bc-b677-43bf-94fb-fc75223b963a" Jul 06 22:08:46.972620 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:46.972607 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l5rc4" Jul 06 22:08:46.972726 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:08:46.972667 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l5rc4" podUID="9ade138e-0e88-4087-8042-92fe2db1e1d8" Jul 06 22:08:48.975380 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:48.975098 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-klc48" Jul 06 22:08:48.975380 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:48.975133 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n9hgj" Jul 06 22:08:48.975380 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:48.975098 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l5rc4" Jul 06 22:08:48.975380 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:08:48.975228 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-klc48" podUID="b42224bc-b677-43bf-94fb-fc75223b963a" Jul 06 22:08:48.975380 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:08:48.975329 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l5rc4" podUID="9ade138e-0e88-4087-8042-92fe2db1e1d8" Jul 06 22:08:48.977370 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:08:48.977303 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n9hgj" podUID="8a83eb6f-f275-4562-b014-0b48c710b624" Jul 06 22:08:50.533810 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:50.533779 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tlvkd\" (UniqueName: \"kubernetes.io/projected/9ade138e-0e88-4087-8042-92fe2db1e1d8-kube-api-access-tlvkd\") pod \"network-check-target-l5rc4\" (UID: \"9ade138e-0e88-4087-8042-92fe2db1e1d8\") " pod="openshift-network-diagnostics/network-check-target-l5rc4" Jul 06 22:08:50.534267 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:50.533821 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8a83eb6f-f275-4562-b014-0b48c710b624-metrics-certs\") pod \"network-metrics-daemon-n9hgj\" (UID: \"8a83eb6f-f275-4562-b014-0b48c710b624\") " pod="openshift-multus/network-metrics-daemon-n9hgj" Jul 06 22:08:50.534267 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:08:50.533928 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 22:08:50.534267 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:08:50.533942 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 22:08:50.534267 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:08:50.533961 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 22:08:50.534267 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:08:50.533974 2581 projected.go:196] Error preparing data for projected volume kube-api-access-tlvkd for pod openshift-network-diagnostics/network-check-target-l5rc4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 22:08:50.534267 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:08:50.533993 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8a83eb6f-f275-4562-b014-0b48c710b624-metrics-certs podName:8a83eb6f-f275-4562-b014-0b48c710b624 nodeName:}" failed. No retries permitted until 2026-07-06 22:08:58.533972794 +0000 UTC m=+18.092768108 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8a83eb6f-f275-4562-b014-0b48c710b624-metrics-certs") pod "network-metrics-daemon-n9hgj" (UID: "8a83eb6f-f275-4562-b014-0b48c710b624") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 22:08:50.534267 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:08:50.534010 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9ade138e-0e88-4087-8042-92fe2db1e1d8-kube-api-access-tlvkd podName:9ade138e-0e88-4087-8042-92fe2db1e1d8 nodeName:}" failed. No retries permitted until 2026-07-06 22:08:58.533999449 +0000 UTC m=+18.092794751 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-tlvkd" (UniqueName: "kubernetes.io/projected/9ade138e-0e88-4087-8042-92fe2db1e1d8-kube-api-access-tlvkd") pod "network-check-target-l5rc4" (UID: "9ade138e-0e88-4087-8042-92fe2db1e1d8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 22:08:50.634555 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:50.634520 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b42224bc-b677-43bf-94fb-fc75223b963a-original-pull-secret\") pod \"global-pull-secret-syncer-klc48\" (UID: \"b42224bc-b677-43bf-94fb-fc75223b963a\") " pod="kube-system/global-pull-secret-syncer-klc48" Jul 06 22:08:50.634729 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:08:50.634676 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 22:08:50.634789 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:08:50.634732 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b42224bc-b677-43bf-94fb-fc75223b963a-original-pull-secret podName:b42224bc-b677-43bf-94fb-fc75223b963a nodeName:}" failed. No retries permitted until 2026-07-06 22:08:58.63471524 +0000 UTC m=+18.193510554 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b42224bc-b677-43bf-94fb-fc75223b963a-original-pull-secret") pod "global-pull-secret-syncer-klc48" (UID: "b42224bc-b677-43bf-94fb-fc75223b963a") : object "kube-system"/"original-pull-secret" not registered Jul 06 22:08:50.971557 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:50.971002 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n9hgj" Jul 06 22:08:50.971557 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:08:50.971151 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n9hgj" podUID="8a83eb6f-f275-4562-b014-0b48c710b624" Jul 06 22:08:50.971557 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:50.971248 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-klc48" Jul 06 22:08:50.971557 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:50.971347 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l5rc4" Jul 06 22:08:50.971557 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:08:50.971395 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-klc48" podUID="b42224bc-b677-43bf-94fb-fc75223b963a" Jul 06 22:08:50.971557 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:08:50.971408 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l5rc4" podUID="9ade138e-0e88-4087-8042-92fe2db1e1d8" Jul 06 22:08:52.970891 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:52.970235 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n9hgj" Jul 06 22:08:52.970891 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:52.970247 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-klc48" Jul 06 22:08:52.970891 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:08:52.970370 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n9hgj" podUID="8a83eb6f-f275-4562-b014-0b48c710b624" Jul 06 22:08:52.970891 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:52.970447 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l5rc4" Jul 06 22:08:52.970891 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:08:52.970442 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-klc48" podUID="b42224bc-b677-43bf-94fb-fc75223b963a" Jul 06 22:08:52.970891 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:08:52.970521 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l5rc4" podUID="9ade138e-0e88-4087-8042-92fe2db1e1d8" Jul 06 22:08:54.974757 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:54.974251 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l5rc4" Jul 06 22:08:54.974757 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:54.974298 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-klc48" Jul 06 22:08:54.974757 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:54.974262 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n9hgj" Jul 06 22:08:54.974757 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:08:54.974402 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l5rc4" podUID="9ade138e-0e88-4087-8042-92fe2db1e1d8" Jul 06 22:08:54.974757 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:08:54.974459 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n9hgj" podUID="8a83eb6f-f275-4562-b014-0b48c710b624" Jul 06 22:08:54.974757 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:08:54.974521 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-klc48" podUID="b42224bc-b677-43bf-94fb-fc75223b963a" Jul 06 22:08:56.969737 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:56.969705 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n9hgj" Jul 06 22:08:56.970165 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:56.969740 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l5rc4" Jul 06 22:08:56.970165 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:56.969706 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-klc48" Jul 06 22:08:56.970165 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:08:56.969845 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n9hgj" podUID="8a83eb6f-f275-4562-b014-0b48c710b624" Jul 06 22:08:56.970165 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:08:56.969923 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-klc48" podUID="b42224bc-b677-43bf-94fb-fc75223b963a" Jul 06 22:08:56.970165 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:08:56.969996 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l5rc4" podUID="9ade138e-0e88-4087-8042-92fe2db1e1d8" Jul 06 22:08:58.596721 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:58.596683 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tlvkd\" (UniqueName: \"kubernetes.io/projected/9ade138e-0e88-4087-8042-92fe2db1e1d8-kube-api-access-tlvkd\") pod \"network-check-target-l5rc4\" (UID: \"9ade138e-0e88-4087-8042-92fe2db1e1d8\") " pod="openshift-network-diagnostics/network-check-target-l5rc4" Jul 06 22:08:58.596721 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:58.596729 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8a83eb6f-f275-4562-b014-0b48c710b624-metrics-certs\") pod \"network-metrics-daemon-n9hgj\" (UID: \"8a83eb6f-f275-4562-b014-0b48c710b624\") " pod="openshift-multus/network-metrics-daemon-n9hgj" Jul 06 22:08:58.597218 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:08:58.596858 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 22:08:58.597218 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:08:58.596883 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 22:08:58.597218 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:08:58.596909 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 22:08:58.597218 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:08:58.596918 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8a83eb6f-f275-4562-b014-0b48c710b624-metrics-certs podName:8a83eb6f-f275-4562-b014-0b48c710b624 nodeName:}" failed. No retries permitted until 2026-07-06 22:09:14.596899678 +0000 UTC m=+34.155694978 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8a83eb6f-f275-4562-b014-0b48c710b624-metrics-certs") pod "network-metrics-daemon-n9hgj" (UID: "8a83eb6f-f275-4562-b014-0b48c710b624") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 22:08:58.597218 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:08:58.596922 2581 projected.go:196] Error preparing data for projected volume kube-api-access-tlvkd for pod openshift-network-diagnostics/network-check-target-l5rc4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 22:08:58.597218 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:08:58.596975 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9ade138e-0e88-4087-8042-92fe2db1e1d8-kube-api-access-tlvkd podName:9ade138e-0e88-4087-8042-92fe2db1e1d8 nodeName:}" failed. No retries permitted until 2026-07-06 22:09:14.596956963 +0000 UTC m=+34.155752276 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-tlvkd" (UniqueName: "kubernetes.io/projected/9ade138e-0e88-4087-8042-92fe2db1e1d8-kube-api-access-tlvkd") pod "network-check-target-l5rc4" (UID: "9ade138e-0e88-4087-8042-92fe2db1e1d8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 22:08:58.697448 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:58.697390 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b42224bc-b677-43bf-94fb-fc75223b963a-original-pull-secret\") pod \"global-pull-secret-syncer-klc48\" (UID: \"b42224bc-b677-43bf-94fb-fc75223b963a\") " pod="kube-system/global-pull-secret-syncer-klc48" Jul 06 22:08:58.697640 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:08:58.697533 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 22:08:58.697640 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:08:58.697602 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b42224bc-b677-43bf-94fb-fc75223b963a-original-pull-secret podName:b42224bc-b677-43bf-94fb-fc75223b963a nodeName:}" failed. No retries permitted until 2026-07-06 22:09:14.697582819 +0000 UTC m=+34.256378141 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b42224bc-b677-43bf-94fb-fc75223b963a-original-pull-secret") pod "global-pull-secret-syncer-klc48" (UID: "b42224bc-b677-43bf-94fb-fc75223b963a") : object "kube-system"/"original-pull-secret" not registered Jul 06 22:08:58.970419 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:58.970382 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l5rc4" Jul 06 22:08:58.970598 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:58.970394 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n9hgj" Jul 06 22:08:58.970598 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:08:58.970496 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l5rc4" podUID="9ade138e-0e88-4087-8042-92fe2db1e1d8" Jul 06 22:08:58.970738 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:08:58.970591 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n9hgj" podUID="8a83eb6f-f275-4562-b014-0b48c710b624" Jul 06 22:08:58.970738 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:08:58.970609 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-klc48" Jul 06 22:08:58.970738 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:08:58.970685 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-klc48" podUID="b42224bc-b677-43bf-94fb-fc75223b963a" Jul 06 22:09:00.971290 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:00.971259 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n9hgj" Jul 06 22:09:00.971749 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:00.971382 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n9hgj" podUID="8a83eb6f-f275-4562-b014-0b48c710b624" Jul 06 22:09:00.971749 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:00.971387 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-klc48" Jul 06 22:09:00.971749 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:00.971420 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l5rc4" Jul 06 22:09:00.971749 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:00.971472 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-klc48" podUID="b42224bc-b677-43bf-94fb-fc75223b963a" Jul 06 22:09:00.971749 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:00.971533 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l5rc4" podUID="9ade138e-0e88-4087-8042-92fe2db1e1d8" Jul 06 22:09:02.970219 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:02.970183 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-klc48" Jul 06 22:09:02.970219 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:02.970201 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l5rc4" Jul 06 22:09:02.970750 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:02.970184 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n9hgj" Jul 06 22:09:02.970750 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:02.970300 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-klc48" podUID="b42224bc-b677-43bf-94fb-fc75223b963a" Jul 06 22:09:02.970750 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:02.970386 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l5rc4" podUID="9ade138e-0e88-4087-8042-92fe2db1e1d8" Jul 06 22:09:02.970750 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:02.970490 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n9hgj" podUID="8a83eb6f-f275-4562-b014-0b48c710b624" Jul 06 22:09:04.970417 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:04.970215 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n9hgj" Jul 06 22:09:04.971110 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:04.970235 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-klc48" Jul 06 22:09:04.971110 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:04.970512 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n9hgj" podUID="8a83eb6f-f275-4562-b014-0b48c710b624" Jul 06 22:09:04.971110 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:04.970619 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-klc48" podUID="b42224bc-b677-43bf-94fb-fc75223b963a" Jul 06 22:09:04.971110 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:04.970296 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l5rc4" Jul 06 22:09:04.971110 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:04.970726 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l5rc4" podUID="9ade138e-0e88-4087-8042-92fe2db1e1d8" Jul 06 22:09:05.321505 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:05.321474 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-4b7pr" event={"ID":"314ffdbc-f9e6-45c0-b37a-d216700a1de5","Type":"ContainerStarted","Data":"743a39c83a012eafb5ecc01c0f000bb25797547843c5527f9bb1e8a6cac75638"} Jul 06 22:09:05.323008 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:05.322981 2581 generic.go:356] "Generic (PLEG): container finished" podID="f1de085f-40f2-4f95-a5bb-78aba60de69e" containerID="9a7e95ddad7e2fdf75207d0aee166495b29a87d391563464b6b09f4fec2b1f1d" exitCode=0 Jul 06 22:09:05.323153 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:05.323083 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5nrhc" event={"ID":"f1de085f-40f2-4f95-a5bb-78aba60de69e","Type":"ContainerDied","Data":"9a7e95ddad7e2fdf75207d0aee166495b29a87d391563464b6b09f4fec2b1f1d"} Jul 06 22:09:05.324628 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:05.324603 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-st84l" event={"ID":"8ef94701-6fd1-4b4d-aa83-e6effd1023bf","Type":"ContainerStarted","Data":"d5add295d0b953a827e2dc28256da09cb71319384181eb2f1be9748f5dfebef4"} Jul 06 22:09:05.326351 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:05.326055 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-w7z7d" event={"ID":"0938d2ec-6cb6-4f74-882c-06081dbbfb88","Type":"ContainerStarted","Data":"32e3006a0b7aeade8a741f20aaa9f5031ab7ddbbc3e2909a610d8ddcc27395f2"} Jul 06 22:09:05.327593 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:05.327569 2581 generic.go:356] "Generic (PLEG): container finished" podID="42603ae9f82038ef3f0cca72faabcc35" containerID="b00f8a5154c454627820e79520d40af89b30fccb1cb99f993d58cdd182ec422c" exitCode=0 Jul 06 22:09:05.327761 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:05.327657 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-101.ec2.internal" event={"ID":"42603ae9f82038ef3f0cca72faabcc35","Type":"ContainerDied","Data":"b00f8a5154c454627820e79520d40af89b30fccb1cb99f993d58cdd182ec422c"} Jul 06 22:09:05.329217 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:05.329194 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-66tt7" event={"ID":"2041467a-e1c5-4702-96c6-8be98208f3eb","Type":"ContainerStarted","Data":"ad0f359e19e75d0ec22c5e6f55b98c1134b7fd531524d57631cb424d4079ae71"} Jul 06 22:09:05.330864 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:05.330838 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-gdwzl" event={"ID":"f2e37c39-7aed-412b-8a5d-2fc1481f707e","Type":"ContainerStarted","Data":"69b8cb2295c3d03f1d7b39da866a54b54445c15bb4a49770b006222efabb1dc1"} Jul 06 22:09:05.332117 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:05.332088 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-101.ec2.internal" event={"ID":"2573bcaf102b258c7dbacbd9609f526e","Type":"ContainerStarted","Data":"a4ef49a7c7d59b48830dd09e5ac875a909a1a65d414176ad7ddd5f6f965a026a"} Jul 06 22:09:05.333317 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:05.333296 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-cmtz2" event={"ID":"9a1fd480-e89a-4522-b82a-87b8c72124d5","Type":"ContainerStarted","Data":"ffeecb1e4c1f27c72871ee9a55fb2b8eaa5d11cb007d48cd036c9d0f59873670"} Jul 06 22:09:05.336030 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:05.336012 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vt6d6" event={"ID":"a7a62305-2195-4227-b5b7-5dec03864283","Type":"ContainerStarted","Data":"bfe7c23e576f1246fa76a2005856bfd38eb3c80fb381a245ad3fbf6c3f6edb49"} Jul 06 22:09:05.336126 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:05.336095 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vt6d6" event={"ID":"a7a62305-2195-4227-b5b7-5dec03864283","Type":"ContainerStarted","Data":"f0104faf2633b2410a96822cc18c269ae4049ca792d3e1cddedf2b49e10d9e51"} Jul 06 22:09:05.336169 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:05.336126 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vt6d6" event={"ID":"a7a62305-2195-4227-b5b7-5dec03864283","Type":"ContainerStarted","Data":"fb4ba45b4a2cd731b9d4635cf9999476418714a1a3b9082516854308b00f7afa"} Jul 06 22:09:05.336169 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:05.336155 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vt6d6" event={"ID":"a7a62305-2195-4227-b5b7-5dec03864283","Type":"ContainerStarted","Data":"d14482038ed57cd119bce7d2c49925e7d07fae6d546fb0f7cae5df786ae823e8"} Jul 06 22:09:05.336229 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:05.336185 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vt6d6" event={"ID":"a7a62305-2195-4227-b5b7-5dec03864283","Type":"ContainerStarted","Data":"da1951a88cf105d37392da054a5c5ef8dd66426211cb5a87b8720b5a40359d4d"} Jul 06 22:09:05.336229 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:05.336211 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vt6d6" event={"ID":"a7a62305-2195-4227-b5b7-5dec03864283","Type":"ContainerStarted","Data":"f01eb13e9d2ef3d8a9c73da191ecfbb93c20e02d8c47934c09af9831fbd24ceb"} Jul 06 22:09:05.339394 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:05.339360 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-4b7pr" podStartSLOduration=12.066731001 podStartE2EDuration="24.339348219s" totalImagesPullingTime="12.272617218s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 22:08:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:09:05.338893649 +0000 UTC m=+24.897688971" watchObservedRunningTime="2026-07-06 22:09:05.339348219 +0000 UTC m=+24.898143541" Jul 06 22:09:05.353595 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:05.353540 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-66tt7" podStartSLOduration=12.488390358 podStartE2EDuration="24.35352896s" totalImagesPullingTime="11.865138602s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:08:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:09:05.353020235 +0000 UTC m=+24.911815556" watchObservedRunningTime="2026-07-06 22:09:05.35352896 +0000 UTC m=+24.912324280" Jul 06 22:09:05.369411 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:05.369336 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-w7z7d" podStartSLOduration=12.02571326 podStartE2EDuration="24.369325866s" totalImagesPullingTime="12.343612606s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 22:08:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:09:05.369083821 +0000 UTC m=+24.927879133" watchObservedRunningTime="2026-07-06 22:09:05.369325866 +0000 UTC m=+24.928121187" Jul 06 22:09:05.398340 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:05.398191 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-101.ec2.internal" podStartSLOduration=23.398178227 podStartE2EDuration="23.398178227s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 22:08:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:09:05.397932307 +0000 UTC m=+24.956727628" watchObservedRunningTime="2026-07-06 22:09:05.398178227 +0000 UTC m=+24.956973547" Jul 06 22:09:05.416453 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:05.416195 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-cmtz2" podStartSLOduration=12.515206176 podStartE2EDuration="24.416178666s" totalImagesPullingTime="11.90097249s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 22:08:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:09:05.415750676 +0000 UTC m=+24.974545997" watchObservedRunningTime="2026-07-06 22:09:05.416178666 +0000 UTC m=+24.974973989" Jul 06 22:09:05.440744 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:05.440706 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-gdwzl" podStartSLOduration=4.078649924 podStartE2EDuration="24.440693162s" totalImagesPullingTime="20.362043238s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 22:08:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:09:05.440587966 +0000 UTC m=+24.999383287" watchObservedRunningTime="2026-07-06 22:09:05.440693162 +0000 UTC m=+24.999488483" Jul 06 22:09:06.340471 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:06.340429 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-fk58k" event={"ID":"e5b109e4-bcab-4c91-8791-3c7fcd077f27","Type":"ContainerStarted","Data":"525156c4d9abd3d4740b694b44d248524fb631739d9692f49a668c4c914b1253"} Jul 06 22:09:06.342373 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:06.342344 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-101.ec2.internal" event={"ID":"42603ae9f82038ef3f0cca72faabcc35","Type":"ContainerStarted","Data":"8469227a953770ab0ee37287b8033ccc91bf96dc2e3c5f64e7567b7cfdef9fd8"} Jul 06 22:09:06.357436 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:06.357382 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-fk58k" podStartSLOduration=13.056556731 podStartE2EDuration="25.35736582s" totalImagesPullingTime="12.300809089s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 22:08:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:09:06.357029668 +0000 UTC m=+25.915825013" watchObservedRunningTime="2026-07-06 22:09:06.35736582 +0000 UTC m=+25.916161142" Jul 06 22:09:06.372278 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:06.372232 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-101.ec2.internal" podStartSLOduration=24.372218016 podStartE2EDuration="24.372218016s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 22:08:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:09:06.372202891 +0000 UTC m=+25.930998216" watchObservedRunningTime="2026-07-06 22:09:06.372218016 +0000 UTC m=+25.931013336" Jul 06 22:09:06.812563 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:06.812520 2581 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 06 22:09:06.970546 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:06.970468 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-klc48" Jul 06 22:09:06.970546 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:06.970509 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l5rc4" Jul 06 22:09:06.970780 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:06.970468 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n9hgj" Jul 06 22:09:06.970780 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:06.970587 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-klc48" podUID="b42224bc-b677-43bf-94fb-fc75223b963a" Jul 06 22:09:06.970780 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:06.970654 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n9hgj" podUID="8a83eb6f-f275-4562-b014-0b48c710b624" Jul 06 22:09:06.970780 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:06.970717 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l5rc4" podUID="9ade138e-0e88-4087-8042-92fe2db1e1d8" Jul 06 22:09:06.979155 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:06.979082 2581 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-06T22:09:06.812550582Z","UUID":"26598412-4cf0-460e-913a-05e59a21cc56","Handler":null,"Name":"","Endpoint":""} Jul 06 22:09:06.980356 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:06.980340 2581 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 06 22:09:06.980453 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:06.980371 2581 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 06 22:09:07.351640 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:07.351536 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vt6d6" event={"ID":"a7a62305-2195-4227-b5b7-5dec03864283","Type":"ContainerStarted","Data":"3e2d21b0d626c45b9421dc19efabd073dcf4350433bfd2e300c8d001f5b10fd0"} Jul 06 22:09:07.354255 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:07.354226 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-st84l" event={"ID":"8ef94701-6fd1-4b4d-aa83-e6effd1023bf","Type":"ContainerStarted","Data":"dde9e7fa28e8436ee80f52877993f96106f1c127e96b86f830f3660c5b75b075"} Jul 06 22:09:08.358029 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:08.357989 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-st84l" event={"ID":"8ef94701-6fd1-4b4d-aa83-e6effd1023bf","Type":"ContainerStarted","Data":"c73289c3ad70087b41ba5f6abcb48859de3e2473ef3c1a45a210eb10272bc8bd"} Jul 06 22:09:08.378728 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:08.378686 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-st84l" podStartSLOduration=12.19952055 podStartE2EDuration="27.378673173s" totalImagesPullingTime="15.179152623s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 22:08:41 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:09:08.378391188 +0000 UTC m=+27.937186508" watchObservedRunningTime="2026-07-06 22:09:08.378673173 +0000 UTC m=+27.937468507" Jul 06 22:09:08.970019 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:08.969979 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-klc48" Jul 06 22:09:08.970019 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:08.969993 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n9hgj" Jul 06 22:09:08.970275 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:08.969981 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l5rc4" Jul 06 22:09:08.970275 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:08.970104 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-klc48" podUID="b42224bc-b677-43bf-94fb-fc75223b963a" Jul 06 22:09:08.970275 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:08.970162 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l5rc4" podUID="9ade138e-0e88-4087-8042-92fe2db1e1d8" Jul 06 22:09:08.970275 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:08.970221 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n9hgj" podUID="8a83eb6f-f275-4562-b014-0b48c710b624" Jul 06 22:09:10.282193 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:10.281984 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-66tt7" Jul 06 22:09:10.282682 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:10.282502 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-66tt7" Jul 06 22:09:10.365402 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:10.365360 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vt6d6" event={"ID":"a7a62305-2195-4227-b5b7-5dec03864283","Type":"ContainerStarted","Data":"f2201588c73befcf758bdc0af6f893ca1eb9271e904517d1495955d3b7d5d11e"} Jul 06 22:09:10.365569 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:10.365466 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-66tt7" Jul 06 22:09:10.365954 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:10.365936 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-66tt7" Jul 06 22:09:10.394881 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:10.394830 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-vt6d6" podStartSLOduration=9.205841126 podStartE2EDuration="29.394815992s" totalImagesPullingTime="20.188974866s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 22:08:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:09:10.394611144 +0000 UTC m=+29.953406466" watchObservedRunningTime="2026-07-06 22:09:10.394815992 +0000 UTC m=+29.953611312" Jul 06 22:09:10.970878 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:10.970845 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-klc48" Jul 06 22:09:10.971098 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:10.970952 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-klc48" podUID="b42224bc-b677-43bf-94fb-fc75223b963a" Jul 06 22:09:10.971534 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:10.971204 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n9hgj" Jul 06 22:09:10.971534 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:10.971280 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l5rc4" Jul 06 22:09:10.971534 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:10.971380 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l5rc4" podUID="9ade138e-0e88-4087-8042-92fe2db1e1d8" Jul 06 22:09:10.971534 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:10.971440 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n9hgj" podUID="8a83eb6f-f275-4562-b014-0b48c710b624" Jul 06 22:09:11.368031 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:11.367934 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-vt6d6" Jul 06 22:09:11.368704 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:11.368430 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-vt6d6" Jul 06 22:09:11.368704 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:11.368489 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-vt6d6" Jul 06 22:09:11.386201 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:11.386160 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vt6d6" Jul 06 22:09:11.386357 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:11.386296 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vt6d6" Jul 06 22:09:11.427760 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:11.427734 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-l5rc4"] Jul 06 22:09:11.427931 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:11.427825 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l5rc4" Jul 06 22:09:11.428002 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:11.427950 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l5rc4" podUID="9ade138e-0e88-4087-8042-92fe2db1e1d8" Jul 06 22:09:11.431280 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:11.431253 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-klc48"] Jul 06 22:09:11.431429 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:11.431385 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-klc48" Jul 06 22:09:11.431508 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:11.431482 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-klc48" podUID="b42224bc-b677-43bf-94fb-fc75223b963a" Jul 06 22:09:11.432048 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:11.432006 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-n9hgj"] Jul 06 22:09:11.432213 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:11.432132 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n9hgj" Jul 06 22:09:11.432283 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:11.432262 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n9hgj" podUID="8a83eb6f-f275-4562-b014-0b48c710b624" Jul 06 22:09:12.969881 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:12.969844 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n9hgj" Jul 06 22:09:12.970312 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:12.969852 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l5rc4" Jul 06 22:09:12.970312 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:12.969988 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n9hgj" podUID="8a83eb6f-f275-4562-b014-0b48c710b624" Jul 06 22:09:12.970312 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:12.969852 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-klc48" Jul 06 22:09:12.970312 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:12.970093 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l5rc4" podUID="9ade138e-0e88-4087-8042-92fe2db1e1d8" Jul 06 22:09:12.970312 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:12.970128 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-klc48" podUID="b42224bc-b677-43bf-94fb-fc75223b963a" Jul 06 22:09:13.374948 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:13.374690 2581 generic.go:356] "Generic (PLEG): container finished" podID="f1de085f-40f2-4f95-a5bb-78aba60de69e" containerID="e1a1cf28d8562f26be67ee9155d397fb9561645ff11f00e9df0c3dd6a33cc7c9" exitCode=0 Jul 06 22:09:13.375124 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:13.374772 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5nrhc" event={"ID":"f1de085f-40f2-4f95-a5bb-78aba60de69e","Type":"ContainerDied","Data":"e1a1cf28d8562f26be67ee9155d397fb9561645ff11f00e9df0c3dd6a33cc7c9"} Jul 06 22:09:14.612205 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:14.612173 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tlvkd\" (UniqueName: \"kubernetes.io/projected/9ade138e-0e88-4087-8042-92fe2db1e1d8-kube-api-access-tlvkd\") pod \"network-check-target-l5rc4\" (UID: \"9ade138e-0e88-4087-8042-92fe2db1e1d8\") " pod="openshift-network-diagnostics/network-check-target-l5rc4" Jul 06 22:09:14.612546 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:14.612215 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8a83eb6f-f275-4562-b014-0b48c710b624-metrics-certs\") pod \"network-metrics-daemon-n9hgj\" (UID: \"8a83eb6f-f275-4562-b014-0b48c710b624\") " pod="openshift-multus/network-metrics-daemon-n9hgj" Jul 06 22:09:14.612546 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:14.612310 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 22:09:14.612546 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:14.612328 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 22:09:14.612546 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:14.612350 2581 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 22:09:14.612546 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:14.612355 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8a83eb6f-f275-4562-b014-0b48c710b624-metrics-certs podName:8a83eb6f-f275-4562-b014-0b48c710b624 nodeName:}" failed. No retries permitted until 2026-07-06 22:09:46.612342833 +0000 UTC m=+66.171138132 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8a83eb6f-f275-4562-b014-0b48c710b624-metrics-certs") pod "network-metrics-daemon-n9hgj" (UID: "8a83eb6f-f275-4562-b014-0b48c710b624") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 22:09:14.612546 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:14.612365 2581 projected.go:196] Error preparing data for projected volume kube-api-access-tlvkd for pod openshift-network-diagnostics/network-check-target-l5rc4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 22:09:14.612546 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:14.612416 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9ade138e-0e88-4087-8042-92fe2db1e1d8-kube-api-access-tlvkd podName:9ade138e-0e88-4087-8042-92fe2db1e1d8 nodeName:}" failed. No retries permitted until 2026-07-06 22:09:46.612399296 +0000 UTC m=+66.171194613 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-tlvkd" (UniqueName: "kubernetes.io/projected/9ade138e-0e88-4087-8042-92fe2db1e1d8-kube-api-access-tlvkd") pod "network-check-target-l5rc4" (UID: "9ade138e-0e88-4087-8042-92fe2db1e1d8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 22:09:14.712781 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:14.712742 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b42224bc-b677-43bf-94fb-fc75223b963a-original-pull-secret\") pod \"global-pull-secret-syncer-klc48\" (UID: \"b42224bc-b677-43bf-94fb-fc75223b963a\") " pod="kube-system/global-pull-secret-syncer-klc48" Jul 06 22:09:14.712942 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:14.712859 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 22:09:14.712942 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:14.712911 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b42224bc-b677-43bf-94fb-fc75223b963a-original-pull-secret podName:b42224bc-b677-43bf-94fb-fc75223b963a nodeName:}" failed. No retries permitted until 2026-07-06 22:09:46.712897962 +0000 UTC m=+66.271693261 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b42224bc-b677-43bf-94fb-fc75223b963a-original-pull-secret") pod "global-pull-secret-syncer-klc48" (UID: "b42224bc-b677-43bf-94fb-fc75223b963a") : object "kube-system"/"original-pull-secret" not registered Jul 06 22:09:14.969848 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:14.969810 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-klc48" Jul 06 22:09:14.969848 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:14.969834 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l5rc4" Jul 06 22:09:14.970086 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:14.969813 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n9hgj" Jul 06 22:09:14.970086 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:14.969934 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-klc48" podUID="b42224bc-b677-43bf-94fb-fc75223b963a" Jul 06 22:09:14.970086 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:14.970019 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n9hgj" podUID="8a83eb6f-f275-4562-b014-0b48c710b624" Jul 06 22:09:14.970250 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:14.970106 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l5rc4" podUID="9ade138e-0e88-4087-8042-92fe2db1e1d8" Jul 06 22:09:15.381885 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:15.381796 2581 generic.go:356] "Generic (PLEG): container finished" podID="f1de085f-40f2-4f95-a5bb-78aba60de69e" containerID="bc79504450aeaafb550400685fe10cdf0295fe1fe92a2b048f6bcf9837de53e1" exitCode=0 Jul 06 22:09:15.381885 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:15.381852 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5nrhc" event={"ID":"f1de085f-40f2-4f95-a5bb-78aba60de69e","Type":"ContainerDied","Data":"bc79504450aeaafb550400685fe10cdf0295fe1fe92a2b048f6bcf9837de53e1"} Jul 06 22:09:16.385761 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:16.385729 2581 generic.go:356] "Generic (PLEG): container finished" podID="f1de085f-40f2-4f95-a5bb-78aba60de69e" containerID="bfacdb43e3bb34f5a02e6247ef24e4d7b615a368a49aef7535f84b46b2c9e9c5" exitCode=0 Jul 06 22:09:16.386117 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:16.385788 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5nrhc" event={"ID":"f1de085f-40f2-4f95-a5bb-78aba60de69e","Type":"ContainerDied","Data":"bfacdb43e3bb34f5a02e6247ef24e4d7b615a368a49aef7535f84b46b2c9e9c5"} Jul 06 22:09:16.970598 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:16.970570 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l5rc4" Jul 06 22:09:16.970598 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:16.970591 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n9hgj" Jul 06 22:09:16.970804 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:16.970672 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-l5rc4" podUID="9ade138e-0e88-4087-8042-92fe2db1e1d8" Jul 06 22:09:16.970804 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:16.970743 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-n9hgj" podUID="8a83eb6f-f275-4562-b014-0b48c710b624" Jul 06 22:09:16.970804 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:16.970769 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-klc48" Jul 06 22:09:16.970903 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:16.970844 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-klc48" podUID="b42224bc-b677-43bf-94fb-fc75223b963a" Jul 06 22:09:18.270756 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:18.270727 2581 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-101.ec2.internal" event="NodeReady" Jul 06 22:09:18.271136 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:18.270841 2581 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 06 22:09:18.305478 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:18.305435 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-65c85c5ff6-4fxss"] Jul 06 22:09:18.307984 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:18.307969 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-65c85c5ff6-4fxss" Jul 06 22:09:18.311111 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:18.310524 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 06 22:09:18.311111 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:18.310825 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-kshfh\"" Jul 06 22:09:18.311111 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:18.310860 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 06 22:09:18.311313 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:18.311178 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 06 22:09:18.316531 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:18.316506 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-nfklb"] Jul 06 22:09:18.316678 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:18.316661 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 06 22:09:18.320155 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:18.320133 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-vf82r"] Jul 06 22:09:18.320373 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:18.320250 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-nfklb" Jul 06 22:09:18.322599 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:18.322579 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-65c85c5ff6-4fxss"] Jul 06 22:09:18.322699 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:18.322665 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-vf82r" Jul 06 22:09:18.323002 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:18.322444 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 06 22:09:18.323101 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:18.323021 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 06 22:09:18.323370 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:18.323353 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-ddjw7\"" Jul 06 22:09:18.324980 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:18.324909 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 06 22:09:18.325215 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:18.325054 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-fbkk6\"" Jul 06 22:09:18.325443 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:18.325392 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 06 22:09:18.325527 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:18.325456 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 06 22:09:18.328976 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:18.328953 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-vf82r"] Jul 06 22:09:18.336306 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:18.336285 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-nfklb"] Jul 06 22:09:18.444434 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:18.444397 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f9z6c\" (UniqueName: \"kubernetes.io/projected/da5ae4fc-9377-4d40-89cf-efc75bac039e-kube-api-access-f9z6c\") pod \"dns-default-nfklb\" (UID: \"da5ae4fc-9377-4d40-89cf-efc75bac039e\") " pod="openshift-dns/dns-default-nfklb" Jul 06 22:09:18.444580 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:18.444443 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/da5ae4fc-9377-4d40-89cf-efc75bac039e-tmp-dir\") pod \"dns-default-nfklb\" (UID: \"da5ae4fc-9377-4d40-89cf-efc75bac039e\") " pod="openshift-dns/dns-default-nfklb" Jul 06 22:09:18.444580 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:18.444506 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bfb8f404-9338-404f-81f9-beb0a5b26d8a-registry-tls\") pod \"image-registry-65c85c5ff6-4fxss\" (UID: \"bfb8f404-9338-404f-81f9-beb0a5b26d8a\") " pod="openshift-image-registry/image-registry-65c85c5ff6-4fxss" Jul 06 22:09:18.444580 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:18.444532 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/da5ae4fc-9377-4d40-89cf-efc75bac039e-metrics-tls\") pod \"dns-default-nfklb\" (UID: \"da5ae4fc-9377-4d40-89cf-efc75bac039e\") " pod="openshift-dns/dns-default-nfklb" Jul 06 22:09:18.444730 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:18.444580 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/bfb8f404-9338-404f-81f9-beb0a5b26d8a-installation-pull-secrets\") pod \"image-registry-65c85c5ff6-4fxss\" (UID: \"bfb8f404-9338-404f-81f9-beb0a5b26d8a\") " pod="openshift-image-registry/image-registry-65c85c5ff6-4fxss" Jul 06 22:09:18.444730 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:18.444607 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/638815a3-fb4e-41ac-9e74-7c3426d561f4-cert\") pod \"ingress-canary-vf82r\" (UID: \"638815a3-fb4e-41ac-9e74-7c3426d561f4\") " pod="openshift-ingress-canary/ingress-canary-vf82r" Jul 06 22:09:18.444730 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:18.444639 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/bfb8f404-9338-404f-81f9-beb0a5b26d8a-ca-trust-extracted\") pod \"image-registry-65c85c5ff6-4fxss\" (UID: \"bfb8f404-9338-404f-81f9-beb0a5b26d8a\") " pod="openshift-image-registry/image-registry-65c85c5ff6-4fxss" Jul 06 22:09:18.444730 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:18.444666 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bfb8f404-9338-404f-81f9-beb0a5b26d8a-bound-sa-token\") pod \"image-registry-65c85c5ff6-4fxss\" (UID: \"bfb8f404-9338-404f-81f9-beb0a5b26d8a\") " pod="openshift-image-registry/image-registry-65c85c5ff6-4fxss" Jul 06 22:09:18.444730 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:18.444688 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pvsz7\" (UniqueName: \"kubernetes.io/projected/638815a3-fb4e-41ac-9e74-7c3426d561f4-kube-api-access-pvsz7\") pod \"ingress-canary-vf82r\" (UID: \"638815a3-fb4e-41ac-9e74-7c3426d561f4\") " pod="openshift-ingress-canary/ingress-canary-vf82r" Jul 06 22:09:18.444730 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:18.444719 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/bfb8f404-9338-404f-81f9-beb0a5b26d8a-image-registry-private-configuration\") pod \"image-registry-65c85c5ff6-4fxss\" (UID: \"bfb8f404-9338-404f-81f9-beb0a5b26d8a\") " pod="openshift-image-registry/image-registry-65c85c5ff6-4fxss" Jul 06 22:09:18.444985 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:18.444744 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/bfb8f404-9338-404f-81f9-beb0a5b26d8a-registry-certificates\") pod \"image-registry-65c85c5ff6-4fxss\" (UID: \"bfb8f404-9338-404f-81f9-beb0a5b26d8a\") " pod="openshift-image-registry/image-registry-65c85c5ff6-4fxss" Jul 06 22:09:18.444985 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:18.444770 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d5gvp\" (UniqueName: \"kubernetes.io/projected/bfb8f404-9338-404f-81f9-beb0a5b26d8a-kube-api-access-d5gvp\") pod \"image-registry-65c85c5ff6-4fxss\" (UID: \"bfb8f404-9338-404f-81f9-beb0a5b26d8a\") " pod="openshift-image-registry/image-registry-65c85c5ff6-4fxss" Jul 06 22:09:18.444985 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:18.444853 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/da5ae4fc-9377-4d40-89cf-efc75bac039e-config-volume\") pod \"dns-default-nfklb\" (UID: \"da5ae4fc-9377-4d40-89cf-efc75bac039e\") " pod="openshift-dns/dns-default-nfklb" Jul 06 22:09:18.444985 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:18.444903 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bfb8f404-9338-404f-81f9-beb0a5b26d8a-trusted-ca\") pod \"image-registry-65c85c5ff6-4fxss\" (UID: \"bfb8f404-9338-404f-81f9-beb0a5b26d8a\") " pod="openshift-image-registry/image-registry-65c85c5ff6-4fxss" Jul 06 22:09:18.468532 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:18.468496 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:09:18.546224 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:18.546144 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/da5ae4fc-9377-4d40-89cf-efc75bac039e-config-volume\") pod \"dns-default-nfklb\" (UID: \"da5ae4fc-9377-4d40-89cf-efc75bac039e\") " pod="openshift-dns/dns-default-nfklb" Jul 06 22:09:18.546224 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:18.546194 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bfb8f404-9338-404f-81f9-beb0a5b26d8a-trusted-ca\") pod \"image-registry-65c85c5ff6-4fxss\" (UID: \"bfb8f404-9338-404f-81f9-beb0a5b26d8a\") " pod="openshift-image-registry/image-registry-65c85c5ff6-4fxss" Jul 06 22:09:18.546224 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:18.546224 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f9z6c\" (UniqueName: \"kubernetes.io/projected/da5ae4fc-9377-4d40-89cf-efc75bac039e-kube-api-access-f9z6c\") pod \"dns-default-nfklb\" (UID: \"da5ae4fc-9377-4d40-89cf-efc75bac039e\") " pod="openshift-dns/dns-default-nfklb" Jul 06 22:09:18.546481 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:18.546247 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/da5ae4fc-9377-4d40-89cf-efc75bac039e-tmp-dir\") pod \"dns-default-nfklb\" (UID: \"da5ae4fc-9377-4d40-89cf-efc75bac039e\") " pod="openshift-dns/dns-default-nfklb" Jul 06 22:09:18.546481 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:18.546286 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bfb8f404-9338-404f-81f9-beb0a5b26d8a-registry-tls\") pod \"image-registry-65c85c5ff6-4fxss\" (UID: \"bfb8f404-9338-404f-81f9-beb0a5b26d8a\") " pod="openshift-image-registry/image-registry-65c85c5ff6-4fxss" Jul 06 22:09:18.546481 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:18.546308 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/da5ae4fc-9377-4d40-89cf-efc75bac039e-metrics-tls\") pod \"dns-default-nfklb\" (UID: \"da5ae4fc-9377-4d40-89cf-efc75bac039e\") " pod="openshift-dns/dns-default-nfklb" Jul 06 22:09:18.546481 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:18.546350 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/bfb8f404-9338-404f-81f9-beb0a5b26d8a-installation-pull-secrets\") pod \"image-registry-65c85c5ff6-4fxss\" (UID: \"bfb8f404-9338-404f-81f9-beb0a5b26d8a\") " pod="openshift-image-registry/image-registry-65c85c5ff6-4fxss" Jul 06 22:09:18.546481 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:18.546373 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/638815a3-fb4e-41ac-9e74-7c3426d561f4-cert\") pod \"ingress-canary-vf82r\" (UID: \"638815a3-fb4e-41ac-9e74-7c3426d561f4\") " pod="openshift-ingress-canary/ingress-canary-vf82r" Jul 06 22:09:18.546481 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:18.546402 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/bfb8f404-9338-404f-81f9-beb0a5b26d8a-ca-trust-extracted\") pod \"image-registry-65c85c5ff6-4fxss\" (UID: \"bfb8f404-9338-404f-81f9-beb0a5b26d8a\") " pod="openshift-image-registry/image-registry-65c85c5ff6-4fxss" Jul 06 22:09:18.546481 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:18.546412 2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 22:09:18.546481 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:18.546433 2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-65c85c5ff6-4fxss: secret "image-registry-tls" not found Jul 06 22:09:18.546796 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:18.546492 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bfb8f404-9338-404f-81f9-beb0a5b26d8a-registry-tls podName:bfb8f404-9338-404f-81f9-beb0a5b26d8a nodeName:}" failed. No retries permitted until 2026-07-06 22:09:19.046472496 +0000 UTC m=+38.605267820 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bfb8f404-9338-404f-81f9-beb0a5b26d8a-registry-tls") pod "image-registry-65c85c5ff6-4fxss" (UID: "bfb8f404-9338-404f-81f9-beb0a5b26d8a") : secret "image-registry-tls" not found Jul 06 22:09:18.546796 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:18.546626 2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 22:09:18.546796 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:18.546671 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/da5ae4fc-9377-4d40-89cf-efc75bac039e-metrics-tls podName:da5ae4fc-9377-4d40-89cf-efc75bac039e nodeName:}" failed. No retries permitted until 2026-07-06 22:09:19.046657071 +0000 UTC m=+38.605452391 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/da5ae4fc-9377-4d40-89cf-efc75bac039e-metrics-tls") pod "dns-default-nfklb" (UID: "da5ae4fc-9377-4d40-89cf-efc75bac039e") : secret "dns-default-metrics-tls" not found Jul 06 22:09:18.546796 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:18.546710 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bfb8f404-9338-404f-81f9-beb0a5b26d8a-bound-sa-token\") pod \"image-registry-65c85c5ff6-4fxss\" (UID: \"bfb8f404-9338-404f-81f9-beb0a5b26d8a\") " pod="openshift-image-registry/image-registry-65c85c5ff6-4fxss" Jul 06 22:09:18.546796 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:18.546740 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pvsz7\" (UniqueName: \"kubernetes.io/projected/638815a3-fb4e-41ac-9e74-7c3426d561f4-kube-api-access-pvsz7\") pod \"ingress-canary-vf82r\" (UID: \"638815a3-fb4e-41ac-9e74-7c3426d561f4\") " pod="openshift-ingress-canary/ingress-canary-vf82r" Jul 06 22:09:18.546796 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:18.546772 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/bfb8f404-9338-404f-81f9-beb0a5b26d8a-image-registry-private-configuration\") pod \"image-registry-65c85c5ff6-4fxss\" (UID: \"bfb8f404-9338-404f-81f9-beb0a5b26d8a\") " pod="openshift-image-registry/image-registry-65c85c5ff6-4fxss" Jul 06 22:09:18.547082 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:18.546802 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/bfb8f404-9338-404f-81f9-beb0a5b26d8a-registry-certificates\") pod \"image-registry-65c85c5ff6-4fxss\" (UID: \"bfb8f404-9338-404f-81f9-beb0a5b26d8a\") " pod="openshift-image-registry/image-registry-65c85c5ff6-4fxss" Jul 06 22:09:18.547082 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:18.546825 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d5gvp\" (UniqueName: \"kubernetes.io/projected/bfb8f404-9338-404f-81f9-beb0a5b26d8a-kube-api-access-d5gvp\") pod \"image-registry-65c85c5ff6-4fxss\" (UID: \"bfb8f404-9338-404f-81f9-beb0a5b26d8a\") " pod="openshift-image-registry/image-registry-65c85c5ff6-4fxss" Jul 06 22:09:18.547082 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:18.546946 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/da5ae4fc-9377-4d40-89cf-efc75bac039e-tmp-dir\") pod \"dns-default-nfklb\" (UID: \"da5ae4fc-9377-4d40-89cf-efc75bac039e\") " pod="openshift-dns/dns-default-nfklb" Jul 06 22:09:18.547358 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:18.547163 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/da5ae4fc-9377-4d40-89cf-efc75bac039e-config-volume\") pod \"dns-default-nfklb\" (UID: \"da5ae4fc-9377-4d40-89cf-efc75bac039e\") " pod="openshift-dns/dns-default-nfklb" Jul 06 22:09:18.547358 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:18.547040 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/bfb8f404-9338-404f-81f9-beb0a5b26d8a-ca-trust-extracted\") pod \"image-registry-65c85c5ff6-4fxss\" (UID: \"bfb8f404-9338-404f-81f9-beb0a5b26d8a\") " pod="openshift-image-registry/image-registry-65c85c5ff6-4fxss" Jul 06 22:09:18.547513 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:18.547396 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bfb8f404-9338-404f-81f9-beb0a5b26d8a-trusted-ca\") pod \"image-registry-65c85c5ff6-4fxss\" (UID: \"bfb8f404-9338-404f-81f9-beb0a5b26d8a\") " pod="openshift-image-registry/image-registry-65c85c5ff6-4fxss" Jul 06 22:09:18.547646 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:18.547625 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/bfb8f404-9338-404f-81f9-beb0a5b26d8a-registry-certificates\") pod \"image-registry-65c85c5ff6-4fxss\" (UID: \"bfb8f404-9338-404f-81f9-beb0a5b26d8a\") " pod="openshift-image-registry/image-registry-65c85c5ff6-4fxss" Jul 06 22:09:18.551538 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:18.551513 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/bfb8f404-9338-404f-81f9-beb0a5b26d8a-image-registry-private-configuration\") pod \"image-registry-65c85c5ff6-4fxss\" (UID: \"bfb8f404-9338-404f-81f9-beb0a5b26d8a\") " pod="openshift-image-registry/image-registry-65c85c5ff6-4fxss" Jul 06 22:09:18.551538 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:18.551533 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/638815a3-fb4e-41ac-9e74-7c3426d561f4-cert\") pod \"ingress-canary-vf82r\" (UID: \"638815a3-fb4e-41ac-9e74-7c3426d561f4\") " pod="openshift-ingress-canary/ingress-canary-vf82r" Jul 06 22:09:18.551678 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:18.551555 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/bfb8f404-9338-404f-81f9-beb0a5b26d8a-installation-pull-secrets\") pod \"image-registry-65c85c5ff6-4fxss\" (UID: \"bfb8f404-9338-404f-81f9-beb0a5b26d8a\") " pod="openshift-image-registry/image-registry-65c85c5ff6-4fxss" Jul 06 22:09:18.556295 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:18.556264 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f9z6c\" (UniqueName: \"kubernetes.io/projected/da5ae4fc-9377-4d40-89cf-efc75bac039e-kube-api-access-f9z6c\") pod \"dns-default-nfklb\" (UID: \"da5ae4fc-9377-4d40-89cf-efc75bac039e\") " pod="openshift-dns/dns-default-nfklb" Jul 06 22:09:18.558018 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:18.557464 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pvsz7\" (UniqueName: \"kubernetes.io/projected/638815a3-fb4e-41ac-9e74-7c3426d561f4-kube-api-access-pvsz7\") pod \"ingress-canary-vf82r\" (UID: \"638815a3-fb4e-41ac-9e74-7c3426d561f4\") " pod="openshift-ingress-canary/ingress-canary-vf82r" Jul 06 22:09:18.558018 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:18.557803 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d5gvp\" (UniqueName: \"kubernetes.io/projected/bfb8f404-9338-404f-81f9-beb0a5b26d8a-kube-api-access-d5gvp\") pod \"image-registry-65c85c5ff6-4fxss\" (UID: \"bfb8f404-9338-404f-81f9-beb0a5b26d8a\") " pod="openshift-image-registry/image-registry-65c85c5ff6-4fxss" Jul 06 22:09:18.558891 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:18.558866 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bfb8f404-9338-404f-81f9-beb0a5b26d8a-bound-sa-token\") pod \"image-registry-65c85c5ff6-4fxss\" (UID: \"bfb8f404-9338-404f-81f9-beb0a5b26d8a\") " pod="openshift-image-registry/image-registry-65c85c5ff6-4fxss" Jul 06 22:09:18.564338 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:18.564310 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:09:18.639395 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:18.639363 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-vf82r" Jul 06 22:09:18.824431 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:18.824364 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-vf82r"] Jul 06 22:09:18.828090 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:09:18.828046 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod638815a3_fb4e_41ac_9e74_7c3426d561f4.slice/crio-f12788b610f228e5597d2c900468fcdd2f65554da83637b4e7f4bdd71d553582 WatchSource:0}: Error finding container f12788b610f228e5597d2c900468fcdd2f65554da83637b4e7f4bdd71d553582: Status 404 returned error can't find the container with id f12788b610f228e5597d2c900468fcdd2f65554da83637b4e7f4bdd71d553582 Jul 06 22:09:18.970102 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:18.970047 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l5rc4" Jul 06 22:09:18.970279 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:18.970111 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-klc48" Jul 06 22:09:18.970279 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:18.970053 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n9hgj" Jul 06 22:09:18.973455 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:18.973237 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 06 22:09:18.973455 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:18.973253 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-pk242\"" Jul 06 22:09:18.973455 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:18.973306 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 06 22:09:18.973455 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:18.973247 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 06 22:09:18.973455 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:18.973378 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-dftmz\"" Jul 06 22:09:18.973853 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:18.973723 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 06 22:09:19.051748 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:19.051713 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bfb8f404-9338-404f-81f9-beb0a5b26d8a-registry-tls\") pod \"image-registry-65c85c5ff6-4fxss\" (UID: \"bfb8f404-9338-404f-81f9-beb0a5b26d8a\") " pod="openshift-image-registry/image-registry-65c85c5ff6-4fxss" Jul 06 22:09:19.051748 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:19.051749 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/da5ae4fc-9377-4d40-89cf-efc75bac039e-metrics-tls\") pod \"dns-default-nfklb\" (UID: \"da5ae4fc-9377-4d40-89cf-efc75bac039e\") " pod="openshift-dns/dns-default-nfklb" Jul 06 22:09:19.051932 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:19.051856 2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 22:09:19.051932 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:19.051862 2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 22:09:19.051932 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:19.051880 2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-65c85c5ff6-4fxss: secret "image-registry-tls" not found Jul 06 22:09:19.051932 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:19.051918 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/da5ae4fc-9377-4d40-89cf-efc75bac039e-metrics-tls podName:da5ae4fc-9377-4d40-89cf-efc75bac039e nodeName:}" failed. No retries permitted until 2026-07-06 22:09:20.051903425 +0000 UTC m=+39.610698724 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/da5ae4fc-9377-4d40-89cf-efc75bac039e-metrics-tls") pod "dns-default-nfklb" (UID: "da5ae4fc-9377-4d40-89cf-efc75bac039e") : secret "dns-default-metrics-tls" not found Jul 06 22:09:19.052095 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:19.051969 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bfb8f404-9338-404f-81f9-beb0a5b26d8a-registry-tls podName:bfb8f404-9338-404f-81f9-beb0a5b26d8a nodeName:}" failed. No retries permitted until 2026-07-06 22:09:20.05195395 +0000 UTC m=+39.610749254 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bfb8f404-9338-404f-81f9-beb0a5b26d8a-registry-tls") pod "image-registry-65c85c5ff6-4fxss" (UID: "bfb8f404-9338-404f-81f9-beb0a5b26d8a") : secret "image-registry-tls" not found Jul 06 22:09:19.395317 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:19.395283 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-vf82r" event={"ID":"638815a3-fb4e-41ac-9e74-7c3426d561f4","Type":"ContainerStarted","Data":"f12788b610f228e5597d2c900468fcdd2f65554da83637b4e7f4bdd71d553582"} Jul 06 22:09:20.060294 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:20.060253 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bfb8f404-9338-404f-81f9-beb0a5b26d8a-registry-tls\") pod \"image-registry-65c85c5ff6-4fxss\" (UID: \"bfb8f404-9338-404f-81f9-beb0a5b26d8a\") " pod="openshift-image-registry/image-registry-65c85c5ff6-4fxss" Jul 06 22:09:20.060478 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:20.060304 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/da5ae4fc-9377-4d40-89cf-efc75bac039e-metrics-tls\") pod \"dns-default-nfklb\" (UID: \"da5ae4fc-9377-4d40-89cf-efc75bac039e\") " pod="openshift-dns/dns-default-nfklb" Jul 06 22:09:20.060478 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:20.060416 2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 22:09:20.060478 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:20.060436 2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-65c85c5ff6-4fxss: secret "image-registry-tls" not found Jul 06 22:09:20.060478 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:20.060453 2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 22:09:20.060674 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:20.060510 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bfb8f404-9338-404f-81f9-beb0a5b26d8a-registry-tls podName:bfb8f404-9338-404f-81f9-beb0a5b26d8a nodeName:}" failed. No retries permitted until 2026-07-06 22:09:22.060488871 +0000 UTC m=+41.619284185 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bfb8f404-9338-404f-81f9-beb0a5b26d8a-registry-tls") pod "image-registry-65c85c5ff6-4fxss" (UID: "bfb8f404-9338-404f-81f9-beb0a5b26d8a") : secret "image-registry-tls" not found Jul 06 22:09:20.060674 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:20.060527 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/da5ae4fc-9377-4d40-89cf-efc75bac039e-metrics-tls podName:da5ae4fc-9377-4d40-89cf-efc75bac039e nodeName:}" failed. No retries permitted until 2026-07-06 22:09:22.06051965 +0000 UTC m=+41.619314952 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/da5ae4fc-9377-4d40-89cf-efc75bac039e-metrics-tls") pod "dns-default-nfklb" (UID: "da5ae4fc-9377-4d40-89cf-efc75bac039e") : secret "dns-default-metrics-tls" not found Jul 06 22:09:22.076346 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:22.076122 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bfb8f404-9338-404f-81f9-beb0a5b26d8a-registry-tls\") pod \"image-registry-65c85c5ff6-4fxss\" (UID: \"bfb8f404-9338-404f-81f9-beb0a5b26d8a\") " pod="openshift-image-registry/image-registry-65c85c5ff6-4fxss" Jul 06 22:09:22.076817 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:22.076364 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/da5ae4fc-9377-4d40-89cf-efc75bac039e-metrics-tls\") pod \"dns-default-nfklb\" (UID: \"da5ae4fc-9377-4d40-89cf-efc75bac039e\") " pod="openshift-dns/dns-default-nfklb" Jul 06 22:09:22.076817 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:22.076285 2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 22:09:22.076817 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:22.076448 2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-65c85c5ff6-4fxss: secret "image-registry-tls" not found Jul 06 22:09:22.076817 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:22.076519 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bfb8f404-9338-404f-81f9-beb0a5b26d8a-registry-tls podName:bfb8f404-9338-404f-81f9-beb0a5b26d8a nodeName:}" failed. No retries permitted until 2026-07-06 22:09:26.076497266 +0000 UTC m=+45.635292579 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bfb8f404-9338-404f-81f9-beb0a5b26d8a-registry-tls") pod "image-registry-65c85c5ff6-4fxss" (UID: "bfb8f404-9338-404f-81f9-beb0a5b26d8a") : secret "image-registry-tls" not found Jul 06 22:09:22.076817 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:22.076549 2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 22:09:22.076817 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:22.076600 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/da5ae4fc-9377-4d40-89cf-efc75bac039e-metrics-tls podName:da5ae4fc-9377-4d40-89cf-efc75bac039e nodeName:}" failed. No retries permitted until 2026-07-06 22:09:26.076588364 +0000 UTC m=+45.635383685 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/da5ae4fc-9377-4d40-89cf-efc75bac039e-metrics-tls") pod "dns-default-nfklb" (UID: "da5ae4fc-9377-4d40-89cf-efc75bac039e") : secret "dns-default-metrics-tls" not found Jul 06 22:09:22.542370 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:22.542338 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:09:22.606348 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:22.606310 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:09:22.727044 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:22.726986 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:09:22.828603 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:22.828515 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:09:22.928815 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:22.928779 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:09:23.028969 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:23.028929 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:09:23.134594 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:23.134503 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:09:23.232006 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:23.231977 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:09:23.296200 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:23.296165 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:09:23.784003 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:23.783974 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:09:23.849246 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:23.849164 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:09:23.997501 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:23.997466 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:09:24.097751 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:24.097716 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:09:24.390245 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:24.390210 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:09:24.409154 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:24.409127 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-vf82r" event={"ID":"638815a3-fb4e-41ac-9e74-7c3426d561f4","Type":"ContainerStarted","Data":"af9ede498878ec17368feb6961d27e35235a4b7c558f8d0377eea55b9a44787f"} Jul 06 22:09:24.411584 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:24.411562 2581 generic.go:356] "Generic (PLEG): container finished" podID="f1de085f-40f2-4f95-a5bb-78aba60de69e" containerID="1a88c9a043b0dfc90ee09eb2f8784cd5252c4eb97cfee67f701a3963080db6d7" exitCode=0 Jul 06 22:09:24.411719 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:24.411601 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5nrhc" event={"ID":"f1de085f-40f2-4f95-a5bb-78aba60de69e","Type":"ContainerDied","Data":"1a88c9a043b0dfc90ee09eb2f8784cd5252c4eb97cfee67f701a3963080db6d7"} Jul 06 22:09:24.426952 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:24.426908 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-vf82r" podStartSLOduration=1.678158907 podStartE2EDuration="6.426895661s" totalImagesPullingTime="4.748736754s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:09:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:09:24.425738499 +0000 UTC m=+43.984533819" watchObservedRunningTime="2026-07-06 22:09:24.426895661 +0000 UTC m=+43.985690978" Jul 06 22:09:24.455609 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:24.455578 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:09:25.416955 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:25.416923 2581 generic.go:356] "Generic (PLEG): container finished" podID="f1de085f-40f2-4f95-a5bb-78aba60de69e" containerID="e09ffffc54450dd8a293bdae33cbe256e57030c888addd8de69058cac871912e" exitCode=0 Jul 06 22:09:25.417438 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:25.417003 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5nrhc" event={"ID":"f1de085f-40f2-4f95-a5bb-78aba60de69e","Type":"ContainerDied","Data":"e09ffffc54450dd8a293bdae33cbe256e57030c888addd8de69058cac871912e"} Jul 06 22:09:25.614506 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:25.614475 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:09:26.103307 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:26.103271 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bfb8f404-9338-404f-81f9-beb0a5b26d8a-registry-tls\") pod \"image-registry-65c85c5ff6-4fxss\" (UID: \"bfb8f404-9338-404f-81f9-beb0a5b26d8a\") " pod="openshift-image-registry/image-registry-65c85c5ff6-4fxss" Jul 06 22:09:26.103307 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:26.103308 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/da5ae4fc-9377-4d40-89cf-efc75bac039e-metrics-tls\") pod \"dns-default-nfklb\" (UID: \"da5ae4fc-9377-4d40-89cf-efc75bac039e\") " pod="openshift-dns/dns-default-nfklb" Jul 06 22:09:26.103512 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:26.103413 2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 22:09:26.103512 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:26.103418 2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 22:09:26.103512 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:26.103434 2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-65c85c5ff6-4fxss: secret "image-registry-tls" not found Jul 06 22:09:26.103512 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:26.103462 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/da5ae4fc-9377-4d40-89cf-efc75bac039e-metrics-tls podName:da5ae4fc-9377-4d40-89cf-efc75bac039e nodeName:}" failed. No retries permitted until 2026-07-06 22:09:34.103449932 +0000 UTC m=+53.662245232 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/da5ae4fc-9377-4d40-89cf-efc75bac039e-metrics-tls") pod "dns-default-nfklb" (UID: "da5ae4fc-9377-4d40-89cf-efc75bac039e") : secret "dns-default-metrics-tls" not found Jul 06 22:09:26.103512 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:26.103475 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bfb8f404-9338-404f-81f9-beb0a5b26d8a-registry-tls podName:bfb8f404-9338-404f-81f9-beb0a5b26d8a nodeName:}" failed. No retries permitted until 2026-07-06 22:09:34.103469402 +0000 UTC m=+53.662264701 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bfb8f404-9338-404f-81f9-beb0a5b26d8a-registry-tls") pod "image-registry-65c85c5ff6-4fxss" (UID: "bfb8f404-9338-404f-81f9-beb0a5b26d8a") : secret "image-registry-tls" not found Jul 06 22:09:26.423195 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:26.423113 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5nrhc" event={"ID":"f1de085f-40f2-4f95-a5bb-78aba60de69e","Type":"ContainerStarted","Data":"71ee0cd222b67ec62ad051d661919d87884fe7b74ea3162f9dc4763b1516185b"} Jul 06 22:09:26.449205 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:26.449155 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-5nrhc" podStartSLOduration=8.118106191 podStartE2EDuration="45.449137874s" totalImagesPullingTime="37.331031683s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 22:08:41 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:09:26.448988918 +0000 UTC m=+46.007784239" watchObservedRunningTime="2026-07-06 22:09:26.449137874 +0000 UTC m=+46.007933198" Jul 06 22:09:26.574412 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:26.574376 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:09:26.675338 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:26.675301 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:09:26.906988 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:26.906952 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:09:31.692996 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:31.692948 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:09:31.792579 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:31.792550 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:09:31.894732 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:31.894697 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:09:34.160572 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.160530 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bfb8f404-9338-404f-81f9-beb0a5b26d8a-registry-tls\") pod \"image-registry-65c85c5ff6-4fxss\" (UID: \"bfb8f404-9338-404f-81f9-beb0a5b26d8a\") " pod="openshift-image-registry/image-registry-65c85c5ff6-4fxss" Jul 06 22:09:34.160572 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.160573 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/da5ae4fc-9377-4d40-89cf-efc75bac039e-metrics-tls\") pod \"dns-default-nfklb\" (UID: \"da5ae4fc-9377-4d40-89cf-efc75bac039e\") " pod="openshift-dns/dns-default-nfklb" Jul 06 22:09:34.161089 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:34.160676 2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 22:09:34.161089 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:34.160678 2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 22:09:34.161089 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:34.160748 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/da5ae4fc-9377-4d40-89cf-efc75bac039e-metrics-tls podName:da5ae4fc-9377-4d40-89cf-efc75bac039e nodeName:}" failed. No retries permitted until 2026-07-06 22:09:50.160727914 +0000 UTC m=+69.719523231 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/da5ae4fc-9377-4d40-89cf-efc75bac039e-metrics-tls") pod "dns-default-nfklb" (UID: "da5ae4fc-9377-4d40-89cf-efc75bac039e") : secret "dns-default-metrics-tls" not found Jul 06 22:09:34.161089 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:34.160750 2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-65c85c5ff6-4fxss: secret "image-registry-tls" not found Jul 06 22:09:34.161089 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:34.160830 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bfb8f404-9338-404f-81f9-beb0a5b26d8a-registry-tls podName:bfb8f404-9338-404f-81f9-beb0a5b26d8a nodeName:}" failed. No retries permitted until 2026-07-06 22:09:50.160818007 +0000 UTC m=+69.719613336 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bfb8f404-9338-404f-81f9-beb0a5b26d8a-registry-tls") pod "image-registry-65c85c5ff6-4fxss" (UID: "bfb8f404-9338-404f-81f9-beb0a5b26d8a") : secret "image-registry-tls" not found Jul 06 22:09:34.724576 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.724547 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-dj4ks"] Jul 06 22:09:34.738241 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.738216 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-7hbvp"] Jul 06 22:09:34.738363 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.738305 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-dj4ks" Jul 06 22:09:34.740838 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.740817 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 06 22:09:34.741939 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.741915 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 06 22:09:34.742020 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.741951 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 06 22:09:34.742020 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.741990 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-6vl8x\"" Jul 06 22:09:34.742366 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.742349 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 06 22:09:34.758835 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.758793 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-dftxz"] Jul 06 22:09:34.758969 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.758917 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-7hbvp" Jul 06 22:09:34.761407 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.761389 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-gwvtn\"" Jul 06 22:09:34.766480 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.766460 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9kmwv\" (UniqueName: \"kubernetes.io/projected/c407c330-5340-4bc6-bf8a-e4dc003ea079-kube-api-access-9kmwv\") pod \"network-check-source-989f8b8dc-7hbvp\" (UID: \"c407c330-5340-4bc6-bf8a-e4dc003ea079\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-7hbvp" Jul 06 22:09:34.766568 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.766493 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/01137b22-9e43-47ba-aee5-e4c51fd35e4d-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-dj4ks\" (UID: \"01137b22-9e43-47ba-aee5-e4c51fd35e4d\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-dj4ks" Jul 06 22:09:34.766568 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.766520 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/01137b22-9e43-47ba-aee5-e4c51fd35e4d-config\") pod \"service-ca-operator-5d46d9ffd4-dj4ks\" (UID: \"01137b22-9e43-47ba-aee5-e4c51fd35e4d\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-dj4ks" Jul 06 22:09:34.766568 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.766542 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xjsrg\" (UniqueName: \"kubernetes.io/projected/01137b22-9e43-47ba-aee5-e4c51fd35e4d-kube-api-access-xjsrg\") pod \"service-ca-operator-5d46d9ffd4-dj4ks\" (UID: \"01137b22-9e43-47ba-aee5-e4c51fd35e4d\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-dj4ks" Jul 06 22:09:34.778880 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.778859 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-2pphn"] Jul 06 22:09:34.778945 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.778912 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-dftxz" Jul 06 22:09:34.781617 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.781590 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 06 22:09:34.781617 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.781600 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 06 22:09:34.781748 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.781652 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-tts4r\"" Jul 06 22:09:34.795020 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.795003 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-rcn5q"] Jul 06 22:09:34.795146 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.795127 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-2pphn" Jul 06 22:09:34.797624 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.797606 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 06 22:09:34.797710 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.797647 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-h8qvr\"" Jul 06 22:09:34.797768 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.797709 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 06 22:09:34.797768 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.797725 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 06 22:09:34.797768 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.797734 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 06 22:09:34.803051 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.803032 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 06 22:09:34.809208 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.809191 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-dj4ks"] Jul 06 22:09:34.809277 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.809218 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-7hbvp"] Jul 06 22:09:34.809277 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.809234 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-dftxz"] Jul 06 22:09:34.809277 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.809246 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-2pphn"] Jul 06 22:09:34.809277 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.809258 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-rcn5q"] Jul 06 22:09:34.809400 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.809297 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-rcn5q" Jul 06 22:09:34.813173 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.813159 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 06 22:09:34.813494 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.813478 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 06 22:09:34.813746 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.813729 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-x5xtw\"" Jul 06 22:09:34.813833 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.813782 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 06 22:09:34.813963 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.813950 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 06 22:09:34.819930 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.819914 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 06 22:09:34.839045 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.839023 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-hfs2q"] Jul 06 22:09:34.853821 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.853789 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-24gv2"] Jul 06 22:09:34.853936 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.853894 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-hfs2q" Jul 06 22:09:34.857733 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.857701 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-z6kzh\"" Jul 06 22:09:34.857834 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.857764 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 06 22:09:34.858026 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.857998 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 06 22:09:34.858105 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.858038 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 06 22:09:34.858331 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.858289 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 06 22:09:34.866973 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.866949 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9kmwv\" (UniqueName: \"kubernetes.io/projected/c407c330-5340-4bc6-bf8a-e4dc003ea079-kube-api-access-9kmwv\") pod \"network-check-source-989f8b8dc-7hbvp\" (UID: \"c407c330-5340-4bc6-bf8a-e4dc003ea079\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-7hbvp" Jul 06 22:09:34.867059 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.866982 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fe3d4288-3a9d-413c-be44-ac70bd65b45f-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-2pphn\" (UID: \"fe3d4288-3a9d-413c-be44-ac70bd65b45f\") " pod="openshift-insights/insights-operator-5c847dcfc4-2pphn" Jul 06 22:09:34.867059 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.867002 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/01137b22-9e43-47ba-aee5-e4c51fd35e4d-config\") pod \"service-ca-operator-5d46d9ffd4-dj4ks\" (UID: \"01137b22-9e43-47ba-aee5-e4c51fd35e4d\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-dj4ks" Jul 06 22:09:34.867059 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.867020 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fe3d4288-3a9d-413c-be44-ac70bd65b45f-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-2pphn\" (UID: \"fe3d4288-3a9d-413c-be44-ac70bd65b45f\") " pod="openshift-insights/insights-operator-5c847dcfc4-2pphn" Jul 06 22:09:34.867059 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.867045 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jqk49\" (UniqueName: \"kubernetes.io/projected/fe3d4288-3a9d-413c-be44-ac70bd65b45f-kube-api-access-jqk49\") pod \"insights-operator-5c847dcfc4-2pphn\" (UID: \"fe3d4288-3a9d-413c-be44-ac70bd65b45f\") " pod="openshift-insights/insights-operator-5c847dcfc4-2pphn" Jul 06 22:09:34.867239 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.867178 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/86b07166-2f34-443b-9c61-86b7d950ad7b-config\") pod \"console-operator-68b757fb67-rcn5q\" (UID: \"86b07166-2f34-443b-9c61-86b7d950ad7b\") " pod="openshift-console-operator/console-operator-68b757fb67-rcn5q" Jul 06 22:09:34.867282 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.867266 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xjsrg\" (UniqueName: \"kubernetes.io/projected/01137b22-9e43-47ba-aee5-e4c51fd35e4d-kube-api-access-xjsrg\") pod \"service-ca-operator-5d46d9ffd4-dj4ks\" (UID: \"01137b22-9e43-47ba-aee5-e4c51fd35e4d\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-dj4ks" Jul 06 22:09:34.867317 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.867285 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-57mqp\" (UniqueName: \"kubernetes.io/projected/c39dd055-b80e-45d4-9882-ae689d849208-kube-api-access-57mqp\") pod \"volume-data-source-validator-5d456fdbcd-dftxz\" (UID: \"c39dd055-b80e-45d4-9882-ae689d849208\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-dftxz" Jul 06 22:09:34.867317 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.867312 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fe3d4288-3a9d-413c-be44-ac70bd65b45f-serving-cert\") pod \"insights-operator-5c847dcfc4-2pphn\" (UID: \"fe3d4288-3a9d-413c-be44-ac70bd65b45f\") " pod="openshift-insights/insights-operator-5c847dcfc4-2pphn" Jul 06 22:09:34.867379 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.867332 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/86b07166-2f34-443b-9c61-86b7d950ad7b-trusted-ca\") pod \"console-operator-68b757fb67-rcn5q\" (UID: \"86b07166-2f34-443b-9c61-86b7d950ad7b\") " pod="openshift-console-operator/console-operator-68b757fb67-rcn5q" Jul 06 22:09:34.867379 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.867351 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d3642934-8f79-4aed-bdfe-5a6777a655aa-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-hfs2q\" (UID: \"d3642934-8f79-4aed-bdfe-5a6777a655aa\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-hfs2q" Jul 06 22:09:34.867379 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.867370 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nbksk\" (UniqueName: \"kubernetes.io/projected/d3642934-8f79-4aed-bdfe-5a6777a655aa-kube-api-access-nbksk\") pod \"kube-storage-version-migrator-operator-846bf668f7-hfs2q\" (UID: \"d3642934-8f79-4aed-bdfe-5a6777a655aa\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-hfs2q" Jul 06 22:09:34.867481 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.867393 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-65qc8\" (UniqueName: \"kubernetes.io/projected/86b07166-2f34-443b-9c61-86b7d950ad7b-kube-api-access-65qc8\") pod \"console-operator-68b757fb67-rcn5q\" (UID: \"86b07166-2f34-443b-9c61-86b7d950ad7b\") " pod="openshift-console-operator/console-operator-68b757fb67-rcn5q" Jul 06 22:09:34.867481 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.867408 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/fe3d4288-3a9d-413c-be44-ac70bd65b45f-snapshots\") pod \"insights-operator-5c847dcfc4-2pphn\" (UID: \"fe3d4288-3a9d-413c-be44-ac70bd65b45f\") " pod="openshift-insights/insights-operator-5c847dcfc4-2pphn" Jul 06 22:09:34.867481 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.867470 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/01137b22-9e43-47ba-aee5-e4c51fd35e4d-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-dj4ks\" (UID: \"01137b22-9e43-47ba-aee5-e4c51fd35e4d\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-dj4ks" Jul 06 22:09:34.867575 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.867511 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d3642934-8f79-4aed-bdfe-5a6777a655aa-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-hfs2q\" (UID: \"d3642934-8f79-4aed-bdfe-5a6777a655aa\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-hfs2q" Jul 06 22:09:34.867637 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.867625 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fe3d4288-3a9d-413c-be44-ac70bd65b45f-tmp\") pod \"insights-operator-5c847dcfc4-2pphn\" (UID: \"fe3d4288-3a9d-413c-be44-ac70bd65b45f\") " pod="openshift-insights/insights-operator-5c847dcfc4-2pphn" Jul 06 22:09:34.867673 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.867660 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/86b07166-2f34-443b-9c61-86b7d950ad7b-serving-cert\") pod \"console-operator-68b757fb67-rcn5q\" (UID: \"86b07166-2f34-443b-9c61-86b7d950ad7b\") " pod="openshift-console-operator/console-operator-68b757fb67-rcn5q" Jul 06 22:09:34.868000 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.867981 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/01137b22-9e43-47ba-aee5-e4c51fd35e4d-config\") pod \"service-ca-operator-5d46d9ffd4-dj4ks\" (UID: \"01137b22-9e43-47ba-aee5-e4c51fd35e4d\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-dj4ks" Jul 06 22:09:34.870410 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.870394 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/01137b22-9e43-47ba-aee5-e4c51fd35e4d-serving-cert\") pod \"service-ca-operator-5d46d9ffd4-dj4ks\" (UID: \"01137b22-9e43-47ba-aee5-e4c51fd35e4d\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-dj4ks" Jul 06 22:09:34.873885 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.873867 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-dkbm6"] Jul 06 22:09:34.874009 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.873986 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-24gv2" Jul 06 22:09:34.878712 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.878026 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 06 22:09:34.878712 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.878266 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 06 22:09:34.879494 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.879150 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 06 22:09:34.880119 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.880096 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 06 22:09:34.880255 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.880233 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-957fd\"" Jul 06 22:09:34.882597 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.882570 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xjsrg\" (UniqueName: \"kubernetes.io/projected/01137b22-9e43-47ba-aee5-e4c51fd35e4d-kube-api-access-xjsrg\") pod \"service-ca-operator-5d46d9ffd4-dj4ks\" (UID: \"01137b22-9e43-47ba-aee5-e4c51fd35e4d\") " pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-dj4ks" Jul 06 22:09:34.882776 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.882761 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9kmwv\" (UniqueName: \"kubernetes.io/projected/c407c330-5340-4bc6-bf8a-e4dc003ea079-kube-api-access-9kmwv\") pod \"network-check-source-989f8b8dc-7hbvp\" (UID: \"c407c330-5340-4bc6-bf8a-e4dc003ea079\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-7hbvp" Jul 06 22:09:34.896355 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.896335 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-78989f99df-sxq46"] Jul 06 22:09:34.896432 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.896400 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-dkbm6" Jul 06 22:09:34.898876 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.898861 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-txvjp\"" Jul 06 22:09:34.898972 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.898960 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 06 22:09:34.899171 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.899158 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 06 22:09:34.899334 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.899321 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 06 22:09:34.916740 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.916720 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-hfs2q"] Jul 06 22:09:34.916820 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.916747 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-24gv2"] Jul 06 22:09:34.916820 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.916762 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-dkbm6"] Jul 06 22:09:34.916820 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.916778 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-78989f99df-sxq46"] Jul 06 22:09:34.916906 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.916835 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-78989f99df-sxq46" Jul 06 22:09:34.919455 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.919439 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 06 22:09:34.919545 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.919468 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 06 22:09:34.919545 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.919472 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 06 22:09:34.919545 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.919518 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 06 22:09:34.919766 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.919549 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 06 22:09:34.919766 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.919518 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-927dm\"" Jul 06 22:09:34.919856 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.919843 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 06 22:09:34.968860 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.968825 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-65qc8\" (UniqueName: \"kubernetes.io/projected/86b07166-2f34-443b-9c61-86b7d950ad7b-kube-api-access-65qc8\") pod \"console-operator-68b757fb67-rcn5q\" (UID: \"86b07166-2f34-443b-9c61-86b7d950ad7b\") " pod="openshift-console-operator/console-operator-68b757fb67-rcn5q" Jul 06 22:09:34.968860 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.968861 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/fe3d4288-3a9d-413c-be44-ac70bd65b45f-snapshots\") pod \"insights-operator-5c847dcfc4-2pphn\" (UID: \"fe3d4288-3a9d-413c-be44-ac70bd65b45f\") " pod="openshift-insights/insights-operator-5c847dcfc4-2pphn" Jul 06 22:09:34.969049 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.968890 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d3642934-8f79-4aed-bdfe-5a6777a655aa-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-hfs2q\" (UID: \"d3642934-8f79-4aed-bdfe-5a6777a655aa\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-hfs2q" Jul 06 22:09:34.969049 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.968921 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6fdb1134-ed1c-4505-a741-6e99dba1e2c8-service-ca-bundle\") pod \"router-default-78989f99df-sxq46\" (UID: \"6fdb1134-ed1c-4505-a741-6e99dba1e2c8\") " pod="openshift-ingress/router-default-78989f99df-sxq46" Jul 06 22:09:34.969049 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.968946 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fe3d4288-3a9d-413c-be44-ac70bd65b45f-tmp\") pod \"insights-operator-5c847dcfc4-2pphn\" (UID: \"fe3d4288-3a9d-413c-be44-ac70bd65b45f\") " pod="openshift-insights/insights-operator-5c847dcfc4-2pphn" Jul 06 22:09:34.969049 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.968971 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/179af188-f2c6-46da-9d1d-b15ce8fcb039-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-24gv2\" (UID: \"179af188-f2c6-46da-9d1d-b15ce8fcb039\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-24gv2" Jul 06 22:09:34.969049 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.969020 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/86b07166-2f34-443b-9c61-86b7d950ad7b-serving-cert\") pod \"console-operator-68b757fb67-rcn5q\" (UID: \"86b07166-2f34-443b-9c61-86b7d950ad7b\") " pod="openshift-console-operator/console-operator-68b757fb67-rcn5q" Jul 06 22:09:34.969308 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.969227 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fe3d4288-3a9d-413c-be44-ac70bd65b45f-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-2pphn\" (UID: \"fe3d4288-3a9d-413c-be44-ac70bd65b45f\") " pod="openshift-insights/insights-operator-5c847dcfc4-2pphn" Jul 06 22:09:34.969308 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.969283 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fe3d4288-3a9d-413c-be44-ac70bd65b45f-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-2pphn\" (UID: \"fe3d4288-3a9d-413c-be44-ac70bd65b45f\") " pod="openshift-insights/insights-operator-5c847dcfc4-2pphn" Jul 06 22:09:34.969407 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.969314 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jqk49\" (UniqueName: \"kubernetes.io/projected/fe3d4288-3a9d-413c-be44-ac70bd65b45f-kube-api-access-jqk49\") pod \"insights-operator-5c847dcfc4-2pphn\" (UID: \"fe3d4288-3a9d-413c-be44-ac70bd65b45f\") " pod="openshift-insights/insights-operator-5c847dcfc4-2pphn" Jul 06 22:09:34.969407 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.969342 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/86b07166-2f34-443b-9c61-86b7d950ad7b-config\") pod \"console-operator-68b757fb67-rcn5q\" (UID: \"86b07166-2f34-443b-9c61-86b7d950ad7b\") " pod="openshift-console-operator/console-operator-68b757fb67-rcn5q" Jul 06 22:09:34.969407 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.969374 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-747zh\" (UniqueName: \"kubernetes.io/projected/179af188-f2c6-46da-9d1d-b15ce8fcb039-kube-api-access-747zh\") pod \"cluster-monitoring-operator-64786d9cbb-24gv2\" (UID: \"179af188-f2c6-46da-9d1d-b15ce8fcb039\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-24gv2" Jul 06 22:09:34.969407 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.969404 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lck7p\" (UniqueName: \"kubernetes.io/projected/6fdb1134-ed1c-4505-a741-6e99dba1e2c8-kube-api-access-lck7p\") pod \"router-default-78989f99df-sxq46\" (UID: \"6fdb1134-ed1c-4505-a741-6e99dba1e2c8\") " pod="openshift-ingress/router-default-78989f99df-sxq46" Jul 06 22:09:34.969610 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.969437 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6fdb1134-ed1c-4505-a741-6e99dba1e2c8-metrics-certs\") pod \"router-default-78989f99df-sxq46\" (UID: \"6fdb1134-ed1c-4505-a741-6e99dba1e2c8\") " pod="openshift-ingress/router-default-78989f99df-sxq46" Jul 06 22:09:34.969610 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.969475 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-57mqp\" (UniqueName: \"kubernetes.io/projected/c39dd055-b80e-45d4-9882-ae689d849208-kube-api-access-57mqp\") pod \"volume-data-source-validator-5d456fdbcd-dftxz\" (UID: \"c39dd055-b80e-45d4-9882-ae689d849208\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-dftxz" Jul 06 22:09:34.969610 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.969491 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fe3d4288-3a9d-413c-be44-ac70bd65b45f-tmp\") pod \"insights-operator-5c847dcfc4-2pphn\" (UID: \"fe3d4288-3a9d-413c-be44-ac70bd65b45f\") " pod="openshift-insights/insights-operator-5c847dcfc4-2pphn" Jul 06 22:09:34.969610 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.969503 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-blhbk\" (UniqueName: \"kubernetes.io/projected/a4aac3c4-83c0-4140-8c7b-5ce240cf2f46-kube-api-access-blhbk\") pod \"cluster-samples-operator-56fcf86bf-dkbm6\" (UID: \"a4aac3c4-83c0-4140-8c7b-5ce240cf2f46\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-dkbm6" Jul 06 22:09:34.969610 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.969552 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/6fdb1134-ed1c-4505-a741-6e99dba1e2c8-default-certificate\") pod \"router-default-78989f99df-sxq46\" (UID: \"6fdb1134-ed1c-4505-a741-6e99dba1e2c8\") " pod="openshift-ingress/router-default-78989f99df-sxq46" Jul 06 22:09:34.969610 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.969584 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/fe3d4288-3a9d-413c-be44-ac70bd65b45f-snapshots\") pod \"insights-operator-5c847dcfc4-2pphn\" (UID: \"fe3d4288-3a9d-413c-be44-ac70bd65b45f\") " pod="openshift-insights/insights-operator-5c847dcfc4-2pphn" Jul 06 22:09:34.969610 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.969606 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fe3d4288-3a9d-413c-be44-ac70bd65b45f-serving-cert\") pod \"insights-operator-5c847dcfc4-2pphn\" (UID: \"fe3d4288-3a9d-413c-be44-ac70bd65b45f\") " pod="openshift-insights/insights-operator-5c847dcfc4-2pphn" Jul 06 22:09:34.969945 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.969654 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/86b07166-2f34-443b-9c61-86b7d950ad7b-trusted-ca\") pod \"console-operator-68b757fb67-rcn5q\" (UID: \"86b07166-2f34-443b-9c61-86b7d950ad7b\") " pod="openshift-console-operator/console-operator-68b757fb67-rcn5q" Jul 06 22:09:34.969945 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.969686 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d3642934-8f79-4aed-bdfe-5a6777a655aa-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-hfs2q\" (UID: \"d3642934-8f79-4aed-bdfe-5a6777a655aa\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-hfs2q" Jul 06 22:09:34.969945 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.969713 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/179af188-f2c6-46da-9d1d-b15ce8fcb039-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-24gv2\" (UID: \"179af188-f2c6-46da-9d1d-b15ce8fcb039\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-24gv2" Jul 06 22:09:34.969945 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.969743 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a4aac3c4-83c0-4140-8c7b-5ce240cf2f46-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-dkbm6\" (UID: \"a4aac3c4-83c0-4140-8c7b-5ce240cf2f46\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-dkbm6" Jul 06 22:09:34.969945 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.969779 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nbksk\" (UniqueName: \"kubernetes.io/projected/d3642934-8f79-4aed-bdfe-5a6777a655aa-kube-api-access-nbksk\") pod \"kube-storage-version-migrator-operator-846bf668f7-hfs2q\" (UID: \"d3642934-8f79-4aed-bdfe-5a6777a655aa\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-hfs2q" Jul 06 22:09:34.969945 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.969800 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/6fdb1134-ed1c-4505-a741-6e99dba1e2c8-stats-auth\") pod \"router-default-78989f99df-sxq46\" (UID: \"6fdb1134-ed1c-4505-a741-6e99dba1e2c8\") " pod="openshift-ingress/router-default-78989f99df-sxq46" Jul 06 22:09:34.970267 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.969952 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fe3d4288-3a9d-413c-be44-ac70bd65b45f-service-ca-bundle\") pod \"insights-operator-5c847dcfc4-2pphn\" (UID: \"fe3d4288-3a9d-413c-be44-ac70bd65b45f\") " pod="openshift-insights/insights-operator-5c847dcfc4-2pphn" Jul 06 22:09:34.970267 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.970099 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/86b07166-2f34-443b-9c61-86b7d950ad7b-config\") pod \"console-operator-68b757fb67-rcn5q\" (UID: \"86b07166-2f34-443b-9c61-86b7d950ad7b\") " pod="openshift-console-operator/console-operator-68b757fb67-rcn5q" Jul 06 22:09:34.970530 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.970507 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d3642934-8f79-4aed-bdfe-5a6777a655aa-config\") pod \"kube-storage-version-migrator-operator-846bf668f7-hfs2q\" (UID: \"d3642934-8f79-4aed-bdfe-5a6777a655aa\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-hfs2q" Jul 06 22:09:34.970859 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.970834 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/86b07166-2f34-443b-9c61-86b7d950ad7b-trusted-ca\") pod \"console-operator-68b757fb67-rcn5q\" (UID: \"86b07166-2f34-443b-9c61-86b7d950ad7b\") " pod="openshift-console-operator/console-operator-68b757fb67-rcn5q" Jul 06 22:09:34.971125 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.971104 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fe3d4288-3a9d-413c-be44-ac70bd65b45f-trusted-ca-bundle\") pod \"insights-operator-5c847dcfc4-2pphn\" (UID: \"fe3d4288-3a9d-413c-be44-ac70bd65b45f\") " pod="openshift-insights/insights-operator-5c847dcfc4-2pphn" Jul 06 22:09:34.971579 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.971560 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d3642934-8f79-4aed-bdfe-5a6777a655aa-serving-cert\") pod \"kube-storage-version-migrator-operator-846bf668f7-hfs2q\" (UID: \"d3642934-8f79-4aed-bdfe-5a6777a655aa\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-hfs2q" Jul 06 22:09:34.971682 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.971617 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/86b07166-2f34-443b-9c61-86b7d950ad7b-serving-cert\") pod \"console-operator-68b757fb67-rcn5q\" (UID: \"86b07166-2f34-443b-9c61-86b7d950ad7b\") " pod="openshift-console-operator/console-operator-68b757fb67-rcn5q" Jul 06 22:09:34.972880 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.972858 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fe3d4288-3a9d-413c-be44-ac70bd65b45f-serving-cert\") pod \"insights-operator-5c847dcfc4-2pphn\" (UID: \"fe3d4288-3a9d-413c-be44-ac70bd65b45f\") " pod="openshift-insights/insights-operator-5c847dcfc4-2pphn" Jul 06 22:09:34.981879 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.981400 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-65qc8\" (UniqueName: \"kubernetes.io/projected/86b07166-2f34-443b-9c61-86b7d950ad7b-kube-api-access-65qc8\") pod \"console-operator-68b757fb67-rcn5q\" (UID: \"86b07166-2f34-443b-9c61-86b7d950ad7b\") " pod="openshift-console-operator/console-operator-68b757fb67-rcn5q" Jul 06 22:09:34.983331 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.982966 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jqk49\" (UniqueName: \"kubernetes.io/projected/fe3d4288-3a9d-413c-be44-ac70bd65b45f-kube-api-access-jqk49\") pod \"insights-operator-5c847dcfc4-2pphn\" (UID: \"fe3d4288-3a9d-413c-be44-ac70bd65b45f\") " pod="openshift-insights/insights-operator-5c847dcfc4-2pphn" Jul 06 22:09:34.985718 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.985677 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nbksk\" (UniqueName: \"kubernetes.io/projected/d3642934-8f79-4aed-bdfe-5a6777a655aa-kube-api-access-nbksk\") pod \"kube-storage-version-migrator-operator-846bf668f7-hfs2q\" (UID: \"d3642934-8f79-4aed-bdfe-5a6777a655aa\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-hfs2q" Jul 06 22:09:34.988174 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:34.988136 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-57mqp\" (UniqueName: \"kubernetes.io/projected/c39dd055-b80e-45d4-9882-ae689d849208-kube-api-access-57mqp\") pod \"volume-data-source-validator-5d456fdbcd-dftxz\" (UID: \"c39dd055-b80e-45d4-9882-ae689d849208\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-dftxz" Jul 06 22:09:35.047646 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:35.047611 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-dj4ks" Jul 06 22:09:35.058585 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:35.058556 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:09:35.067445 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:35.067421 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-7hbvp" Jul 06 22:09:35.070777 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:35.070751 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-747zh\" (UniqueName: \"kubernetes.io/projected/179af188-f2c6-46da-9d1d-b15ce8fcb039-kube-api-access-747zh\") pod \"cluster-monitoring-operator-64786d9cbb-24gv2\" (UID: \"179af188-f2c6-46da-9d1d-b15ce8fcb039\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-24gv2" Jul 06 22:09:35.070887 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:35.070779 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lck7p\" (UniqueName: \"kubernetes.io/projected/6fdb1134-ed1c-4505-a741-6e99dba1e2c8-kube-api-access-lck7p\") pod \"router-default-78989f99df-sxq46\" (UID: \"6fdb1134-ed1c-4505-a741-6e99dba1e2c8\") " pod="openshift-ingress/router-default-78989f99df-sxq46" Jul 06 22:09:35.070887 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:35.070799 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6fdb1134-ed1c-4505-a741-6e99dba1e2c8-metrics-certs\") pod \"router-default-78989f99df-sxq46\" (UID: \"6fdb1134-ed1c-4505-a741-6e99dba1e2c8\") " pod="openshift-ingress/router-default-78989f99df-sxq46" Jul 06 22:09:35.070887 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:35.070818 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-blhbk\" (UniqueName: \"kubernetes.io/projected/a4aac3c4-83c0-4140-8c7b-5ce240cf2f46-kube-api-access-blhbk\") pod \"cluster-samples-operator-56fcf86bf-dkbm6\" (UID: \"a4aac3c4-83c0-4140-8c7b-5ce240cf2f46\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-dkbm6" Jul 06 22:09:35.070887 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:35.070837 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/6fdb1134-ed1c-4505-a741-6e99dba1e2c8-default-certificate\") pod \"router-default-78989f99df-sxq46\" (UID: \"6fdb1134-ed1c-4505-a741-6e99dba1e2c8\") " pod="openshift-ingress/router-default-78989f99df-sxq46" Jul 06 22:09:35.070887 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:35.070884 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/179af188-f2c6-46da-9d1d-b15ce8fcb039-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-24gv2\" (UID: \"179af188-f2c6-46da-9d1d-b15ce8fcb039\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-24gv2" Jul 06 22:09:35.071164 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:35.070909 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a4aac3c4-83c0-4140-8c7b-5ce240cf2f46-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-dkbm6\" (UID: \"a4aac3c4-83c0-4140-8c7b-5ce240cf2f46\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-dkbm6" Jul 06 22:09:35.071164 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:35.070926 2581 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 06 22:09:35.071164 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:35.070945 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/6fdb1134-ed1c-4505-a741-6e99dba1e2c8-stats-auth\") pod \"router-default-78989f99df-sxq46\" (UID: \"6fdb1134-ed1c-4505-a741-6e99dba1e2c8\") " pod="openshift-ingress/router-default-78989f99df-sxq46" Jul 06 22:09:35.071164 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:35.070985 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6fdb1134-ed1c-4505-a741-6e99dba1e2c8-metrics-certs podName:6fdb1134-ed1c-4505-a741-6e99dba1e2c8 nodeName:}" failed. No retries permitted until 2026-07-06 22:09:35.57096773 +0000 UTC m=+55.129763032 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6fdb1134-ed1c-4505-a741-6e99dba1e2c8-metrics-certs") pod "router-default-78989f99df-sxq46" (UID: "6fdb1134-ed1c-4505-a741-6e99dba1e2c8") : secret "router-metrics-certs-default" not found Jul 06 22:09:35.071164 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:35.071008 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6fdb1134-ed1c-4505-a741-6e99dba1e2c8-service-ca-bundle\") pod \"router-default-78989f99df-sxq46\" (UID: \"6fdb1134-ed1c-4505-a741-6e99dba1e2c8\") " pod="openshift-ingress/router-default-78989f99df-sxq46" Jul 06 22:09:35.071164 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:35.071143 2581 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 06 22:09:35.071450 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:35.071147 2581 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 06 22:09:35.071450 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:35.071195 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a4aac3c4-83c0-4140-8c7b-5ce240cf2f46-samples-operator-tls podName:a4aac3c4-83c0-4140-8c7b-5ce240cf2f46 nodeName:}" failed. No retries permitted until 2026-07-06 22:09:35.571178235 +0000 UTC m=+55.129973535 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/a4aac3c4-83c0-4140-8c7b-5ce240cf2f46-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-dkbm6" (UID: "a4aac3c4-83c0-4140-8c7b-5ce240cf2f46") : secret "samples-operator-tls" not found Jul 06 22:09:35.071450 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:35.071283 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6fdb1134-ed1c-4505-a741-6e99dba1e2c8-service-ca-bundle podName:6fdb1134-ed1c-4505-a741-6e99dba1e2c8 nodeName:}" failed. No retries permitted until 2026-07-06 22:09:35.571264646 +0000 UTC m=+55.130059961 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/6fdb1134-ed1c-4505-a741-6e99dba1e2c8-service-ca-bundle") pod "router-default-78989f99df-sxq46" (UID: "6fdb1134-ed1c-4505-a741-6e99dba1e2c8") : configmap references non-existent config key: service-ca.crt Jul 06 22:09:35.071450 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:35.071304 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/179af188-f2c6-46da-9d1d-b15ce8fcb039-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-24gv2\" (UID: \"179af188-f2c6-46da-9d1d-b15ce8fcb039\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-24gv2" Jul 06 22:09:35.071450 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:35.071355 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/179af188-f2c6-46da-9d1d-b15ce8fcb039-cluster-monitoring-operator-tls podName:179af188-f2c6-46da-9d1d-b15ce8fcb039 nodeName:}" failed. No retries permitted until 2026-07-06 22:09:35.57134407 +0000 UTC m=+55.130139370 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/179af188-f2c6-46da-9d1d-b15ce8fcb039-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-24gv2" (UID: "179af188-f2c6-46da-9d1d-b15ce8fcb039") : secret "cluster-monitoring-operator-tls" not found Jul 06 22:09:35.071922 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:35.071859 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/179af188-f2c6-46da-9d1d-b15ce8fcb039-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-24gv2\" (UID: \"179af188-f2c6-46da-9d1d-b15ce8fcb039\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-24gv2" Jul 06 22:09:35.073272 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:35.073252 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/6fdb1134-ed1c-4505-a741-6e99dba1e2c8-default-certificate\") pod \"router-default-78989f99df-sxq46\" (UID: \"6fdb1134-ed1c-4505-a741-6e99dba1e2c8\") " pod="openshift-ingress/router-default-78989f99df-sxq46" Jul 06 22:09:35.073579 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:35.073563 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/6fdb1134-ed1c-4505-a741-6e99dba1e2c8-stats-auth\") pod \"router-default-78989f99df-sxq46\" (UID: \"6fdb1134-ed1c-4505-a741-6e99dba1e2c8\") " pod="openshift-ingress/router-default-78989f99df-sxq46" Jul 06 22:09:35.080985 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:35.080953 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-747zh\" (UniqueName: \"kubernetes.io/projected/179af188-f2c6-46da-9d1d-b15ce8fcb039-kube-api-access-747zh\") pod \"cluster-monitoring-operator-64786d9cbb-24gv2\" (UID: \"179af188-f2c6-46da-9d1d-b15ce8fcb039\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-24gv2" Jul 06 22:09:35.081646 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:35.081623 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lck7p\" (UniqueName: \"kubernetes.io/projected/6fdb1134-ed1c-4505-a741-6e99dba1e2c8-kube-api-access-lck7p\") pod \"router-default-78989f99df-sxq46\" (UID: \"6fdb1134-ed1c-4505-a741-6e99dba1e2c8\") " pod="openshift-ingress/router-default-78989f99df-sxq46" Jul 06 22:09:35.082095 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:35.082058 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-blhbk\" (UniqueName: \"kubernetes.io/projected/a4aac3c4-83c0-4140-8c7b-5ce240cf2f46-kube-api-access-blhbk\") pod \"cluster-samples-operator-56fcf86bf-dkbm6\" (UID: \"a4aac3c4-83c0-4140-8c7b-5ce240cf2f46\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-dkbm6" Jul 06 22:09:35.087472 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:35.087455 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-dftxz" Jul 06 22:09:35.103732 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:35.103707 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5c847dcfc4-2pphn" Jul 06 22:09:35.118324 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:35.118241 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-68b757fb67-rcn5q" Jul 06 22:09:35.143572 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:35.143462 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:09:35.165089 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:35.164812 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-hfs2q" Jul 06 22:09:35.217633 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:35.217262 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-dj4ks"] Jul 06 22:09:35.222515 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:35.222440 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:09:35.259388 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:35.259236 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-7hbvp"] Jul 06 22:09:35.263791 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:09:35.263746 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc407c330_5340_4bc6_bf8a_e4dc003ea079.slice/crio-0f6d89d77ee22af1d49d1a339107c49260e1e396e688ce4802518ed675b6a224 WatchSource:0}: Error finding container 0f6d89d77ee22af1d49d1a339107c49260e1e396e688ce4802518ed675b6a224: Status 404 returned error can't find the container with id 0f6d89d77ee22af1d49d1a339107c49260e1e396e688ce4802518ed675b6a224 Jul 06 22:09:35.304274 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:35.304247 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-dftxz"] Jul 06 22:09:35.307781 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:09:35.307743 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc39dd055_b80e_45d4_9882_ae689d849208.slice/crio-e0e519d11095ec658f01379569def3b0ec2a1fb2aa564e36e476502ec7e306d5 WatchSource:0}: Error finding container e0e519d11095ec658f01379569def3b0ec2a1fb2aa564e36e476502ec7e306d5: Status 404 returned error can't find the container with id e0e519d11095ec658f01379569def3b0ec2a1fb2aa564e36e476502ec7e306d5 Jul 06 22:09:35.331898 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:35.331867 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5c847dcfc4-2pphn"] Jul 06 22:09:35.335013 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:09:35.334972 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe3d4288_3a9d_413c_be44_ac70bd65b45f.slice/crio-84b67b8f1097cae4f978e2cf6e7c188ac864d1c566c7986f99d7713d5a23b8cb WatchSource:0}: Error finding container 84b67b8f1097cae4f978e2cf6e7c188ac864d1c566c7986f99d7713d5a23b8cb: Status 404 returned error can't find the container with id 84b67b8f1097cae4f978e2cf6e7c188ac864d1c566c7986f99d7713d5a23b8cb Jul 06 22:09:35.347370 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:35.347336 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-68b757fb67-rcn5q"] Jul 06 22:09:35.349632 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:09:35.349609 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86b07166_2f34_443b_9c61_86b7d950ad7b.slice/crio-262b1a52a2fbcbd48a6b21aceb7d4c78b43c145fdb364bc5dd9e95a11f56084d WatchSource:0}: Error finding container 262b1a52a2fbcbd48a6b21aceb7d4c78b43c145fdb364bc5dd9e95a11f56084d: Status 404 returned error can't find the container with id 262b1a52a2fbcbd48a6b21aceb7d4c78b43c145fdb364bc5dd9e95a11f56084d Jul 06 22:09:35.365007 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:35.364978 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-hfs2q"] Jul 06 22:09:35.368706 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:09:35.368678 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3642934_8f79_4aed_bdfe_5a6777a655aa.slice/crio-4a63423d68e5e005aba5659b5fda3dbb660878e364e8a6e3b7ef66b8733faf97 WatchSource:0}: Error finding container 4a63423d68e5e005aba5659b5fda3dbb660878e364e8a6e3b7ef66b8733faf97: Status 404 returned error can't find the container with id 4a63423d68e5e005aba5659b5fda3dbb660878e364e8a6e3b7ef66b8733faf97 Jul 06 22:09:35.444599 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:35.444567 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-2pphn" event={"ID":"fe3d4288-3a9d-413c-be44-ac70bd65b45f","Type":"ContainerStarted","Data":"84b67b8f1097cae4f978e2cf6e7c188ac864d1c566c7986f99d7713d5a23b8cb"} Jul 06 22:09:35.445691 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:35.445655 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-7hbvp" event={"ID":"c407c330-5340-4bc6-bf8a-e4dc003ea079","Type":"ContainerStarted","Data":"0f6d89d77ee22af1d49d1a339107c49260e1e396e688ce4802518ed675b6a224"} Jul 06 22:09:35.446744 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:35.446719 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-hfs2q" event={"ID":"d3642934-8f79-4aed-bdfe-5a6777a655aa","Type":"ContainerStarted","Data":"4a63423d68e5e005aba5659b5fda3dbb660878e364e8a6e3b7ef66b8733faf97"} Jul 06 22:09:35.447793 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:35.447760 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-dftxz" event={"ID":"c39dd055-b80e-45d4-9882-ae689d849208","Type":"ContainerStarted","Data":"e0e519d11095ec658f01379569def3b0ec2a1fb2aa564e36e476502ec7e306d5"} Jul 06 22:09:35.448813 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:35.448792 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-dj4ks" event={"ID":"01137b22-9e43-47ba-aee5-e4c51fd35e4d","Type":"ContainerStarted","Data":"ade6e33e51f15399fd831313fcf632e9455993334e804b4f9525243ff0d061eb"} Jul 06 22:09:35.449762 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:35.449742 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-rcn5q" event={"ID":"86b07166-2f34-443b-9c61-86b7d950ad7b","Type":"ContainerStarted","Data":"262b1a52a2fbcbd48a6b21aceb7d4c78b43c145fdb364bc5dd9e95a11f56084d"} Jul 06 22:09:35.576125 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:35.576024 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6fdb1134-ed1c-4505-a741-6e99dba1e2c8-metrics-certs\") pod \"router-default-78989f99df-sxq46\" (UID: \"6fdb1134-ed1c-4505-a741-6e99dba1e2c8\") " pod="openshift-ingress/router-default-78989f99df-sxq46" Jul 06 22:09:35.576125 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:35.576103 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/179af188-f2c6-46da-9d1d-b15ce8fcb039-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-24gv2\" (UID: \"179af188-f2c6-46da-9d1d-b15ce8fcb039\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-24gv2" Jul 06 22:09:35.576125 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:35.576121 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a4aac3c4-83c0-4140-8c7b-5ce240cf2f46-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-dkbm6\" (UID: \"a4aac3c4-83c0-4140-8c7b-5ce240cf2f46\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-dkbm6" Jul 06 22:09:35.576351 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:35.576150 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6fdb1134-ed1c-4505-a741-6e99dba1e2c8-service-ca-bundle\") pod \"router-default-78989f99df-sxq46\" (UID: \"6fdb1134-ed1c-4505-a741-6e99dba1e2c8\") " pod="openshift-ingress/router-default-78989f99df-sxq46" Jul 06 22:09:35.576351 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:35.576184 2581 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 06 22:09:35.576351 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:35.576242 2581 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 06 22:09:35.576351 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:35.576267 2581 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 06 22:09:35.576351 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:35.576246 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6fdb1134-ed1c-4505-a741-6e99dba1e2c8-metrics-certs podName:6fdb1134-ed1c-4505-a741-6e99dba1e2c8 nodeName:}" failed. No retries permitted until 2026-07-06 22:09:36.576230848 +0000 UTC m=+56.135026147 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6fdb1134-ed1c-4505-a741-6e99dba1e2c8-metrics-certs") pod "router-default-78989f99df-sxq46" (UID: "6fdb1134-ed1c-4505-a741-6e99dba1e2c8") : secret "router-metrics-certs-default" not found Jul 06 22:09:35.576351 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:35.576308 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/179af188-f2c6-46da-9d1d-b15ce8fcb039-cluster-monitoring-operator-tls podName:179af188-f2c6-46da-9d1d-b15ce8fcb039 nodeName:}" failed. No retries permitted until 2026-07-06 22:09:36.576293075 +0000 UTC m=+56.135088392 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/179af188-f2c6-46da-9d1d-b15ce8fcb039-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-24gv2" (UID: "179af188-f2c6-46da-9d1d-b15ce8fcb039") : secret "cluster-monitoring-operator-tls" not found Jul 06 22:09:35.576351 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:35.576328 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6fdb1134-ed1c-4505-a741-6e99dba1e2c8-service-ca-bundle podName:6fdb1134-ed1c-4505-a741-6e99dba1e2c8 nodeName:}" failed. No retries permitted until 2026-07-06 22:09:36.576319983 +0000 UTC m=+56.135115285 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/6fdb1134-ed1c-4505-a741-6e99dba1e2c8-service-ca-bundle") pod "router-default-78989f99df-sxq46" (UID: "6fdb1134-ed1c-4505-a741-6e99dba1e2c8") : configmap references non-existent config key: service-ca.crt Jul 06 22:09:35.576351 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:35.576349 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a4aac3c4-83c0-4140-8c7b-5ce240cf2f46-samples-operator-tls podName:a4aac3c4-83c0-4140-8c7b-5ce240cf2f46 nodeName:}" failed. No retries permitted until 2026-07-06 22:09:36.576339609 +0000 UTC m=+56.135134913 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/a4aac3c4-83c0-4140-8c7b-5ce240cf2f46-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-dkbm6" (UID: "a4aac3c4-83c0-4140-8c7b-5ce240cf2f46") : secret "samples-operator-tls" not found Jul 06 22:09:36.588552 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:36.587540 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6fdb1134-ed1c-4505-a741-6e99dba1e2c8-metrics-certs\") pod \"router-default-78989f99df-sxq46\" (UID: \"6fdb1134-ed1c-4505-a741-6e99dba1e2c8\") " pod="openshift-ingress/router-default-78989f99df-sxq46" Jul 06 22:09:36.588552 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:36.587626 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/179af188-f2c6-46da-9d1d-b15ce8fcb039-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-24gv2\" (UID: \"179af188-f2c6-46da-9d1d-b15ce8fcb039\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-24gv2" Jul 06 22:09:36.588552 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:36.587658 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a4aac3c4-83c0-4140-8c7b-5ce240cf2f46-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-dkbm6\" (UID: \"a4aac3c4-83c0-4140-8c7b-5ce240cf2f46\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-dkbm6" Jul 06 22:09:36.588552 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:36.587705 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6fdb1134-ed1c-4505-a741-6e99dba1e2c8-service-ca-bundle\") pod \"router-default-78989f99df-sxq46\" (UID: \"6fdb1134-ed1c-4505-a741-6e99dba1e2c8\") " pod="openshift-ingress/router-default-78989f99df-sxq46" Jul 06 22:09:36.588552 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:36.587843 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6fdb1134-ed1c-4505-a741-6e99dba1e2c8-service-ca-bundle podName:6fdb1134-ed1c-4505-a741-6e99dba1e2c8 nodeName:}" failed. No retries permitted until 2026-07-06 22:09:38.587823774 +0000 UTC m=+58.146619081 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/6fdb1134-ed1c-4505-a741-6e99dba1e2c8-service-ca-bundle") pod "router-default-78989f99df-sxq46" (UID: "6fdb1134-ed1c-4505-a741-6e99dba1e2c8") : configmap references non-existent config key: service-ca.crt Jul 06 22:09:36.588552 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:36.587930 2581 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 06 22:09:36.588552 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:36.587968 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6fdb1134-ed1c-4505-a741-6e99dba1e2c8-metrics-certs podName:6fdb1134-ed1c-4505-a741-6e99dba1e2c8 nodeName:}" failed. No retries permitted until 2026-07-06 22:09:38.587955112 +0000 UTC m=+58.146750415 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6fdb1134-ed1c-4505-a741-6e99dba1e2c8-metrics-certs") pod "router-default-78989f99df-sxq46" (UID: "6fdb1134-ed1c-4505-a741-6e99dba1e2c8") : secret "router-metrics-certs-default" not found Jul 06 22:09:36.588552 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:36.588023 2581 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 06 22:09:36.588552 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:36.588053 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/179af188-f2c6-46da-9d1d-b15ce8fcb039-cluster-monitoring-operator-tls podName:179af188-f2c6-46da-9d1d-b15ce8fcb039 nodeName:}" failed. No retries permitted until 2026-07-06 22:09:38.588042203 +0000 UTC m=+58.146837601 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/179af188-f2c6-46da-9d1d-b15ce8fcb039-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-24gv2" (UID: "179af188-f2c6-46da-9d1d-b15ce8fcb039") : secret "cluster-monitoring-operator-tls" not found Jul 06 22:09:36.588552 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:36.588147 2581 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 06 22:09:36.588552 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:36.588181 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a4aac3c4-83c0-4140-8c7b-5ce240cf2f46-samples-operator-tls podName:a4aac3c4-83c0-4140-8c7b-5ce240cf2f46 nodeName:}" failed. No retries permitted until 2026-07-06 22:09:38.588169868 +0000 UTC m=+58.146965170 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/a4aac3c4-83c0-4140-8c7b-5ce240cf2f46-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-dkbm6" (UID: "a4aac3c4-83c0-4140-8c7b-5ce240cf2f46") : secret "samples-operator-tls" not found Jul 06 22:09:38.607590 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:38.607542 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/179af188-f2c6-46da-9d1d-b15ce8fcb039-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-24gv2\" (UID: \"179af188-f2c6-46da-9d1d-b15ce8fcb039\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-24gv2" Jul 06 22:09:38.607590 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:38.607590 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a4aac3c4-83c0-4140-8c7b-5ce240cf2f46-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-dkbm6\" (UID: \"a4aac3c4-83c0-4140-8c7b-5ce240cf2f46\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-dkbm6" Jul 06 22:09:38.608087 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:38.607698 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6fdb1134-ed1c-4505-a741-6e99dba1e2c8-service-ca-bundle\") pod \"router-default-78989f99df-sxq46\" (UID: \"6fdb1134-ed1c-4505-a741-6e99dba1e2c8\") " pod="openshift-ingress/router-default-78989f99df-sxq46" Jul 06 22:09:38.608087 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:38.607743 2581 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 06 22:09:38.608087 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:38.607767 2581 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 06 22:09:38.608087 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:38.607827 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/179af188-f2c6-46da-9d1d-b15ce8fcb039-cluster-monitoring-operator-tls podName:179af188-f2c6-46da-9d1d-b15ce8fcb039 nodeName:}" failed. No retries permitted until 2026-07-06 22:09:42.607800491 +0000 UTC m=+62.166595805 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/179af188-f2c6-46da-9d1d-b15ce8fcb039-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-24gv2" (UID: "179af188-f2c6-46da-9d1d-b15ce8fcb039") : secret "cluster-monitoring-operator-tls" not found Jul 06 22:09:38.608087 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:38.607865 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6fdb1134-ed1c-4505-a741-6e99dba1e2c8-service-ca-bundle podName:6fdb1134-ed1c-4505-a741-6e99dba1e2c8 nodeName:}" failed. No retries permitted until 2026-07-06 22:09:42.607855991 +0000 UTC m=+62.166651290 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/6fdb1134-ed1c-4505-a741-6e99dba1e2c8-service-ca-bundle") pod "router-default-78989f99df-sxq46" (UID: "6fdb1134-ed1c-4505-a741-6e99dba1e2c8") : configmap references non-existent config key: service-ca.crt Jul 06 22:09:38.608087 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:38.607886 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a4aac3c4-83c0-4140-8c7b-5ce240cf2f46-samples-operator-tls podName:a4aac3c4-83c0-4140-8c7b-5ce240cf2f46 nodeName:}" failed. No retries permitted until 2026-07-06 22:09:42.607875949 +0000 UTC m=+62.166671248 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/a4aac3c4-83c0-4140-8c7b-5ce240cf2f46-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-dkbm6" (UID: "a4aac3c4-83c0-4140-8c7b-5ce240cf2f46") : secret "samples-operator-tls" not found Jul 06 22:09:38.608087 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:38.607962 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6fdb1134-ed1c-4505-a741-6e99dba1e2c8-metrics-certs\") pod \"router-default-78989f99df-sxq46\" (UID: \"6fdb1134-ed1c-4505-a741-6e99dba1e2c8\") " pod="openshift-ingress/router-default-78989f99df-sxq46" Jul 06 22:09:38.608087 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:38.608082 2581 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 06 22:09:38.608383 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:38.608129 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6fdb1134-ed1c-4505-a741-6e99dba1e2c8-metrics-certs podName:6fdb1134-ed1c-4505-a741-6e99dba1e2c8 nodeName:}" failed. No retries permitted until 2026-07-06 22:09:42.60811538 +0000 UTC m=+62.166910679 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6fdb1134-ed1c-4505-a741-6e99dba1e2c8-metrics-certs") pod "router-default-78989f99df-sxq46" (UID: "6fdb1134-ed1c-4505-a741-6e99dba1e2c8") : secret "router-metrics-certs-default" not found Jul 06 22:09:39.811540 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:39.811450 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:09:39.932879 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:39.932805 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:09:40.271874 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:40.271831 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:09:40.372868 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:40.372833 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:09:41.058014 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:41.057972 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:09:41.123704 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:41.123664 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:09:41.473467 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:41.473247 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-dftxz" event={"ID":"c39dd055-b80e-45d4-9882-ae689d849208","Type":"ContainerStarted","Data":"86a4cab0bc0f4213ed3febba1155c68d495cee9bf94b31bf389f22f185295946"} Jul 06 22:09:41.475043 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:41.474995 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-dj4ks" event={"ID":"01137b22-9e43-47ba-aee5-e4c51fd35e4d","Type":"ContainerStarted","Data":"a9cffbd956375580bb03a7f9172ebfcfb59dc4e10d62486d7d0ec361e58f3212"} Jul 06 22:09:41.476753 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:41.476726 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-rcn5q_86b07166-2f34-443b-9c61-86b7d950ad7b/console-operator/0.log" Jul 06 22:09:41.476864 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:41.476766 2581 generic.go:356] "Generic (PLEG): container finished" podID="86b07166-2f34-443b-9c61-86b7d950ad7b" containerID="675e58cfc4f367e3fe3f7da478562fb84cb69e5712a9ab8a69f79b8e52a4a343" exitCode=255 Jul 06 22:09:41.476921 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:41.476860 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-rcn5q" event={"ID":"86b07166-2f34-443b-9c61-86b7d950ad7b","Type":"ContainerDied","Data":"675e58cfc4f367e3fe3f7da478562fb84cb69e5712a9ab8a69f79b8e52a4a343"} Jul 06 22:09:41.477106 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:41.477093 2581 scope.go:117] "RemoveContainer" containerID="675e58cfc4f367e3fe3f7da478562fb84cb69e5712a9ab8a69f79b8e52a4a343" Jul 06 22:09:41.478962 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:41.478916 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-2pphn" event={"ID":"fe3d4288-3a9d-413c-be44-ac70bd65b45f","Type":"ContainerStarted","Data":"740e1a39643fd71e51690103fd9decb887bde25f635e3db81f2a702487cfc2ac"} Jul 06 22:09:41.481220 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:41.481041 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-7hbvp" event={"ID":"c407c330-5340-4bc6-bf8a-e4dc003ea079","Type":"ContainerStarted","Data":"b49ef70b93f5ac5387de174a8c01f2f9cb632782fb7c9c2393f47329d1c8cb05"} Jul 06 22:09:41.483133 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:41.482959 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-hfs2q" event={"ID":"d3642934-8f79-4aed-bdfe-5a6777a655aa","Type":"ContainerStarted","Data":"9fe650b74821066a2251e6b72296ebe37b92ebffadb728707740a465cf81d3a5"} Jul 06 22:09:41.489572 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:41.489448 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-5d456fdbcd-dftxz" podStartSLOduration=2.290980448 podStartE2EDuration="7.489437443s" totalImagesPullingTime="5.198456995s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:09:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:09:41.489232822 +0000 UTC m=+61.048028144" watchObservedRunningTime="2026-07-06 22:09:41.489437443 +0000 UTC m=+61.048232763" Jul 06 22:09:41.508377 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:41.508171 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-7hbvp" podStartSLOduration=2.203757923 podStartE2EDuration="7.508150907s" totalImagesPullingTime="5.304392984s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:09:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:09:41.506809987 +0000 UTC m=+61.065605310" watchObservedRunningTime="2026-07-06 22:09:41.508150907 +0000 UTC m=+61.066946231" Jul 06 22:09:41.525180 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:41.525122 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5c847dcfc4-2pphn" podStartSLOduration=2.342337296 podStartE2EDuration="7.525105068s" totalImagesPullingTime="5.182767772s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:09:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:09:41.52494243 +0000 UTC m=+61.083737753" watchObservedRunningTime="2026-07-06 22:09:41.525105068 +0000 UTC m=+61.083900369" Jul 06 22:09:41.542729 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:41.542675 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-dj4ks" podStartSLOduration=2.253398589 podStartE2EDuration="7.542655586s" totalImagesPullingTime="5.289256997s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:09:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:09:41.540879208 +0000 UTC m=+61.099674525" watchObservedRunningTime="2026-07-06 22:09:41.542655586 +0000 UTC m=+61.101450910" Jul 06 22:09:41.583438 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:41.583406 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:09:41.654502 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:41.654473 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:09:42.485811 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:42.485744 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-hfs2q" podStartSLOduration=3.337785423 podStartE2EDuration="8.485722985s" totalImagesPullingTime="5.147937562s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:09:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:09:41.58075864 +0000 UTC m=+61.139553962" watchObservedRunningTime="2026-07-06 22:09:42.485722985 +0000 UTC m=+62.044518307" Jul 06 22:09:42.486907 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:42.486884 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-68gjp"] Jul 06 22:09:42.489862 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:42.489838 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-rcn5q_86b07166-2f34-443b-9c61-86b7d950ad7b/console-operator/1.log" Jul 06 22:09:42.490383 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:42.490366 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-rcn5q_86b07166-2f34-443b-9c61-86b7d950ad7b/console-operator/0.log" Jul 06 22:09:42.490448 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:42.490407 2581 generic.go:356] "Generic (PLEG): container finished" podID="86b07166-2f34-443b-9c61-86b7d950ad7b" containerID="11d06e0eaa8ccb153c499b1e7fe6f156124abdccb2a253d555b84244c1fc5e8f" exitCode=255 Jul 06 22:09:42.490500 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:42.490458 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-rcn5q" event={"ID":"86b07166-2f34-443b-9c61-86b7d950ad7b","Type":"ContainerDied","Data":"11d06e0eaa8ccb153c499b1e7fe6f156124abdccb2a253d555b84244c1fc5e8f"} Jul 06 22:09:42.490551 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:42.490498 2581 scope.go:117] "RemoveContainer" containerID="675e58cfc4f367e3fe3f7da478562fb84cb69e5712a9ab8a69f79b8e52a4a343" Jul 06 22:09:42.491151 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:42.491130 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-68gjp" Jul 06 22:09:42.491321 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:42.491305 2581 scope.go:117] "RemoveContainer" containerID="11d06e0eaa8ccb153c499b1e7fe6f156124abdccb2a253d555b84244c1fc5e8f" Jul 06 22:09:42.491521 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:42.491450 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-rcn5q_openshift-console-operator(86b07166-2f34-443b-9c61-86b7d950ad7b)\"" pod="openshift-console-operator/console-operator-68b757fb67-rcn5q" podUID="86b07166-2f34-443b-9c61-86b7d950ad7b" Jul 06 22:09:42.496282 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:42.496262 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 06 22:09:42.496573 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:42.496555 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-66l9r\"" Jul 06 22:09:42.496996 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:42.496944 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 06 22:09:42.510553 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:42.510532 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-68gjp"] Jul 06 22:09:42.541563 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:42.541529 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:09:42.547462 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:42.547438 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t9wcj\" (UniqueName: \"kubernetes.io/projected/bd42289a-105e-46c7-ad15-a6341ce43f15-kube-api-access-t9wcj\") pod \"insights-runtime-extractor-68gjp\" (UID: \"bd42289a-105e-46c7-ad15-a6341ce43f15\") " pod="openshift-insights/insights-runtime-extractor-68gjp" Jul 06 22:09:42.547577 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:42.547519 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bd42289a-105e-46c7-ad15-a6341ce43f15-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-68gjp\" (UID: \"bd42289a-105e-46c7-ad15-a6341ce43f15\") " pod="openshift-insights/insights-runtime-extractor-68gjp" Jul 06 22:09:42.548118 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:42.548093 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/bd42289a-105e-46c7-ad15-a6341ce43f15-crio-socket\") pod \"insights-runtime-extractor-68gjp\" (UID: \"bd42289a-105e-46c7-ad15-a6341ce43f15\") " pod="openshift-insights/insights-runtime-extractor-68gjp" Jul 06 22:09:42.548299 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:42.548279 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bd42289a-105e-46c7-ad15-a6341ce43f15-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-68gjp\" (UID: \"bd42289a-105e-46c7-ad15-a6341ce43f15\") " pod="openshift-insights/insights-runtime-extractor-68gjp" Jul 06 22:09:42.548384 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:42.548367 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/bd42289a-105e-46c7-ad15-a6341ce43f15-data-volume\") pod \"insights-runtime-extractor-68gjp\" (UID: \"bd42289a-105e-46c7-ad15-a6341ce43f15\") " pod="openshift-insights/insights-runtime-extractor-68gjp" Jul 06 22:09:42.601656 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:42.601622 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:09:42.649007 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:42.648973 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/bd42289a-105e-46c7-ad15-a6341ce43f15-crio-socket\") pod \"insights-runtime-extractor-68gjp\" (UID: \"bd42289a-105e-46c7-ad15-a6341ce43f15\") " pod="openshift-insights/insights-runtime-extractor-68gjp" Jul 06 22:09:42.649007 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:42.649012 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bd42289a-105e-46c7-ad15-a6341ce43f15-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-68gjp\" (UID: \"bd42289a-105e-46c7-ad15-a6341ce43f15\") " pod="openshift-insights/insights-runtime-extractor-68gjp" Jul 06 22:09:42.649237 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:42.649036 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6fdb1134-ed1c-4505-a741-6e99dba1e2c8-service-ca-bundle\") pod \"router-default-78989f99df-sxq46\" (UID: \"6fdb1134-ed1c-4505-a741-6e99dba1e2c8\") " pod="openshift-ingress/router-default-78989f99df-sxq46" Jul 06 22:09:42.649237 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:42.649130 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/bd42289a-105e-46c7-ad15-a6341ce43f15-crio-socket\") pod \"insights-runtime-extractor-68gjp\" (UID: \"bd42289a-105e-46c7-ad15-a6341ce43f15\") " pod="openshift-insights/insights-runtime-extractor-68gjp" Jul 06 22:09:42.649237 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:42.649154 2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 22:09:42.649237 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:42.649170 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6fdb1134-ed1c-4505-a741-6e99dba1e2c8-service-ca-bundle podName:6fdb1134-ed1c-4505-a741-6e99dba1e2c8 nodeName:}" failed. No retries permitted until 2026-07-06 22:09:50.649156694 +0000 UTC m=+70.207951993 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/6fdb1134-ed1c-4505-a741-6e99dba1e2c8-service-ca-bundle") pod "router-default-78989f99df-sxq46" (UID: "6fdb1134-ed1c-4505-a741-6e99dba1e2c8") : configmap references non-existent config key: service-ca.crt Jul 06 22:09:42.649237 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:42.649194 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/bd42289a-105e-46c7-ad15-a6341ce43f15-data-volume\") pod \"insights-runtime-extractor-68gjp\" (UID: \"bd42289a-105e-46c7-ad15-a6341ce43f15\") " pod="openshift-insights/insights-runtime-extractor-68gjp" Jul 06 22:09:42.649237 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:42.649205 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bd42289a-105e-46c7-ad15-a6341ce43f15-insights-runtime-extractor-tls podName:bd42289a-105e-46c7-ad15-a6341ce43f15 nodeName:}" failed. No retries permitted until 2026-07-06 22:09:43.149191272 +0000 UTC m=+62.707986571 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/bd42289a-105e-46c7-ad15-a6341ce43f15-insights-runtime-extractor-tls") pod "insights-runtime-extractor-68gjp" (UID: "bd42289a-105e-46c7-ad15-a6341ce43f15") : secret "insights-runtime-extractor-tls" not found Jul 06 22:09:42.649471 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:42.649268 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t9wcj\" (UniqueName: \"kubernetes.io/projected/bd42289a-105e-46c7-ad15-a6341ce43f15-kube-api-access-t9wcj\") pod \"insights-runtime-extractor-68gjp\" (UID: \"bd42289a-105e-46c7-ad15-a6341ce43f15\") " pod="openshift-insights/insights-runtime-extractor-68gjp" Jul 06 22:09:42.649471 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:42.649311 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bd42289a-105e-46c7-ad15-a6341ce43f15-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-68gjp\" (UID: \"bd42289a-105e-46c7-ad15-a6341ce43f15\") " pod="openshift-insights/insights-runtime-extractor-68gjp" Jul 06 22:09:42.649471 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:42.649353 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6fdb1134-ed1c-4505-a741-6e99dba1e2c8-metrics-certs\") pod \"router-default-78989f99df-sxq46\" (UID: \"6fdb1134-ed1c-4505-a741-6e99dba1e2c8\") " pod="openshift-ingress/router-default-78989f99df-sxq46" Jul 06 22:09:42.649471 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:42.649456 2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 22:09:42.649669 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:42.649486 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/bd42289a-105e-46c7-ad15-a6341ce43f15-data-volume\") pod \"insights-runtime-extractor-68gjp\" (UID: \"bd42289a-105e-46c7-ad15-a6341ce43f15\") " pod="openshift-insights/insights-runtime-extractor-68gjp" Jul 06 22:09:42.649669 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:42.649504 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/179af188-f2c6-46da-9d1d-b15ce8fcb039-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-24gv2\" (UID: \"179af188-f2c6-46da-9d1d-b15ce8fcb039\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-24gv2" Jul 06 22:09:42.649669 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:42.649511 2581 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 06 22:09:42.649669 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:42.649521 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bd42289a-105e-46c7-ad15-a6341ce43f15-kube-rbac-proxy-cm podName:bd42289a-105e-46c7-ad15-a6341ce43f15 nodeName:}" failed. No retries permitted until 2026-07-06 22:09:43.149506291 +0000 UTC m=+62.708301593 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/bd42289a-105e-46c7-ad15-a6341ce43f15-kube-rbac-proxy-cm") pod "insights-runtime-extractor-68gjp" (UID: "bd42289a-105e-46c7-ad15-a6341ce43f15") : configmap "kube-rbac-proxy" not found Jul 06 22:09:42.649669 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:42.649565 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a4aac3c4-83c0-4140-8c7b-5ce240cf2f46-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-dkbm6\" (UID: \"a4aac3c4-83c0-4140-8c7b-5ce240cf2f46\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-dkbm6" Jul 06 22:09:42.649669 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:42.649575 2581 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 06 22:09:42.649669 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:42.649577 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6fdb1134-ed1c-4505-a741-6e99dba1e2c8-metrics-certs podName:6fdb1134-ed1c-4505-a741-6e99dba1e2c8 nodeName:}" failed. No retries permitted until 2026-07-06 22:09:50.649564645 +0000 UTC m=+70.208359957 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6fdb1134-ed1c-4505-a741-6e99dba1e2c8-metrics-certs") pod "router-default-78989f99df-sxq46" (UID: "6fdb1134-ed1c-4505-a741-6e99dba1e2c8") : secret "router-metrics-certs-default" not found Jul 06 22:09:42.649669 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:42.649618 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/179af188-f2c6-46da-9d1d-b15ce8fcb039-cluster-monitoring-operator-tls podName:179af188-f2c6-46da-9d1d-b15ce8fcb039 nodeName:}" failed. No retries permitted until 2026-07-06 22:09:50.64961017 +0000 UTC m=+70.208405468 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/179af188-f2c6-46da-9d1d-b15ce8fcb039-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-24gv2" (UID: "179af188-f2c6-46da-9d1d-b15ce8fcb039") : secret "cluster-monitoring-operator-tls" not found Jul 06 22:09:42.649669 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:42.649634 2581 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 06 22:09:42.649669 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:42.649672 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a4aac3c4-83c0-4140-8c7b-5ce240cf2f46-samples-operator-tls podName:a4aac3c4-83c0-4140-8c7b-5ce240cf2f46 nodeName:}" failed. No retries permitted until 2026-07-06 22:09:50.649661017 +0000 UTC m=+70.208456321 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/a4aac3c4-83c0-4140-8c7b-5ce240cf2f46-samples-operator-tls") pod "cluster-samples-operator-56fcf86bf-dkbm6" (UID: "a4aac3c4-83c0-4140-8c7b-5ce240cf2f46") : secret "samples-operator-tls" not found Jul 06 22:09:42.658401 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:42.658375 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t9wcj\" (UniqueName: \"kubernetes.io/projected/bd42289a-105e-46c7-ad15-a6341ce43f15-kube-api-access-t9wcj\") pod \"insights-runtime-extractor-68gjp\" (UID: \"bd42289a-105e-46c7-ad15-a6341ce43f15\") " pod="openshift-insights/insights-runtime-extractor-68gjp" Jul 06 22:09:42.678417 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:42.678386 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:09:42.781373 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:42.781276 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:09:42.908562 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:42.908523 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:09:43.006546 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:43.006509 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:09:43.106576 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:43.106491 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:09:43.155727 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:43.155698 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bd42289a-105e-46c7-ad15-a6341ce43f15-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-68gjp\" (UID: \"bd42289a-105e-46c7-ad15-a6341ce43f15\") " pod="openshift-insights/insights-runtime-extractor-68gjp" Jul 06 22:09:43.155872 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:43.155811 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bd42289a-105e-46c7-ad15-a6341ce43f15-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-68gjp\" (UID: \"bd42289a-105e-46c7-ad15-a6341ce43f15\") " pod="openshift-insights/insights-runtime-extractor-68gjp" Jul 06 22:09:43.155872 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:43.155852 2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 22:09:43.155946 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:43.155910 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bd42289a-105e-46c7-ad15-a6341ce43f15-insights-runtime-extractor-tls podName:bd42289a-105e-46c7-ad15-a6341ce43f15 nodeName:}" failed. No retries permitted until 2026-07-06 22:09:44.15589655 +0000 UTC m=+63.714691849 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/bd42289a-105e-46c7-ad15-a6341ce43f15-insights-runtime-extractor-tls") pod "insights-runtime-extractor-68gjp" (UID: "bd42289a-105e-46c7-ad15-a6341ce43f15") : secret "insights-runtime-extractor-tls" not found Jul 06 22:09:43.155946 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:43.155934 2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 22:09:43.156014 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:43.155985 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bd42289a-105e-46c7-ad15-a6341ce43f15-kube-rbac-proxy-cm podName:bd42289a-105e-46c7-ad15-a6341ce43f15 nodeName:}" failed. No retries permitted until 2026-07-06 22:09:44.155971007 +0000 UTC m=+63.714766306 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/bd42289a-105e-46c7-ad15-a6341ce43f15-kube-rbac-proxy-cm") pod "insights-runtime-extractor-68gjp" (UID: "bd42289a-105e-46c7-ad15-a6341ce43f15") : configmap "kube-rbac-proxy" not found Jul 06 22:09:43.208454 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:43.208414 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:09:43.384004 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:43.383908 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:09:43.391697 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:43.391678 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vt6d6" Jul 06 22:09:43.484931 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:43.484901 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:09:43.496948 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:43.496925 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-rcn5q_86b07166-2f34-443b-9c61-86b7d950ad7b/console-operator/1.log" Jul 06 22:09:43.497295 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:43.497250 2581 scope.go:117] "RemoveContainer" containerID="11d06e0eaa8ccb153c499b1e7fe6f156124abdccb2a253d555b84244c1fc5e8f" Jul 06 22:09:43.497405 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:43.497387 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-rcn5q_openshift-console-operator(86b07166-2f34-443b-9c61-86b7d950ad7b)\"" pod="openshift-console-operator/console-operator-68b757fb67-rcn5q" podUID="86b07166-2f34-443b-9c61-86b7d950ad7b" Jul 06 22:09:43.585397 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:43.585365 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:09:43.646482 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:43.646396 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:09:43.874701 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:43.874673 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-4b7pr_314ffdbc-f9e6-45c0-b37a-d216700a1de5/dns-node-resolver/0.log" Jul 06 22:09:44.153461 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:44.153423 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:09:44.167162 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:44.167133 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bd42289a-105e-46c7-ad15-a6341ce43f15-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-68gjp\" (UID: \"bd42289a-105e-46c7-ad15-a6341ce43f15\") " pod="openshift-insights/insights-runtime-extractor-68gjp" Jul 06 22:09:44.167270 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:44.167221 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bd42289a-105e-46c7-ad15-a6341ce43f15-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-68gjp\" (UID: \"bd42289a-105e-46c7-ad15-a6341ce43f15\") " pod="openshift-insights/insights-runtime-extractor-68gjp" Jul 06 22:09:44.167313 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:44.167292 2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 22:09:44.167313 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:44.167301 2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 22:09:44.167371 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:44.167358 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bd42289a-105e-46c7-ad15-a6341ce43f15-insights-runtime-extractor-tls podName:bd42289a-105e-46c7-ad15-a6341ce43f15 nodeName:}" failed. No retries permitted until 2026-07-06 22:09:46.167339651 +0000 UTC m=+65.726134952 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/bd42289a-105e-46c7-ad15-a6341ce43f15-insights-runtime-extractor-tls") pod "insights-runtime-extractor-68gjp" (UID: "bd42289a-105e-46c7-ad15-a6341ce43f15") : secret "insights-runtime-extractor-tls" not found Jul 06 22:09:44.167407 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:44.167375 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bd42289a-105e-46c7-ad15-a6341ce43f15-kube-rbac-proxy-cm podName:bd42289a-105e-46c7-ad15-a6341ce43f15 nodeName:}" failed. No retries permitted until 2026-07-06 22:09:46.167368211 +0000 UTC m=+65.726163510 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/bd42289a-105e-46c7-ad15-a6341ce43f15-kube-rbac-proxy-cm") pod "insights-runtime-extractor-68gjp" (UID: "bd42289a-105e-46c7-ad15-a6341ce43f15") : configmap "kube-rbac-proxy" not found Jul 06 22:09:44.233277 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:44.233230 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:09:44.986101 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:44.986056 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-758f599c5-mdkhd"] Jul 06 22:09:44.989751 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:44.989735 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-mdkhd" Jul 06 22:09:44.992434 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:44.992412 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 06 22:09:44.992565 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:44.992418 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 06 22:09:44.993664 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:44.993646 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 06 22:09:44.993767 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:44.993651 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-q6w5l\"" Jul 06 22:09:44.993767 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:44.993685 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 06 22:09:44.998635 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:44.998614 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-mdkhd"] Jul 06 22:09:45.074847 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:45.074813 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r7lwz\" (UniqueName: \"kubernetes.io/projected/a24e78a0-301a-426f-a189-55a6e20f8d43-kube-api-access-r7lwz\") pod \"service-ca-758f599c5-mdkhd\" (UID: \"a24e78a0-301a-426f-a189-55a6e20f8d43\") " pod="openshift-service-ca/service-ca-758f599c5-mdkhd" Jul 06 22:09:45.075036 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:45.074912 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/a24e78a0-301a-426f-a189-55a6e20f8d43-signing-key\") pod \"service-ca-758f599c5-mdkhd\" (UID: \"a24e78a0-301a-426f-a189-55a6e20f8d43\") " pod="openshift-service-ca/service-ca-758f599c5-mdkhd" Jul 06 22:09:45.075118 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:45.075074 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/a24e78a0-301a-426f-a189-55a6e20f8d43-signing-cabundle\") pod \"service-ca-758f599c5-mdkhd\" (UID: \"a24e78a0-301a-426f-a189-55a6e20f8d43\") " pod="openshift-service-ca/service-ca-758f599c5-mdkhd" Jul 06 22:09:45.075230 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:45.075214 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-cmtz2_9a1fd480-e89a-4522-b82a-87b8c72124d5/node-ca/0.log" Jul 06 22:09:45.091347 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:45.091311 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:09:45.118410 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:45.118366 2581 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-68b757fb67-rcn5q" Jul 06 22:09:45.118491 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:45.118436 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-rcn5q" Jul 06 22:09:45.118825 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:45.118809 2581 scope.go:117] "RemoveContainer" containerID="11d06e0eaa8ccb153c499b1e7fe6f156124abdccb2a253d555b84244c1fc5e8f" Jul 06 22:09:45.118980 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:45.118964 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-68b757fb67-rcn5q_openshift-console-operator(86b07166-2f34-443b-9c61-86b7d950ad7b)\"" pod="openshift-console-operator/console-operator-68b757fb67-rcn5q" podUID="86b07166-2f34-443b-9c61-86b7d950ad7b" Jul 06 22:09:45.176488 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:45.176454 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/a24e78a0-301a-426f-a189-55a6e20f8d43-signing-cabundle\") pod \"service-ca-758f599c5-mdkhd\" (UID: \"a24e78a0-301a-426f-a189-55a6e20f8d43\") " pod="openshift-service-ca/service-ca-758f599c5-mdkhd" Jul 06 22:09:45.176644 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:45.176555 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r7lwz\" (UniqueName: \"kubernetes.io/projected/a24e78a0-301a-426f-a189-55a6e20f8d43-kube-api-access-r7lwz\") pod \"service-ca-758f599c5-mdkhd\" (UID: \"a24e78a0-301a-426f-a189-55a6e20f8d43\") " pod="openshift-service-ca/service-ca-758f599c5-mdkhd" Jul 06 22:09:45.176644 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:45.176621 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/a24e78a0-301a-426f-a189-55a6e20f8d43-signing-key\") pod \"service-ca-758f599c5-mdkhd\" (UID: \"a24e78a0-301a-426f-a189-55a6e20f8d43\") " pod="openshift-service-ca/service-ca-758f599c5-mdkhd" Jul 06 22:09:45.177168 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:45.177146 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/a24e78a0-301a-426f-a189-55a6e20f8d43-signing-cabundle\") pod \"service-ca-758f599c5-mdkhd\" (UID: \"a24e78a0-301a-426f-a189-55a6e20f8d43\") " pod="openshift-service-ca/service-ca-758f599c5-mdkhd" Jul 06 22:09:45.178913 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:45.178895 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/a24e78a0-301a-426f-a189-55a6e20f8d43-signing-key\") pod \"service-ca-758f599c5-mdkhd\" (UID: \"a24e78a0-301a-426f-a189-55a6e20f8d43\") " pod="openshift-service-ca/service-ca-758f599c5-mdkhd" Jul 06 22:09:45.185969 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:45.185948 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r7lwz\" (UniqueName: \"kubernetes.io/projected/a24e78a0-301a-426f-a189-55a6e20f8d43-kube-api-access-r7lwz\") pod \"service-ca-758f599c5-mdkhd\" (UID: \"a24e78a0-301a-426f-a189-55a6e20f8d43\") " pod="openshift-service-ca/service-ca-758f599c5-mdkhd" Jul 06 22:09:45.192492 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:45.192467 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:09:45.298600 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:45.298523 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-mdkhd" Jul 06 22:09:45.422154 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:45.422123 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-mdkhd"] Jul 06 22:09:45.424866 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:09:45.424833 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda24e78a0_301a_426f_a189_55a6e20f8d43.slice/crio-96298c4b77b50c2c5fce208474b65b1dad6660c4930c80c2595fde2fb1152cd2 WatchSource:0}: Error finding container 96298c4b77b50c2c5fce208474b65b1dad6660c4930c80c2595fde2fb1152cd2: Status 404 returned error can't find the container with id 96298c4b77b50c2c5fce208474b65b1dad6660c4930c80c2595fde2fb1152cd2 Jul 06 22:09:45.503928 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:45.503892 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-mdkhd" event={"ID":"a24e78a0-301a-426f-a189-55a6e20f8d43","Type":"ContainerStarted","Data":"fb29f3dd9d85630d3afdc8bd82f8d9b314d5366205f569ff2c9d6b5acd1fd945"} Jul 06 22:09:45.504043 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:45.503943 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-mdkhd" event={"ID":"a24e78a0-301a-426f-a189-55a6e20f8d43","Type":"ContainerStarted","Data":"96298c4b77b50c2c5fce208474b65b1dad6660c4930c80c2595fde2fb1152cd2"} Jul 06 22:09:45.521896 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:45.521843 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-758f599c5-mdkhd" podStartSLOduration=1.521822 podStartE2EDuration="1.521822s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:09:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:09:45.521429134 +0000 UTC m=+65.080224455" watchObservedRunningTime="2026-07-06 22:09:45.521822 +0000 UTC m=+65.080617324" Jul 06 22:09:45.676307 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:45.676275 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-vf82r_638815a3-fb4e-41ac-9e74-7c3426d561f4/serve-healthcheck-canary/0.log" Jul 06 22:09:45.804119 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:45.804083 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:09:46.188026 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:46.187994 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bd42289a-105e-46c7-ad15-a6341ce43f15-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-68gjp\" (UID: \"bd42289a-105e-46c7-ad15-a6341ce43f15\") " pod="openshift-insights/insights-runtime-extractor-68gjp" Jul 06 22:09:46.188462 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:46.188129 2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 22:09:46.188462 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:46.188189 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bd42289a-105e-46c7-ad15-a6341ce43f15-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-68gjp\" (UID: \"bd42289a-105e-46c7-ad15-a6341ce43f15\") " pod="openshift-insights/insights-runtime-extractor-68gjp" Jul 06 22:09:46.188462 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:46.188235 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bd42289a-105e-46c7-ad15-a6341ce43f15-kube-rbac-proxy-cm podName:bd42289a-105e-46c7-ad15-a6341ce43f15 nodeName:}" failed. No retries permitted until 2026-07-06 22:09:50.188208888 +0000 UTC m=+69.747004192 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/bd42289a-105e-46c7-ad15-a6341ce43f15-kube-rbac-proxy-cm") pod "insights-runtime-extractor-68gjp" (UID: "bd42289a-105e-46c7-ad15-a6341ce43f15") : configmap "kube-rbac-proxy" not found Jul 06 22:09:46.188462 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:46.188265 2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 22:09:46.188462 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:46.188312 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bd42289a-105e-46c7-ad15-a6341ce43f15-insights-runtime-extractor-tls podName:bd42289a-105e-46c7-ad15-a6341ce43f15 nodeName:}" failed. No retries permitted until 2026-07-06 22:09:50.188300331 +0000 UTC m=+69.747095636 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/bd42289a-105e-46c7-ad15-a6341ce43f15-insights-runtime-extractor-tls") pod "insights-runtime-extractor-68gjp" (UID: "bd42289a-105e-46c7-ad15-a6341ce43f15") : secret "insights-runtime-extractor-tls" not found Jul 06 22:09:46.693667 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:46.693588 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tlvkd\" (UniqueName: \"kubernetes.io/projected/9ade138e-0e88-4087-8042-92fe2db1e1d8-kube-api-access-tlvkd\") pod \"network-check-target-l5rc4\" (UID: \"9ade138e-0e88-4087-8042-92fe2db1e1d8\") " pod="openshift-network-diagnostics/network-check-target-l5rc4" Jul 06 22:09:46.693667 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:46.693642 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8a83eb6f-f275-4562-b014-0b48c710b624-metrics-certs\") pod \"network-metrics-daemon-n9hgj\" (UID: \"8a83eb6f-f275-4562-b014-0b48c710b624\") " pod="openshift-multus/network-metrics-daemon-n9hgj" Jul 06 22:09:46.696490 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:46.696460 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 06 22:09:46.697345 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:46.697320 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tlvkd\" (UniqueName: \"kubernetes.io/projected/9ade138e-0e88-4087-8042-92fe2db1e1d8-kube-api-access-tlvkd\") pod \"network-check-target-l5rc4\" (UID: \"9ade138e-0e88-4087-8042-92fe2db1e1d8\") " pod="openshift-network-diagnostics/network-check-target-l5rc4" Jul 06 22:09:46.704972 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:46.704946 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 06 22:09:46.705134 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:46.705117 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8a83eb6f-f275-4562-b014-0b48c710b624-metrics-certs podName:8a83eb6f-f275-4562-b014-0b48c710b624 nodeName:}" failed. No retries permitted until 2026-07-06 22:10:50.705018066 +0000 UTC m=+130.263813365 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8a83eb6f-f275-4562-b014-0b48c710b624-metrics-certs") pod "network-metrics-daemon-n9hgj" (UID: "8a83eb6f-f275-4562-b014-0b48c710b624") : secret "metrics-daemon-secret" not found Jul 06 22:09:46.744595 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:46.744556 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:09:46.794516 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:46.794479 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b42224bc-b677-43bf-94fb-fc75223b963a-original-pull-secret\") pod \"global-pull-secret-syncer-klc48\" (UID: \"b42224bc-b677-43bf-94fb-fc75223b963a\") " pod="kube-system/global-pull-secret-syncer-klc48" Jul 06 22:09:46.797579 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:46.797557 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 06 22:09:46.807684 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:46.807666 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b42224bc-b677-43bf-94fb-fc75223b963a-original-pull-secret\") pod \"global-pull-secret-syncer-klc48\" (UID: \"b42224bc-b677-43bf-94fb-fc75223b963a\") " pod="kube-system/global-pull-secret-syncer-klc48" Jul 06 22:09:46.886221 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:46.886191 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-pk242\"" Jul 06 22:09:46.893976 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:46.893945 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-l5rc4" Jul 06 22:09:46.896812 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:46.896767 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-klc48" Jul 06 22:09:47.043312 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:47.042812 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:09:47.097191 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:47.097119 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-l5rc4"] Jul 06 22:09:47.105040 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:09:47.105015 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ade138e_0e88_4087_8042_92fe2db1e1d8.slice/crio-8c7f7354439aa432e27f54c47df2568127113e3804d320f296f4e2aae3f5e574 WatchSource:0}: Error finding container 8c7f7354439aa432e27f54c47df2568127113e3804d320f296f4e2aae3f5e574: Status 404 returned error can't find the container with id 8c7f7354439aa432e27f54c47df2568127113e3804d320f296f4e2aae3f5e574 Jul 06 22:09:47.116945 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:47.116919 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-klc48"] Jul 06 22:09:47.122054 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:09:47.122026 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb42224bc_b677_43bf_94fb_fc75223b963a.slice/crio-0e0a29c7ce084ef06f9fdffe33ad72bc3dc1197da157bf153afb651c72ef4b58 WatchSource:0}: Error finding container 0e0a29c7ce084ef06f9fdffe33ad72bc3dc1197da157bf153afb651c72ef4b58: Status 404 returned error can't find the container with id 0e0a29c7ce084ef06f9fdffe33ad72bc3dc1197da157bf153afb651c72ef4b58 Jul 06 22:09:47.172206 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:47.172180 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:09:47.275368 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:47.275243 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:09:47.504266 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:47.504231 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:09:47.515634 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:47.515593 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-klc48" event={"ID":"b42224bc-b677-43bf-94fb-fc75223b963a","Type":"ContainerStarted","Data":"0e0a29c7ce084ef06f9fdffe33ad72bc3dc1197da157bf153afb651c72ef4b58"} Jul 06 22:09:47.517686 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:47.517610 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-l5rc4" event={"ID":"9ade138e-0e88-4087-8042-92fe2db1e1d8","Type":"ContainerStarted","Data":"1817d759095bc46096295f77a45430f5a4e35d86c102fcf9ddcee838326d2ad2"} Jul 06 22:09:47.517686 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:47.517656 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-l5rc4" event={"ID":"9ade138e-0e88-4087-8042-92fe2db1e1d8","Type":"ContainerStarted","Data":"8c7f7354439aa432e27f54c47df2568127113e3804d320f296f4e2aae3f5e574"} Jul 06 22:09:47.518002 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:47.517952 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-l5rc4" Jul 06 22:09:47.547326 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:47.547239 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-l5rc4" podStartSLOduration=66.547225647 podStartE2EDuration="1m6.547225647s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:08:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:09:47.544913644 +0000 UTC m=+67.103708965" watchObservedRunningTime="2026-07-06 22:09:47.547225647 +0000 UTC m=+67.106020968" Jul 06 22:09:47.650365 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:47.650330 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:09:47.830275 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:47.830187 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:09:48.840457 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:48.840419 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:09:49.036132 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:49.036097 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:09:49.256438 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:49.256400 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:09:49.449632 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:49.449577 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:09:49.816683 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:49.816632 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:09:50.228212 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:50.228163 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bfb8f404-9338-404f-81f9-beb0a5b26d8a-registry-tls\") pod \"image-registry-65c85c5ff6-4fxss\" (UID: \"bfb8f404-9338-404f-81f9-beb0a5b26d8a\") " pod="openshift-image-registry/image-registry-65c85c5ff6-4fxss" Jul 06 22:09:50.228692 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:50.228268 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bd42289a-105e-46c7-ad15-a6341ce43f15-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-68gjp\" (UID: \"bd42289a-105e-46c7-ad15-a6341ce43f15\") " pod="openshift-insights/insights-runtime-extractor-68gjp" Jul 06 22:09:50.228692 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:50.228311 2581 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 22:09:50.228692 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:50.228320 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/da5ae4fc-9377-4d40-89cf-efc75bac039e-metrics-tls\") pod \"dns-default-nfklb\" (UID: \"da5ae4fc-9377-4d40-89cf-efc75bac039e\") " pod="openshift-dns/dns-default-nfklb" Jul 06 22:09:50.228692 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:50.228333 2581 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-65c85c5ff6-4fxss: secret "image-registry-tls" not found Jul 06 22:09:50.228692 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:50.228388 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bfb8f404-9338-404f-81f9-beb0a5b26d8a-registry-tls podName:bfb8f404-9338-404f-81f9-beb0a5b26d8a nodeName:}" failed. No retries permitted until 2026-07-06 22:10:22.228373299 +0000 UTC m=+101.787168598 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bfb8f404-9338-404f-81f9-beb0a5b26d8a-registry-tls") pod "image-registry-65c85c5ff6-4fxss" (UID: "bfb8f404-9338-404f-81f9-beb0a5b26d8a") : secret "image-registry-tls" not found Jul 06 22:09:50.228692 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:50.228403 2581 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 22:09:50.228692 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:50.228410 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bd42289a-105e-46c7-ad15-a6341ce43f15-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-68gjp\" (UID: \"bd42289a-105e-46c7-ad15-a6341ce43f15\") " pod="openshift-insights/insights-runtime-extractor-68gjp" Jul 06 22:09:50.228692 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:50.228473 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bd42289a-105e-46c7-ad15-a6341ce43f15-insights-runtime-extractor-tls podName:bd42289a-105e-46c7-ad15-a6341ce43f15 nodeName:}" failed. No retries permitted until 2026-07-06 22:09:58.228459492 +0000 UTC m=+77.787254790 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/bd42289a-105e-46c7-ad15-a6341ce43f15-insights-runtime-extractor-tls") pod "insights-runtime-extractor-68gjp" (UID: "bd42289a-105e-46c7-ad15-a6341ce43f15") : secret "insights-runtime-extractor-tls" not found Jul 06 22:09:50.228692 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:50.228523 2581 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 22:09:50.228692 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:50.228562 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bd42289a-105e-46c7-ad15-a6341ce43f15-kube-rbac-proxy-cm podName:bd42289a-105e-46c7-ad15-a6341ce43f15 nodeName:}" failed. No retries permitted until 2026-07-06 22:09:58.22855134 +0000 UTC m=+77.787346641 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/bd42289a-105e-46c7-ad15-a6341ce43f15-kube-rbac-proxy-cm") pod "insights-runtime-extractor-68gjp" (UID: "bd42289a-105e-46c7-ad15-a6341ce43f15") : configmap "kube-rbac-proxy" not found Jul 06 22:09:50.230634 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:50.230614 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/da5ae4fc-9377-4d40-89cf-efc75bac039e-metrics-tls\") pod \"dns-default-nfklb\" (UID: \"da5ae4fc-9377-4d40-89cf-efc75bac039e\") " pod="openshift-dns/dns-default-nfklb" Jul 06 22:09:50.435990 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:50.435801 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-ddjw7\"" Jul 06 22:09:50.437312 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:50.437265 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:09:50.444041 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:50.444000 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-nfklb" Jul 06 22:09:50.504197 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:50.504164 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:09:50.592733 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:50.592685 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-nfklb"] Jul 06 22:09:50.595861 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:09:50.595827 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda5ae4fc_9377_4d40_89cf_efc75bac039e.slice/crio-968c7c5359d892f7d8b10bccb432ec1e0f3358edc4cdd8bdafcfdc6d31786154 WatchSource:0}: Error finding container 968c7c5359d892f7d8b10bccb432ec1e0f3358edc4cdd8bdafcfdc6d31786154: Status 404 returned error can't find the container with id 968c7c5359d892f7d8b10bccb432ec1e0f3358edc4cdd8bdafcfdc6d31786154 Jul 06 22:09:50.648289 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:50.648254 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:09:50.734549 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:50.734507 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6fdb1134-ed1c-4505-a741-6e99dba1e2c8-service-ca-bundle\") pod \"router-default-78989f99df-sxq46\" (UID: \"6fdb1134-ed1c-4505-a741-6e99dba1e2c8\") " pod="openshift-ingress/router-default-78989f99df-sxq46" Jul 06 22:09:50.734726 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:50.734633 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6fdb1134-ed1c-4505-a741-6e99dba1e2c8-metrics-certs\") pod \"router-default-78989f99df-sxq46\" (UID: \"6fdb1134-ed1c-4505-a741-6e99dba1e2c8\") " pod="openshift-ingress/router-default-78989f99df-sxq46" Jul 06 22:09:50.734726 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:50.734704 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/179af188-f2c6-46da-9d1d-b15ce8fcb039-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-24gv2\" (UID: \"179af188-f2c6-46da-9d1d-b15ce8fcb039\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-24gv2" Jul 06 22:09:50.734844 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:50.734735 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a4aac3c4-83c0-4140-8c7b-5ce240cf2f46-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-dkbm6\" (UID: \"a4aac3c4-83c0-4140-8c7b-5ce240cf2f46\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-dkbm6" Jul 06 22:09:50.734844 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:50.734766 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6fdb1134-ed1c-4505-a741-6e99dba1e2c8-service-ca-bundle podName:6fdb1134-ed1c-4505-a741-6e99dba1e2c8 nodeName:}" failed. No retries permitted until 2026-07-06 22:10:06.734741472 +0000 UTC m=+86.293536777 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/6fdb1134-ed1c-4505-a741-6e99dba1e2c8-service-ca-bundle") pod "router-default-78989f99df-sxq46" (UID: "6fdb1134-ed1c-4505-a741-6e99dba1e2c8") : configmap references non-existent config key: service-ca.crt Jul 06 22:09:50.734958 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:50.734896 2581 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 06 22:09:50.734958 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:50.734944 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/179af188-f2c6-46da-9d1d-b15ce8fcb039-cluster-monitoring-operator-tls podName:179af188-f2c6-46da-9d1d-b15ce8fcb039 nodeName:}" failed. No retries permitted until 2026-07-06 22:10:06.734930281 +0000 UTC m=+86.293725591 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/179af188-f2c6-46da-9d1d-b15ce8fcb039-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-24gv2" (UID: "179af188-f2c6-46da-9d1d-b15ce8fcb039") : secret "cluster-monitoring-operator-tls" not found Jul 06 22:09:50.735090 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:50.734990 2581 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 06 22:09:50.735090 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:09:50.735029 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6fdb1134-ed1c-4505-a741-6e99dba1e2c8-metrics-certs podName:6fdb1134-ed1c-4505-a741-6e99dba1e2c8 nodeName:}" failed. No retries permitted until 2026-07-06 22:10:06.735018001 +0000 UTC m=+86.293813308 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6fdb1134-ed1c-4505-a741-6e99dba1e2c8-metrics-certs") pod "router-default-78989f99df-sxq46" (UID: "6fdb1134-ed1c-4505-a741-6e99dba1e2c8") : secret "router-metrics-certs-default" not found Jul 06 22:09:50.737379 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:50.737358 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a4aac3c4-83c0-4140-8c7b-5ce240cf2f46-samples-operator-tls\") pod \"cluster-samples-operator-56fcf86bf-dkbm6\" (UID: \"a4aac3c4-83c0-4140-8c7b-5ce240cf2f46\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-dkbm6" Jul 06 22:09:50.808149 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:50.808062 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-txvjp\"" Jul 06 22:09:50.815580 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:50.815557 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-dkbm6" Jul 06 22:09:51.041516 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:51.041483 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:09:51.180982 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:51.180946 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:09:51.282458 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:51.282416 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:09:51.534807 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:51.534717 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-nfklb" event={"ID":"da5ae4fc-9377-4d40-89cf-efc75bac039e","Type":"ContainerStarted","Data":"968c7c5359d892f7d8b10bccb432ec1e0f3358edc4cdd8bdafcfdc6d31786154"} Jul 06 22:09:51.639904 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:51.639826 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:09:51.743311 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:51.743267 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:09:52.113580 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:52.113541 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:09:52.448669 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:52.448638 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-dkbm6"] Jul 06 22:09:52.539910 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:52.539867 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-klc48" event={"ID":"b42224bc-b677-43bf-94fb-fc75223b963a","Type":"ContainerStarted","Data":"82234ac71fb8910d96af5b306b9dafc7860377c9775bf62b44dd7d378e4663bb"} Jul 06 22:09:52.541295 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:52.541258 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-dkbm6" event={"ID":"a4aac3c4-83c0-4140-8c7b-5ce240cf2f46","Type":"ContainerStarted","Data":"17711301dab45706cee3b470d9131197b417815c6260f7bc0d3aefe92371e31d"} Jul 06 22:09:52.557927 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:52.557880 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-klc48" podStartSLOduration=66.309453563 podStartE2EDuration="1m11.557861129s" totalImagesPullingTime="5.248407566s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 22:08:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:09:52.557277671 +0000 UTC m=+72.116072994" watchObservedRunningTime="2026-07-06 22:09:52.557861129 +0000 UTC m=+72.116656452" Jul 06 22:09:52.627414 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:52.627319 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:09:53.030515 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:53.030479 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:09:53.216383 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:53.216346 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:09:53.435727 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:53.435656 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:09:53.635839 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:53.635802 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:09:54.055305 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:54.055212 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:09:54.551720 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:54.551674 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-nfklb" event={"ID":"da5ae4fc-9377-4d40-89cf-efc75bac039e","Type":"ContainerStarted","Data":"fdbd24332d69816a3719b9b3b0d967be627426256472b7cbf872762821afbad0"} Jul 06 22:09:54.551910 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:54.551735 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-nfklb" event={"ID":"da5ae4fc-9377-4d40-89cf-efc75bac039e","Type":"ContainerStarted","Data":"0a1a99a96310f26ff63cdb49e32bc8e959663745283a65a2f3ca1483f7593402"} Jul 06 22:09:54.551910 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:54.551831 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-nfklb" Jul 06 22:09:54.570771 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:54.570726 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-nfklb" podStartSLOduration=33.258545664 podStartE2EDuration="36.570711057s" totalImagesPullingTime="3.312165393s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:09:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:09:54.569324117 +0000 UTC m=+74.128119440" watchObservedRunningTime="2026-07-06 22:09:54.570711057 +0000 UTC m=+74.129506378" Jul 06 22:09:54.628975 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:54.628949 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:09:55.042914 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:55.042879 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:09:55.233347 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:55.233310 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:09:55.557844 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:55.557810 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-dkbm6" event={"ID":"a4aac3c4-83c0-4140-8c7b-5ce240cf2f46","Type":"ContainerStarted","Data":"ba637200984d6917427d3f227520a058e685191cc3bb50b674af5fdaf087eb8c"} Jul 06 22:09:55.558010 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:55.557865 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-dkbm6" event={"ID":"a4aac3c4-83c0-4140-8c7b-5ce240cf2f46","Type":"ContainerStarted","Data":"7096ffc16d5b0e75b6495d0a1b0e93e057108df5bbedbf0877af121d223fde50"} Jul 06 22:09:55.576514 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:55.576458 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-56fcf86bf-dkbm6" podStartSLOduration=19.444839228 podStartE2EDuration="21.576444752s" totalImagesPullingTime="2.131605524s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:09:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:09:55.575009724 +0000 UTC m=+75.133805053" watchObservedRunningTime="2026-07-06 22:09:55.576444752 +0000 UTC m=+75.135240094" Jul 06 22:09:55.633292 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:55.633253 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:09:55.733606 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:55.733574 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:09:56.021571 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:56.021539 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:09:56.086577 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:56.086535 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:09:56.238671 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:56.238631 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:09:56.638774 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:56.638739 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:09:57.039668 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:57.039624 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:09:57.236869 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:57.236834 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:09:57.361892 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:57.361805 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:09:57.466524 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:57.466495 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:09:58.305193 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:58.305149 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bd42289a-105e-46c7-ad15-a6341ce43f15-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-68gjp\" (UID: \"bd42289a-105e-46c7-ad15-a6341ce43f15\") " pod="openshift-insights/insights-runtime-extractor-68gjp" Jul 06 22:09:58.305653 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:58.305249 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bd42289a-105e-46c7-ad15-a6341ce43f15-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-68gjp\" (UID: \"bd42289a-105e-46c7-ad15-a6341ce43f15\") " pod="openshift-insights/insights-runtime-extractor-68gjp" Jul 06 22:09:58.305742 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:58.305722 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bd42289a-105e-46c7-ad15-a6341ce43f15-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-68gjp\" (UID: \"bd42289a-105e-46c7-ad15-a6341ce43f15\") " pod="openshift-insights/insights-runtime-extractor-68gjp" Jul 06 22:09:58.307541 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:58.307522 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bd42289a-105e-46c7-ad15-a6341ce43f15-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-68gjp\" (UID: \"bd42289a-105e-46c7-ad15-a6341ce43f15\") " pod="openshift-insights/insights-runtime-extractor-68gjp" Jul 06 22:09:58.401972 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:58.401939 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-68gjp" Jul 06 22:09:58.530816 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:58.530791 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-68gjp"] Jul 06 22:09:58.533304 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:09:58.533276 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd42289a_105e_46c7_ad15_a6341ce43f15.slice/crio-ca869f4de9803da7c7e056bafad02268d4f3a6057ff20b3c860ffeba9a15384f WatchSource:0}: Error finding container ca869f4de9803da7c7e056bafad02268d4f3a6057ff20b3c860ffeba9a15384f: Status 404 returned error can't find the container with id ca869f4de9803da7c7e056bafad02268d4f3a6057ff20b3c860ffeba9a15384f Jul 06 22:09:58.569711 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:09:58.569632 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-68gjp" event={"ID":"bd42289a-105e-46c7-ad15-a6341ce43f15","Type":"ContainerStarted","Data":"ca869f4de9803da7c7e056bafad02268d4f3a6057ff20b3c860ffeba9a15384f"} Jul 06 22:10:00.971975 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:00.971939 2581 scope.go:117] "RemoveContainer" containerID="11d06e0eaa8ccb153c499b1e7fe6f156124abdccb2a253d555b84244c1fc5e8f" Jul 06 22:10:02.585702 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:02.585673 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-rcn5q_86b07166-2f34-443b-9c61-86b7d950ad7b/console-operator/1.log" Jul 06 22:10:02.586207 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:02.585801 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-68b757fb67-rcn5q" event={"ID":"86b07166-2f34-443b-9c61-86b7d950ad7b","Type":"ContainerStarted","Data":"a7ead61a462f226adef441d677399970e3960f58cc3d41f6ea43ee373af34b79"} Jul 06 22:10:02.586207 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:02.586143 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-68b757fb67-rcn5q" Jul 06 22:10:02.587403 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:02.587374 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-68gjp" event={"ID":"bd42289a-105e-46c7-ad15-a6341ce43f15","Type":"ContainerStarted","Data":"711723785aed34e788ddb65aa18eff8bb09f2934080cbdf242b36bc8d6af5004"} Jul 06 22:10:02.592706 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:02.592678 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-68b757fb67-rcn5q" Jul 06 22:10:02.607857 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:02.607803 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-68b757fb67-rcn5q" podStartSLOduration=23.434100958 podStartE2EDuration="28.607787644s" totalImagesPullingTime="5.173686686s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:09:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:10:02.60711291 +0000 UTC m=+82.165908231" watchObservedRunningTime="2026-07-06 22:10:02.607787644 +0000 UTC m=+82.166582966" Jul 06 22:10:02.647163 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:02.647126 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:10:02.780246 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:02.780198 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:10:02.956804 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:02.956761 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:10:03.326518 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:03.326328 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:10:03.427099 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:03.427025 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:10:03.592692 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:03.592608 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-68gjp" event={"ID":"bd42289a-105e-46c7-ad15-a6341ce43f15","Type":"ContainerStarted","Data":"c541346d366ac90dda41634784608633aeb3361d43228e48a4e7e3e0ca28bb00"} Jul 06 22:10:03.726013 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:03.725982 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:10:03.827469 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:03.827423 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:10:03.954031 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:03.953976 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:10:04.013247 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:04.013202 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:10:04.133218 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:04.133176 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:10:04.335977 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:04.335869 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:10:04.560788 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:04.560760 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-nfklb" Jul 06 22:10:04.640178 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:04.640092 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:10:05.554565 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:05.554520 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:10:05.629134 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:05.629097 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:10:06.156786 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:06.156694 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-554fdf44bf-lkgh7"] Jul 06 22:10:06.160193 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:06.160170 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-554fdf44bf-lkgh7" Jul 06 22:10:06.163462 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:06.163439 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 06 22:10:06.163462 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:06.163451 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 06 22:10:06.164463 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:06.164446 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jul 06 22:10:06.164555 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:06.164470 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jul 06 22:10:06.164555 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:06.164475 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jul 06 22:10:06.164555 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:06.164524 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 06 22:10:06.164713 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:06.164562 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jul 06 22:10:06.174811 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:06.174780 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-554fdf44bf-lkgh7"] Jul 06 22:10:06.273402 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:06.273366 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:10:06.278345 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:06.278318 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r9ds6\" (UniqueName: \"kubernetes.io/projected/5a952e96-6982-4a3b-8c64-9ab5f242a692-kube-api-access-r9ds6\") pod \"cluster-proxy-proxy-agent-554fdf44bf-lkgh7\" (UID: \"5a952e96-6982-4a3b-8c64-9ab5f242a692\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-554fdf44bf-lkgh7" Jul 06 22:10:06.278480 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:06.278366 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/5a952e96-6982-4a3b-8c64-9ab5f242a692-ca\") pod \"cluster-proxy-proxy-agent-554fdf44bf-lkgh7\" (UID: \"5a952e96-6982-4a3b-8c64-9ab5f242a692\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-554fdf44bf-lkgh7" Jul 06 22:10:06.278480 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:06.278389 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/5a952e96-6982-4a3b-8c64-9ab5f242a692-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-554fdf44bf-lkgh7\" (UID: \"5a952e96-6982-4a3b-8c64-9ab5f242a692\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-554fdf44bf-lkgh7" Jul 06 22:10:06.278480 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:06.278443 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5a952e96-6982-4a3b-8c64-9ab5f242a692-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-554fdf44bf-lkgh7\" (UID: \"5a952e96-6982-4a3b-8c64-9ab5f242a692\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-554fdf44bf-lkgh7" Jul 06 22:10:06.278591 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:06.278524 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/5a952e96-6982-4a3b-8c64-9ab5f242a692-additional-service-ca\") pod \"cluster-proxy-proxy-agent-554fdf44bf-lkgh7\" (UID: \"5a952e96-6982-4a3b-8c64-9ab5f242a692\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-554fdf44bf-lkgh7" Jul 06 22:10:06.278591 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:06.278557 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/5a952e96-6982-4a3b-8c64-9ab5f242a692-hub\") pod \"cluster-proxy-proxy-agent-554fdf44bf-lkgh7\" (UID: \"5a952e96-6982-4a3b-8c64-9ab5f242a692\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-554fdf44bf-lkgh7" Jul 06 22:10:06.355228 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:06.355187 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:10:06.379943 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:06.379902 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/5a952e96-6982-4a3b-8c64-9ab5f242a692-additional-service-ca\") pod \"cluster-proxy-proxy-agent-554fdf44bf-lkgh7\" (UID: \"5a952e96-6982-4a3b-8c64-9ab5f242a692\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-554fdf44bf-lkgh7" Jul 06 22:10:06.379943 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:06.379943 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/5a952e96-6982-4a3b-8c64-9ab5f242a692-hub\") pod \"cluster-proxy-proxy-agent-554fdf44bf-lkgh7\" (UID: \"5a952e96-6982-4a3b-8c64-9ab5f242a692\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-554fdf44bf-lkgh7" Jul 06 22:10:06.380230 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:06.379972 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r9ds6\" (UniqueName: \"kubernetes.io/projected/5a952e96-6982-4a3b-8c64-9ab5f242a692-kube-api-access-r9ds6\") pod \"cluster-proxy-proxy-agent-554fdf44bf-lkgh7\" (UID: \"5a952e96-6982-4a3b-8c64-9ab5f242a692\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-554fdf44bf-lkgh7" Jul 06 22:10:06.380230 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:06.380004 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/5a952e96-6982-4a3b-8c64-9ab5f242a692-ca\") pod \"cluster-proxy-proxy-agent-554fdf44bf-lkgh7\" (UID: \"5a952e96-6982-4a3b-8c64-9ab5f242a692\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-554fdf44bf-lkgh7" Jul 06 22:10:06.380335 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:06.380253 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/5a952e96-6982-4a3b-8c64-9ab5f242a692-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-554fdf44bf-lkgh7\" (UID: \"5a952e96-6982-4a3b-8c64-9ab5f242a692\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-554fdf44bf-lkgh7" Jul 06 22:10:06.380335 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:06.380304 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5a952e96-6982-4a3b-8c64-9ab5f242a692-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-554fdf44bf-lkgh7\" (UID: \"5a952e96-6982-4a3b-8c64-9ab5f242a692\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-554fdf44bf-lkgh7" Jul 06 22:10:06.380747 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:06.380722 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/5a952e96-6982-4a3b-8c64-9ab5f242a692-additional-service-ca\") pod \"cluster-proxy-proxy-agent-554fdf44bf-lkgh7\" (UID: \"5a952e96-6982-4a3b-8c64-9ab5f242a692\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-554fdf44bf-lkgh7" Jul 06 22:10:06.382532 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:06.382502 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/5a952e96-6982-4a3b-8c64-9ab5f242a692-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-554fdf44bf-lkgh7\" (UID: \"5a952e96-6982-4a3b-8c64-9ab5f242a692\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-554fdf44bf-lkgh7" Jul 06 22:10:06.382643 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:06.382507 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/5a952e96-6982-4a3b-8c64-9ab5f242a692-ca\") pod \"cluster-proxy-proxy-agent-554fdf44bf-lkgh7\" (UID: \"5a952e96-6982-4a3b-8c64-9ab5f242a692\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-554fdf44bf-lkgh7" Jul 06 22:10:06.382970 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:06.382952 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/5a952e96-6982-4a3b-8c64-9ab5f242a692-hub\") pod \"cluster-proxy-proxy-agent-554fdf44bf-lkgh7\" (UID: \"5a952e96-6982-4a3b-8c64-9ab5f242a692\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-554fdf44bf-lkgh7" Jul 06 22:10:06.383118 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:06.383098 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/5a952e96-6982-4a3b-8c64-9ab5f242a692-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-554fdf44bf-lkgh7\" (UID: \"5a952e96-6982-4a3b-8c64-9ab5f242a692\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-554fdf44bf-lkgh7" Jul 06 22:10:06.394382 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:06.394348 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r9ds6\" (UniqueName: \"kubernetes.io/projected/5a952e96-6982-4a3b-8c64-9ab5f242a692-kube-api-access-r9ds6\") pod \"cluster-proxy-proxy-agent-554fdf44bf-lkgh7\" (UID: \"5a952e96-6982-4a3b-8c64-9ab5f242a692\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-554fdf44bf-lkgh7" Jul 06 22:10:06.455171 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:06.455138 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:10:06.485126 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:06.485093 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-554fdf44bf-lkgh7" Jul 06 22:10:06.557974 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:06.557938 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:10:06.606674 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:06.606636 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-68gjp" event={"ID":"bd42289a-105e-46c7-ad15-a6341ce43f15","Type":"ContainerStarted","Data":"e1f3193300b91db90378ef22f39fa0651f492f896e4a3be7b8606e30ade804a9"} Jul 06 22:10:06.615261 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:06.615235 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-554fdf44bf-lkgh7"] Jul 06 22:10:06.618520 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:10:06.618487 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5a952e96_6982_4a3b_8c64_9ab5f242a692.slice/crio-de5094e1a61a43fd4bcc3b653ef34683e68572886ddab2d1b9a9717880df4acb WatchSource:0}: Error finding container de5094e1a61a43fd4bcc3b653ef34683e68572886ddab2d1b9a9717880df4acb: Status 404 returned error can't find the container with id de5094e1a61a43fd4bcc3b653ef34683e68572886ddab2d1b9a9717880df4acb Jul 06 22:10:06.632165 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:06.632116 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-68gjp" podStartSLOduration=17.421239549 podStartE2EDuration="24.632099748s" totalImagesPullingTime="7.210860199s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 22:09:42 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:10:06.628510834 +0000 UTC m=+86.187306154" watchObservedRunningTime="2026-07-06 22:10:06.632099748 +0000 UTC m=+86.190895069" Jul 06 22:10:06.634352 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:06.634328 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:10:06.744761 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:06.744678 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:10:06.783808 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:06.783773 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6fdb1134-ed1c-4505-a741-6e99dba1e2c8-metrics-certs\") pod \"router-default-78989f99df-sxq46\" (UID: \"6fdb1134-ed1c-4505-a741-6e99dba1e2c8\") " pod="openshift-ingress/router-default-78989f99df-sxq46" Jul 06 22:10:06.784026 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:06.783842 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/179af188-f2c6-46da-9d1d-b15ce8fcb039-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-24gv2\" (UID: \"179af188-f2c6-46da-9d1d-b15ce8fcb039\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-24gv2" Jul 06 22:10:06.784026 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:06.783902 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6fdb1134-ed1c-4505-a741-6e99dba1e2c8-service-ca-bundle\") pod \"router-default-78989f99df-sxq46\" (UID: \"6fdb1134-ed1c-4505-a741-6e99dba1e2c8\") " pod="openshift-ingress/router-default-78989f99df-sxq46" Jul 06 22:10:06.784596 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:06.784570 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6fdb1134-ed1c-4505-a741-6e99dba1e2c8-service-ca-bundle\") pod \"router-default-78989f99df-sxq46\" (UID: \"6fdb1134-ed1c-4505-a741-6e99dba1e2c8\") " pod="openshift-ingress/router-default-78989f99df-sxq46" Jul 06 22:10:06.786334 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:06.786300 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6fdb1134-ed1c-4505-a741-6e99dba1e2c8-metrics-certs\") pod \"router-default-78989f99df-sxq46\" (UID: \"6fdb1134-ed1c-4505-a741-6e99dba1e2c8\") " pod="openshift-ingress/router-default-78989f99df-sxq46" Jul 06 22:10:06.786418 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:06.786344 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/179af188-f2c6-46da-9d1d-b15ce8fcb039-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-24gv2\" (UID: \"179af188-f2c6-46da-9d1d-b15ce8fcb039\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-24gv2" Jul 06 22:10:06.990789 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:06.990760 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-957fd\"" Jul 06 22:10:06.998942 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:06.998887 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-24gv2" Jul 06 22:10:07.026476 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:07.026449 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-927dm\"" Jul 06 22:10:07.035452 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:07.035427 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-78989f99df-sxq46" Jul 06 22:10:07.110883 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:07.110844 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:10:07.137709 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:07.137677 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-24gv2"] Jul 06 22:10:07.141218 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:10:07.141193 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod179af188_f2c6_46da_9d1d_b15ce8fcb039.slice/crio-d0237d8ce39c4a6fe8b0792a062a92d157704754783a13bdc14b9a44c3c9dc19 WatchSource:0}: Error finding container d0237d8ce39c4a6fe8b0792a062a92d157704754783a13bdc14b9a44c3c9dc19: Status 404 returned error can't find the container with id d0237d8ce39c4a6fe8b0792a062a92d157704754783a13bdc14b9a44c3c9dc19 Jul 06 22:10:07.175517 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:07.175478 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:10:07.186571 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:07.186545 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-78989f99df-sxq46"] Jul 06 22:10:07.188774 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:10:07.188750 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6fdb1134_ed1c_4505_a741_6e99dba1e2c8.slice/crio-e8ecb5a70b21f9d58ce658ab8097239c0a6bf8f815353c422da12ceea46ef4cc WatchSource:0}: Error finding container e8ecb5a70b21f9d58ce658ab8097239c0a6bf8f815353c422da12ceea46ef4cc: Status 404 returned error can't find the container with id e8ecb5a70b21f9d58ce658ab8097239c0a6bf8f815353c422da12ceea46ef4cc Jul 06 22:10:07.612014 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:07.611921 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-78989f99df-sxq46" event={"ID":"6fdb1134-ed1c-4505-a741-6e99dba1e2c8","Type":"ContainerStarted","Data":"6aa7eeba142b758edea475af80277f899e13baaa0bde544c32aba3e49065e953"} Jul 06 22:10:07.612014 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:07.611981 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-78989f99df-sxq46" event={"ID":"6fdb1134-ed1c-4505-a741-6e99dba1e2c8","Type":"ContainerStarted","Data":"e8ecb5a70b21f9d58ce658ab8097239c0a6bf8f815353c422da12ceea46ef4cc"} Jul 06 22:10:07.612993 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:07.612971 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-24gv2" event={"ID":"179af188-f2c6-46da-9d1d-b15ce8fcb039","Type":"ContainerStarted","Data":"d0237d8ce39c4a6fe8b0792a062a92d157704754783a13bdc14b9a44c3c9dc19"} Jul 06 22:10:07.613954 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:07.613934 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-554fdf44bf-lkgh7" event={"ID":"5a952e96-6982-4a3b-8c64-9ab5f242a692","Type":"ContainerStarted","Data":"de5094e1a61a43fd4bcc3b653ef34683e68572886ddab2d1b9a9717880df4acb"} Jul 06 22:10:07.633397 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:07.633350 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-78989f99df-sxq46" podStartSLOduration=33.633336136 podStartE2EDuration="33.633336136s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:09:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:10:07.631966099 +0000 UTC m=+87.190761420" watchObservedRunningTime="2026-07-06 22:10:07.633336136 +0000 UTC m=+87.192131457" Jul 06 22:10:08.036444 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:08.036405 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-78989f99df-sxq46" Jul 06 22:10:08.038881 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:08.038858 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-78989f99df-sxq46" Jul 06 22:10:08.158136 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:08.158053 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:10:08.239691 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:08.239657 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:10:08.618125 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:08.618083 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-78989f99df-sxq46" Jul 06 22:10:08.619372 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:08.619345 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-78989f99df-sxq46" Jul 06 22:10:10.021491 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:10.021445 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:10:10.193201 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:10.193161 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:10:10.257154 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:10.257104 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:10:10.316595 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:10.316511 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:10:10.408208 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:10.408174 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:10:10.625976 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:10.625893 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-24gv2" event={"ID":"179af188-f2c6-46da-9d1d-b15ce8fcb039","Type":"ContainerStarted","Data":"23dd403b36e4745a1ec72b9d894b8ec8aeccca9cbc0094d8304dc717e6915e65"} Jul 06 22:10:10.653322 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:10.653275 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-24gv2" podStartSLOduration=34.145960973 podStartE2EDuration="36.653259585s" totalImagesPullingTime="2.507298612s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:09:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:10:10.652197057 +0000 UTC m=+90.210992380" watchObservedRunningTime="2026-07-06 22:10:10.653259585 +0000 UTC m=+90.212054905" Jul 06 22:10:13.769657 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:13.769624 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:10:13.870160 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:13.870124 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:10:13.930084 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:13.930034 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:10:16.227346 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:16.227283 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:10:16.303232 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:16.303201 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:10:16.472696 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:16.472660 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:10:17.988212 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:17.988173 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:10:18.052755 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:18.052708 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:10:18.221689 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:18.221656 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:10:18.283477 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:18.283387 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:10:18.524342 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:18.524310 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-l5rc4" Jul 06 22:10:19.661424 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:19.661342 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-554fdf44bf-lkgh7" event={"ID":"5a952e96-6982-4a3b-8c64-9ab5f242a692","Type":"ContainerStarted","Data":"cdc6a70be691aef9359721ed85ed2a6270458e18985ab7074f137d80bfa7eb66"} Jul 06 22:10:19.661424 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:19.661383 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-554fdf44bf-lkgh7" event={"ID":"5a952e96-6982-4a3b-8c64-9ab5f242a692","Type":"ContainerStarted","Data":"6033a2525949ee277d84123ae8faeb215f6483a41fc30ea6fb88c22541e20414"} Jul 06 22:10:19.661424 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:19.661399 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-554fdf44bf-lkgh7" event={"ID":"5a952e96-6982-4a3b-8c64-9ab5f242a692","Type":"ContainerStarted","Data":"816fd561da6555fd7f8490a2921aa3562ff599427fb75ee875d30cbb4fa2b025"} Jul 06 22:10:19.681003 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:19.680953 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-554fdf44bf-lkgh7" podStartSLOduration=0.913471743 podStartE2EDuration="13.680939752s" totalImagesPullingTime="12.767468009s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:10:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:10:19.67950249 +0000 UTC m=+99.238297810" watchObservedRunningTime="2026-07-06 22:10:19.680939752 +0000 UTC m=+99.239735051" Jul 06 22:10:19.911196 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:19.911158 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:10:20.029549 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:20.029513 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:10:20.088800 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:20.088760 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:10:20.769289 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:20.769256 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:10:20.869767 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:20.869732 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:10:21.955975 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:21.955940 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:10:22.318563 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:22.318470 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bfb8f404-9338-404f-81f9-beb0a5b26d8a-registry-tls\") pod \"image-registry-65c85c5ff6-4fxss\" (UID: \"bfb8f404-9338-404f-81f9-beb0a5b26d8a\") " pod="openshift-image-registry/image-registry-65c85c5ff6-4fxss" Jul 06 22:10:22.321712 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:22.321678 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bfb8f404-9338-404f-81f9-beb0a5b26d8a-registry-tls\") pod \"image-registry-65c85c5ff6-4fxss\" (UID: \"bfb8f404-9338-404f-81f9-beb0a5b26d8a\") " pod="openshift-image-registry/image-registry-65c85c5ff6-4fxss" Jul 06 22:10:22.371769 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:22.371724 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:10:22.485836 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:22.485797 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:10:22.522974 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:22.522943 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-kshfh\"" Jul 06 22:10:22.531277 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:22.531251 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-65c85c5ff6-4fxss" Jul 06 22:10:22.623060 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:22.622972 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:10:22.694222 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:22.694081 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-65c85c5ff6-4fxss"] Jul 06 22:10:22.702368 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:10:22.702314 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbfb8f404_9338_404f_81f9_beb0a5b26d8a.slice/crio-dcd745eb35ab2ec6c37bbc8d758909d5d3ab6ec6af0ed99f7edcd03d0adb1da0 WatchSource:0}: Error finding container dcd745eb35ab2ec6c37bbc8d758909d5d3ab6ec6af0ed99f7edcd03d0adb1da0: Status 404 returned error can't find the container with id dcd745eb35ab2ec6c37bbc8d758909d5d3ab6ec6af0ed99f7edcd03d0adb1da0 Jul 06 22:10:22.705820 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:22.705791 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-b6dcb"] Jul 06 22:10:22.712216 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:22.712196 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-b6dcb" Jul 06 22:10:22.714960 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:22.714924 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 06 22:10:22.715475 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:22.714946 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 06 22:10:22.716136 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:22.715781 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-g2mfw\"" Jul 06 22:10:22.716136 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:22.715810 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 06 22:10:22.716136 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:22.715975 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 06 22:10:22.736801 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:22.736668 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:10:22.822876 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:22.822833 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4fe91138-7f43-4d7a-b7a7-df08b1804376-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-b6dcb\" (UID: \"4fe91138-7f43-4d7a-b7a7-df08b1804376\") " pod="openshift-monitoring/node-exporter-b6dcb" Jul 06 22:10:22.822876 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:22.822880 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g6cnm\" (UniqueName: \"kubernetes.io/projected/4fe91138-7f43-4d7a-b7a7-df08b1804376-kube-api-access-g6cnm\") pod \"node-exporter-b6dcb\" (UID: \"4fe91138-7f43-4d7a-b7a7-df08b1804376\") " pod="openshift-monitoring/node-exporter-b6dcb" Jul 06 22:10:22.823105 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:22.822928 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4fe91138-7f43-4d7a-b7a7-df08b1804376-node-exporter-accelerators-collector-config\") pod \"node-exporter-b6dcb\" (UID: \"4fe91138-7f43-4d7a-b7a7-df08b1804376\") " pod="openshift-monitoring/node-exporter-b6dcb" Jul 06 22:10:22.823105 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:22.822957 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4fe91138-7f43-4d7a-b7a7-df08b1804376-node-exporter-wtmp\") pod \"node-exporter-b6dcb\" (UID: \"4fe91138-7f43-4d7a-b7a7-df08b1804376\") " pod="openshift-monitoring/node-exporter-b6dcb" Jul 06 22:10:22.823105 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:22.822985 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4fe91138-7f43-4d7a-b7a7-df08b1804376-metrics-client-ca\") pod \"node-exporter-b6dcb\" (UID: \"4fe91138-7f43-4d7a-b7a7-df08b1804376\") " pod="openshift-monitoring/node-exporter-b6dcb" Jul 06 22:10:22.823214 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:22.823133 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4fe91138-7f43-4d7a-b7a7-df08b1804376-node-exporter-tls\") pod \"node-exporter-b6dcb\" (UID: \"4fe91138-7f43-4d7a-b7a7-df08b1804376\") " pod="openshift-monitoring/node-exporter-b6dcb" Jul 06 22:10:22.823214 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:22.823194 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4fe91138-7f43-4d7a-b7a7-df08b1804376-root\") pod \"node-exporter-b6dcb\" (UID: \"4fe91138-7f43-4d7a-b7a7-df08b1804376\") " pod="openshift-monitoring/node-exporter-b6dcb" Jul 06 22:10:22.823292 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:22.823267 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4fe91138-7f43-4d7a-b7a7-df08b1804376-sys\") pod \"node-exporter-b6dcb\" (UID: \"4fe91138-7f43-4d7a-b7a7-df08b1804376\") " pod="openshift-monitoring/node-exporter-b6dcb" Jul 06 22:10:22.823336 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:22.823294 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4fe91138-7f43-4d7a-b7a7-df08b1804376-node-exporter-textfile\") pod \"node-exporter-b6dcb\" (UID: \"4fe91138-7f43-4d7a-b7a7-df08b1804376\") " pod="openshift-monitoring/node-exporter-b6dcb" Jul 06 22:10:22.827966 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:22.827929 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:10:22.923906 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:22.923863 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4fe91138-7f43-4d7a-b7a7-df08b1804376-node-exporter-tls\") pod \"node-exporter-b6dcb\" (UID: \"4fe91138-7f43-4d7a-b7a7-df08b1804376\") " pod="openshift-monitoring/node-exporter-b6dcb" Jul 06 22:10:22.924095 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:22.923940 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4fe91138-7f43-4d7a-b7a7-df08b1804376-root\") pod \"node-exporter-b6dcb\" (UID: \"4fe91138-7f43-4d7a-b7a7-df08b1804376\") " pod="openshift-monitoring/node-exporter-b6dcb" Jul 06 22:10:22.924095 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:22.923979 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4fe91138-7f43-4d7a-b7a7-df08b1804376-sys\") pod \"node-exporter-b6dcb\" (UID: \"4fe91138-7f43-4d7a-b7a7-df08b1804376\") " pod="openshift-monitoring/node-exporter-b6dcb" Jul 06 22:10:22.924095 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:22.924014 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4fe91138-7f43-4d7a-b7a7-df08b1804376-node-exporter-textfile\") pod \"node-exporter-b6dcb\" (UID: \"4fe91138-7f43-4d7a-b7a7-df08b1804376\") " pod="openshift-monitoring/node-exporter-b6dcb" Jul 06 22:10:22.924095 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:22.924086 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4fe91138-7f43-4d7a-b7a7-df08b1804376-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-b6dcb\" (UID: \"4fe91138-7f43-4d7a-b7a7-df08b1804376\") " pod="openshift-monitoring/node-exporter-b6dcb" Jul 06 22:10:22.924343 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:22.924113 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g6cnm\" (UniqueName: \"kubernetes.io/projected/4fe91138-7f43-4d7a-b7a7-df08b1804376-kube-api-access-g6cnm\") pod \"node-exporter-b6dcb\" (UID: \"4fe91138-7f43-4d7a-b7a7-df08b1804376\") " pod="openshift-monitoring/node-exporter-b6dcb" Jul 06 22:10:22.924343 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:22.924150 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4fe91138-7f43-4d7a-b7a7-df08b1804376-node-exporter-accelerators-collector-config\") pod \"node-exporter-b6dcb\" (UID: \"4fe91138-7f43-4d7a-b7a7-df08b1804376\") " pod="openshift-monitoring/node-exporter-b6dcb" Jul 06 22:10:22.924343 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:22.924179 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4fe91138-7f43-4d7a-b7a7-df08b1804376-node-exporter-wtmp\") pod \"node-exporter-b6dcb\" (UID: \"4fe91138-7f43-4d7a-b7a7-df08b1804376\") " pod="openshift-monitoring/node-exporter-b6dcb" Jul 06 22:10:22.924343 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:22.924208 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4fe91138-7f43-4d7a-b7a7-df08b1804376-metrics-client-ca\") pod \"node-exporter-b6dcb\" (UID: \"4fe91138-7f43-4d7a-b7a7-df08b1804376\") " pod="openshift-monitoring/node-exporter-b6dcb" Jul 06 22:10:22.924860 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:22.924837 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4fe91138-7f43-4d7a-b7a7-df08b1804376-metrics-client-ca\") pod \"node-exporter-b6dcb\" (UID: \"4fe91138-7f43-4d7a-b7a7-df08b1804376\") " pod="openshift-monitoring/node-exporter-b6dcb" Jul 06 22:10:22.924982 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:10:22.924958 2581 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 06 22:10:22.925051 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:10:22.925037 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4fe91138-7f43-4d7a-b7a7-df08b1804376-node-exporter-tls podName:4fe91138-7f43-4d7a-b7a7-df08b1804376 nodeName:}" failed. No retries permitted until 2026-07-06 22:10:23.425019292 +0000 UTC m=+102.983814591 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/4fe91138-7f43-4d7a-b7a7-df08b1804376-node-exporter-tls") pod "node-exporter-b6dcb" (UID: "4fe91138-7f43-4d7a-b7a7-df08b1804376") : secret "node-exporter-tls" not found Jul 06 22:10:22.925329 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:22.925309 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4fe91138-7f43-4d7a-b7a7-df08b1804376-root\") pod \"node-exporter-b6dcb\" (UID: \"4fe91138-7f43-4d7a-b7a7-df08b1804376\") " pod="openshift-monitoring/node-exporter-b6dcb" Jul 06 22:10:22.925395 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:22.925360 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4fe91138-7f43-4d7a-b7a7-df08b1804376-sys\") pod \"node-exporter-b6dcb\" (UID: \"4fe91138-7f43-4d7a-b7a7-df08b1804376\") " pod="openshift-monitoring/node-exporter-b6dcb" Jul 06 22:10:22.925618 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:22.925600 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4fe91138-7f43-4d7a-b7a7-df08b1804376-node-exporter-textfile\") pod \"node-exporter-b6dcb\" (UID: \"4fe91138-7f43-4d7a-b7a7-df08b1804376\") " pod="openshift-monitoring/node-exporter-b6dcb" Jul 06 22:10:22.926117 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:22.926086 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4fe91138-7f43-4d7a-b7a7-df08b1804376-node-exporter-wtmp\") pod \"node-exporter-b6dcb\" (UID: \"4fe91138-7f43-4d7a-b7a7-df08b1804376\") " pod="openshift-monitoring/node-exporter-b6dcb" Jul 06 22:10:22.926462 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:22.926391 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4fe91138-7f43-4d7a-b7a7-df08b1804376-node-exporter-accelerators-collector-config\") pod \"node-exporter-b6dcb\" (UID: \"4fe91138-7f43-4d7a-b7a7-df08b1804376\") " pod="openshift-monitoring/node-exporter-b6dcb" Jul 06 22:10:22.928233 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:22.928194 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4fe91138-7f43-4d7a-b7a7-df08b1804376-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-b6dcb\" (UID: \"4fe91138-7f43-4d7a-b7a7-df08b1804376\") " pod="openshift-monitoring/node-exporter-b6dcb" Jul 06 22:10:22.933983 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:22.933956 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g6cnm\" (UniqueName: \"kubernetes.io/projected/4fe91138-7f43-4d7a-b7a7-df08b1804376-kube-api-access-g6cnm\") pod \"node-exporter-b6dcb\" (UID: \"4fe91138-7f43-4d7a-b7a7-df08b1804376\") " pod="openshift-monitoring/node-exporter-b6dcb" Jul 06 22:10:22.960856 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:22.960818 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:10:23.058678 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:23.058648 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:10:23.197963 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:23.197851 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:10:23.301311 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:23.301279 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:10:23.403236 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:23.403201 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:10:23.429678 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:23.429643 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4fe91138-7f43-4d7a-b7a7-df08b1804376-node-exporter-tls\") pod \"node-exporter-b6dcb\" (UID: \"4fe91138-7f43-4d7a-b7a7-df08b1804376\") " pod="openshift-monitoring/node-exporter-b6dcb" Jul 06 22:10:23.432013 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:23.431983 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4fe91138-7f43-4d7a-b7a7-df08b1804376-node-exporter-tls\") pod \"node-exporter-b6dcb\" (UID: \"4fe91138-7f43-4d7a-b7a7-df08b1804376\") " pod="openshift-monitoring/node-exporter-b6dcb" Jul 06 22:10:23.499592 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:23.499507 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:10:23.651104 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:23.651059 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-b6dcb" Jul 06 22:10:23.663762 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:10:23.663736 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4fe91138_7f43_4d7a_b7a7_df08b1804376.slice/crio-db8628b203ae30a6e7132b2698d6b28e811525f11dea83a59777d2f4cd7899bb WatchSource:0}: Error finding container db8628b203ae30a6e7132b2698d6b28e811525f11dea83a59777d2f4cd7899bb: Status 404 returned error can't find the container with id db8628b203ae30a6e7132b2698d6b28e811525f11dea83a59777d2f4cd7899bb Jul 06 22:10:23.678352 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:23.678319 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-b6dcb" event={"ID":"4fe91138-7f43-4d7a-b7a7-df08b1804376","Type":"ContainerStarted","Data":"db8628b203ae30a6e7132b2698d6b28e811525f11dea83a59777d2f4cd7899bb"} Jul 06 22:10:23.680140 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:23.680112 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-65c85c5ff6-4fxss" event={"ID":"bfb8f404-9338-404f-81f9-beb0a5b26d8a","Type":"ContainerStarted","Data":"d97d6ade0c5fc87c21c8a5866c330dda71dc50e4487d584e25433c4e061365b2"} Jul 06 22:10:23.680237 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:23.680155 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-65c85c5ff6-4fxss" event={"ID":"bfb8f404-9338-404f-81f9-beb0a5b26d8a","Type":"ContainerStarted","Data":"dcd745eb35ab2ec6c37bbc8d758909d5d3ab6ec6af0ed99f7edcd03d0adb1da0"} Jul 06 22:10:23.680308 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:23.680262 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-65c85c5ff6-4fxss" Jul 06 22:10:23.681595 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:23.681559 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:10:23.706890 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:23.706836 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-65c85c5ff6-4fxss" podStartSLOduration=78.706816516 podStartE2EDuration="1m18.706816516s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:09:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:10:23.706159914 +0000 UTC m=+103.264955239" watchObservedRunningTime="2026-07-06 22:10:23.706816516 +0000 UTC m=+103.265611829" Jul 06 22:10:23.821388 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:23.821289 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:10:24.277361 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:24.277319 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:10:25.182422 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:25.182387 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:10:25.588505 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:25.588460 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:10:25.642679 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:25.642649 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:10:25.689547 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:25.689513 2581 generic.go:356] "Generic (PLEG): container finished" podID="4fe91138-7f43-4d7a-b7a7-df08b1804376" containerID="758a22880bac030314d90a2264aff8cee34e58f299e3906507e4372bae529cf7" exitCode=0 Jul 06 22:10:25.689721 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:25.689593 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-b6dcb" event={"ID":"4fe91138-7f43-4d7a-b7a7-df08b1804376","Type":"ContainerDied","Data":"758a22880bac030314d90a2264aff8cee34e58f299e3906507e4372bae529cf7"} Jul 06 22:10:25.846969 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:25.846879 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:10:25.949257 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:25.949226 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:10:26.695236 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:26.695197 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-b6dcb" event={"ID":"4fe91138-7f43-4d7a-b7a7-df08b1804376","Type":"ContainerStarted","Data":"5489fab4bb0a0fc2ee8b4db67a4650343a32e8270a8aef403e7301e551a01e14"} Jul 06 22:10:26.695615 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:26.695261 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-b6dcb" event={"ID":"4fe91138-7f43-4d7a-b7a7-df08b1804376","Type":"ContainerStarted","Data":"0385b61db8a59747c5e1fdb07ed79d0b812bd917c9adc5a31324521c23d43acc"} Jul 06 22:10:26.716617 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:26.716571 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-b6dcb" podStartSLOduration=3.208242179 podStartE2EDuration="4.716554999s" totalImagesPullingTime="1.407052103s" totalInitContainerRuntime="101.260717ms" isStatefulPod=true podCreationTimestamp="2026-07-06 22:10:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:10:26.715345133 +0000 UTC m=+106.274140466" watchObservedRunningTime="2026-07-06 22:10:26.716554999 +0000 UTC m=+106.275350319" Jul 06 22:10:27.156465 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:27.156383 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:10:27.256556 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:27.256512 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:10:27.468209 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:27.468165 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:10:27.591624 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:27.591584 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:10:27.686825 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:27.686789 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:10:28.937783 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:28.937747 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:10:29.091038 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:29.091007 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-65c85c5ff6-4fxss"] Jul 06 22:10:29.217319 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:29.217221 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:10:29.313143 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:29.313106 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:10:30.847546 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:30.847490 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:10:30.966441 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:30.966407 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:10:31.060472 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:31.060441 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:10:32.921987 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:32.921949 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:10:33.021584 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:33.021548 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:10:33.238026 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:33.237968 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:10:33.327260 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:33.327229 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:10:33.821047 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:33.821010 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:10:38.236370 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:38.236319 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:10:38.309195 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:38.309163 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:10:39.097015 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:39.096986 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-65c85c5ff6-4fxss" Jul 06 22:10:44.367861 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:44.367818 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:10:44.561552 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:44.561521 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:10:44.621450 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:44.621362 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:10:47.769941 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:47.769905 2581 generic.go:356] "Generic (PLEG): container finished" podID="fe3d4288-3a9d-413c-be44-ac70bd65b45f" containerID="740e1a39643fd71e51690103fd9decb887bde25f635e3db81f2a702487cfc2ac" exitCode=0 Jul 06 22:10:47.770384 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:47.769981 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-2pphn" event={"ID":"fe3d4288-3a9d-413c-be44-ac70bd65b45f","Type":"ContainerDied","Data":"740e1a39643fd71e51690103fd9decb887bde25f635e3db81f2a702487cfc2ac"} Jul 06 22:10:47.770437 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:47.770394 2581 scope.go:117] "RemoveContainer" containerID="740e1a39643fd71e51690103fd9decb887bde25f635e3db81f2a702487cfc2ac" Jul 06 22:10:47.771387 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:47.771369 2581 generic.go:356] "Generic (PLEG): container finished" podID="d3642934-8f79-4aed-bdfe-5a6777a655aa" containerID="9fe650b74821066a2251e6b72296ebe37b92ebffadb728707740a465cf81d3a5" exitCode=0 Jul 06 22:10:47.771454 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:47.771397 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-hfs2q" event={"ID":"d3642934-8f79-4aed-bdfe-5a6777a655aa","Type":"ContainerDied","Data":"9fe650b74821066a2251e6b72296ebe37b92ebffadb728707740a465cf81d3a5"} Jul 06 22:10:47.771681 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:47.771665 2581 scope.go:117] "RemoveContainer" containerID="9fe650b74821066a2251e6b72296ebe37b92ebffadb728707740a465cf81d3a5" Jul 06 22:10:48.777295 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:48.777260 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5c847dcfc4-2pphn" event={"ID":"fe3d4288-3a9d-413c-be44-ac70bd65b45f","Type":"ContainerStarted","Data":"55330a13c1a68b8c8b793ceef7e9f83019119994ada4221df4819209fbc56622"} Jul 06 22:10:48.779197 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:48.779172 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-846bf668f7-hfs2q" event={"ID":"d3642934-8f79-4aed-bdfe-5a6777a655aa","Type":"ContainerStarted","Data":"4f3935ad1dfee4991fd03c63965b6429874362815c01f802d7e637637dee14c8"} Jul 06 22:10:50.768165 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:50.768125 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8a83eb6f-f275-4562-b014-0b48c710b624-metrics-certs\") pod \"network-metrics-daemon-n9hgj\" (UID: \"8a83eb6f-f275-4562-b014-0b48c710b624\") " pod="openshift-multus/network-metrics-daemon-n9hgj" Jul 06 22:10:50.770437 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:50.770414 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8a83eb6f-f275-4562-b014-0b48c710b624-metrics-certs\") pod \"network-metrics-daemon-n9hgj\" (UID: \"8a83eb6f-f275-4562-b014-0b48c710b624\") " pod="openshift-multus/network-metrics-daemon-n9hgj" Jul 06 22:10:50.793626 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:50.793601 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-dftmz\"" Jul 06 22:10:50.801772 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:50.801752 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-n9hgj" Jul 06 22:10:50.935514 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:50.935489 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-n9hgj"] Jul 06 22:10:50.937960 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:10:50.937929 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a83eb6f_f275_4562_b014_0b48c710b624.slice/crio-082b0ea0ea44e3b522cd9d389701b92ab39276a27e47f40a94c5af2d8718f357 WatchSource:0}: Error finding container 082b0ea0ea44e3b522cd9d389701b92ab39276a27e47f40a94c5af2d8718f357: Status 404 returned error can't find the container with id 082b0ea0ea44e3b522cd9d389701b92ab39276a27e47f40a94c5af2d8718f357 Jul 06 22:10:51.791723 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:51.791688 2581 generic.go:356] "Generic (PLEG): container finished" podID="01137b22-9e43-47ba-aee5-e4c51fd35e4d" containerID="a9cffbd956375580bb03a7f9172ebfcfb59dc4e10d62486d7d0ec361e58f3212" exitCode=0 Jul 06 22:10:51.792198 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:51.791756 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-dj4ks" event={"ID":"01137b22-9e43-47ba-aee5-e4c51fd35e4d","Type":"ContainerDied","Data":"a9cffbd956375580bb03a7f9172ebfcfb59dc4e10d62486d7d0ec361e58f3212"} Jul 06 22:10:51.792198 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:51.792154 2581 scope.go:117] "RemoveContainer" containerID="a9cffbd956375580bb03a7f9172ebfcfb59dc4e10d62486d7d0ec361e58f3212" Jul 06 22:10:51.793008 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:51.792984 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-n9hgj" event={"ID":"8a83eb6f-f275-4562-b014-0b48c710b624","Type":"ContainerStarted","Data":"082b0ea0ea44e3b522cd9d389701b92ab39276a27e47f40a94c5af2d8718f357"} Jul 06 22:10:52.798133 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:52.798095 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-n9hgj" event={"ID":"8a83eb6f-f275-4562-b014-0b48c710b624","Type":"ContainerStarted","Data":"7cc653b284643b28bce26ce90f465dda8955d3b9f5a7cc5edb400da76b0392c8"} Jul 06 22:10:52.798610 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:52.798175 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-n9hgj" event={"ID":"8a83eb6f-f275-4562-b014-0b48c710b624","Type":"ContainerStarted","Data":"5232a478f24a21393e91a28479424fe055a7c00998c81b3b0f8a07fccde049be"} Jul 06 22:10:52.799779 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:52.799747 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d46d9ffd4-dj4ks" event={"ID":"01137b22-9e43-47ba-aee5-e4c51fd35e4d","Type":"ContainerStarted","Data":"2238b6da5029d62923621edb0d147083e99b7d5df3356f72893bbf6868c0e2d5"} Jul 06 22:10:52.832964 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:52.832918 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-n9hgj" podStartSLOduration=130.286572064 podStartE2EDuration="2m11.832903655s" totalImagesPullingTime="1.546331591s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:08:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:10:52.818287693 +0000 UTC m=+132.377083017" watchObservedRunningTime="2026-07-06 22:10:52.832903655 +0000 UTC m=+132.391698975" Jul 06 22:10:52.931839 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:52.931786 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:10:53.032902 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:53.032813 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:10:53.138418 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:53.138388 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:10:53.203076 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:53.203026 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:10:54.111461 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:54.111412 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-65c85c5ff6-4fxss" podUID="bfb8f404-9338-404f-81f9-beb0a5b26d8a" containerName="registry" containerID="cri-o://d97d6ade0c5fc87c21c8a5866c330dda71dc50e4487d584e25433c4e061365b2" gracePeriod=30 Jul 06 22:10:54.358693 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:54.358666 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-65c85c5ff6-4fxss" Jul 06 22:10:54.501881 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:54.501848 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/bfb8f404-9338-404f-81f9-beb0a5b26d8a-ca-trust-extracted\") pod \"bfb8f404-9338-404f-81f9-beb0a5b26d8a\" (UID: \"bfb8f404-9338-404f-81f9-beb0a5b26d8a\") " Jul 06 22:10:54.502037 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:54.501929 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/bfb8f404-9338-404f-81f9-beb0a5b26d8a-installation-pull-secrets\") pod \"bfb8f404-9338-404f-81f9-beb0a5b26d8a\" (UID: \"bfb8f404-9338-404f-81f9-beb0a5b26d8a\") " Jul 06 22:10:54.502037 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:54.501953 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/bfb8f404-9338-404f-81f9-beb0a5b26d8a-registry-certificates\") pod \"bfb8f404-9338-404f-81f9-beb0a5b26d8a\" (UID: \"bfb8f404-9338-404f-81f9-beb0a5b26d8a\") " Jul 06 22:10:54.502037 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:54.501971 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/bfb8f404-9338-404f-81f9-beb0a5b26d8a-image-registry-private-configuration\") pod \"bfb8f404-9338-404f-81f9-beb0a5b26d8a\" (UID: \"bfb8f404-9338-404f-81f9-beb0a5b26d8a\") " Jul 06 22:10:54.502037 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:54.501990 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bfb8f404-9338-404f-81f9-beb0a5b26d8a-registry-tls\") pod \"bfb8f404-9338-404f-81f9-beb0a5b26d8a\" (UID: \"bfb8f404-9338-404f-81f9-beb0a5b26d8a\") " Jul 06 22:10:54.502037 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:54.502009 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bfb8f404-9338-404f-81f9-beb0a5b26d8a-trusted-ca\") pod \"bfb8f404-9338-404f-81f9-beb0a5b26d8a\" (UID: \"bfb8f404-9338-404f-81f9-beb0a5b26d8a\") " Jul 06 22:10:54.502037 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:54.502035 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bfb8f404-9338-404f-81f9-beb0a5b26d8a-bound-sa-token\") pod \"bfb8f404-9338-404f-81f9-beb0a5b26d8a\" (UID: \"bfb8f404-9338-404f-81f9-beb0a5b26d8a\") " Jul 06 22:10:54.502379 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:54.502092 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d5gvp\" (UniqueName: \"kubernetes.io/projected/bfb8f404-9338-404f-81f9-beb0a5b26d8a-kube-api-access-d5gvp\") pod \"bfb8f404-9338-404f-81f9-beb0a5b26d8a\" (UID: \"bfb8f404-9338-404f-81f9-beb0a5b26d8a\") " Jul 06 22:10:54.502694 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:54.502429 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bfb8f404-9338-404f-81f9-beb0a5b26d8a-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "bfb8f404-9338-404f-81f9-beb0a5b26d8a" (UID: "bfb8f404-9338-404f-81f9-beb0a5b26d8a"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 22:10:54.502786 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:54.502759 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bfb8f404-9338-404f-81f9-beb0a5b26d8a-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "bfb8f404-9338-404f-81f9-beb0a5b26d8a" (UID: "bfb8f404-9338-404f-81f9-beb0a5b26d8a"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 22:10:54.504488 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:54.504449 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bfb8f404-9338-404f-81f9-beb0a5b26d8a-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "bfb8f404-9338-404f-81f9-beb0a5b26d8a" (UID: "bfb8f404-9338-404f-81f9-beb0a5b26d8a"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 22:10:54.504716 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:54.504685 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bfb8f404-9338-404f-81f9-beb0a5b26d8a-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "bfb8f404-9338-404f-81f9-beb0a5b26d8a" (UID: "bfb8f404-9338-404f-81f9-beb0a5b26d8a"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 22:10:54.504820 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:54.504794 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bfb8f404-9338-404f-81f9-beb0a5b26d8a-kube-api-access-d5gvp" (OuterVolumeSpecName: "kube-api-access-d5gvp") pod "bfb8f404-9338-404f-81f9-beb0a5b26d8a" (UID: "bfb8f404-9338-404f-81f9-beb0a5b26d8a"). InnerVolumeSpecName "kube-api-access-d5gvp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 22:10:54.504891 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:54.504844 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bfb8f404-9338-404f-81f9-beb0a5b26d8a-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "bfb8f404-9338-404f-81f9-beb0a5b26d8a" (UID: "bfb8f404-9338-404f-81f9-beb0a5b26d8a"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 22:10:54.504932 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:54.504895 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bfb8f404-9338-404f-81f9-beb0a5b26d8a-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "bfb8f404-9338-404f-81f9-beb0a5b26d8a" (UID: "bfb8f404-9338-404f-81f9-beb0a5b26d8a"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 22:10:54.510728 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:54.510703 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bfb8f404-9338-404f-81f9-beb0a5b26d8a-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "bfb8f404-9338-404f-81f9-beb0a5b26d8a" (UID: "bfb8f404-9338-404f-81f9-beb0a5b26d8a"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:10:54.603103 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:54.603045 2581 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/bfb8f404-9338-404f-81f9-beb0a5b26d8a-installation-pull-secrets\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:10:54.603103 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:54.603097 2581 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/bfb8f404-9338-404f-81f9-beb0a5b26d8a-registry-certificates\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:10:54.603103 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:54.603107 2581 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/bfb8f404-9338-404f-81f9-beb0a5b26d8a-image-registry-private-configuration\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:10:54.603103 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:54.603117 2581 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bfb8f404-9338-404f-81f9-beb0a5b26d8a-registry-tls\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:10:54.603366 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:54.603126 2581 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bfb8f404-9338-404f-81f9-beb0a5b26d8a-trusted-ca\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:10:54.603366 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:54.603134 2581 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bfb8f404-9338-404f-81f9-beb0a5b26d8a-bound-sa-token\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:10:54.603366 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:54.603142 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d5gvp\" (UniqueName: \"kubernetes.io/projected/bfb8f404-9338-404f-81f9-beb0a5b26d8a-kube-api-access-d5gvp\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:10:54.603366 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:54.603151 2581 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/bfb8f404-9338-404f-81f9-beb0a5b26d8a-ca-trust-extracted\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:10:54.809357 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:54.809268 2581 generic.go:356] "Generic (PLEG): container finished" podID="bfb8f404-9338-404f-81f9-beb0a5b26d8a" containerID="d97d6ade0c5fc87c21c8a5866c330dda71dc50e4487d584e25433c4e061365b2" exitCode=0 Jul 06 22:10:54.809357 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:54.809313 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-65c85c5ff6-4fxss" Jul 06 22:10:54.809575 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:54.809366 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-65c85c5ff6-4fxss" event={"ID":"bfb8f404-9338-404f-81f9-beb0a5b26d8a","Type":"ContainerDied","Data":"d97d6ade0c5fc87c21c8a5866c330dda71dc50e4487d584e25433c4e061365b2"} Jul 06 22:10:54.809575 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:54.809420 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-65c85c5ff6-4fxss" event={"ID":"bfb8f404-9338-404f-81f9-beb0a5b26d8a","Type":"ContainerDied","Data":"dcd745eb35ab2ec6c37bbc8d758909d5d3ab6ec6af0ed99f7edcd03d0adb1da0"} Jul 06 22:10:54.809575 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:54.809446 2581 scope.go:117] "RemoveContainer" containerID="d97d6ade0c5fc87c21c8a5866c330dda71dc50e4487d584e25433c4e061365b2" Jul 06 22:10:54.818560 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:54.818545 2581 scope.go:117] "RemoveContainer" containerID="d97d6ade0c5fc87c21c8a5866c330dda71dc50e4487d584e25433c4e061365b2" Jul 06 22:10:54.818821 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:10:54.818803 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d97d6ade0c5fc87c21c8a5866c330dda71dc50e4487d584e25433c4e061365b2\": container with ID starting with d97d6ade0c5fc87c21c8a5866c330dda71dc50e4487d584e25433c4e061365b2 not found: ID does not exist" containerID="d97d6ade0c5fc87c21c8a5866c330dda71dc50e4487d584e25433c4e061365b2" Jul 06 22:10:54.818868 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:54.818829 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d97d6ade0c5fc87c21c8a5866c330dda71dc50e4487d584e25433c4e061365b2"} err="failed to get container status \"d97d6ade0c5fc87c21c8a5866c330dda71dc50e4487d584e25433c4e061365b2\": rpc error: code = NotFound desc = could not find container \"d97d6ade0c5fc87c21c8a5866c330dda71dc50e4487d584e25433c4e061365b2\": container with ID starting with d97d6ade0c5fc87c21c8a5866c330dda71dc50e4487d584e25433c4e061365b2 not found: ID does not exist" Jul 06 22:10:54.829631 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:54.829610 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-65c85c5ff6-4fxss"] Jul 06 22:10:54.835794 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:54.835777 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-65c85c5ff6-4fxss"] Jul 06 22:10:54.974871 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:54.974834 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bfb8f404-9338-404f-81f9-beb0a5b26d8a" path="/var/lib/kubelet/pods/bfb8f404-9338-404f-81f9-beb0a5b26d8a/volumes" Jul 06 22:10:55.336350 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:10:55.336319 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:11:25.817967 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:11:25.817923 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:11:38.788061 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:11:38.787976 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:11:38.846786 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:11:38.846748 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:11:50.608079 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:11:50.608017 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:11:50.674534 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:11:50.674496 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:12:02.746163 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:12:02.746124 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:12:03.071998 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:12:03.071908 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:12:03.174077 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:12:03.174030 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:12:03.855612 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:12:03.855574 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:12:03.912680 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:12:03.912642 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:12:08.146042 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:12:08.145997 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:12:08.246424 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:12:08.246381 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:12:13.649617 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:12:13.649575 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:12:13.751257 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:12:13.751223 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:12:13.809930 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:12:13.809879 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:12:16.767704 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:12:16.767667 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:12:18.905184 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:12:18.905134 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:12:19.004703 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:12:19.004666 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:12:51.776981 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:12:51.776940 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:12:51.875143 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:12:51.875106 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:12:55.604351 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:12:55.604317 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:13:23.550522 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:13:23.550478 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:13:24.430907 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:13:24.430874 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:13:27.162970 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:13:27.162931 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:13:27.266815 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:13:27.266779 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:13:28.233492 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:13:28.233461 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:13:28.390459 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:13:28.390416 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:13:28.558614 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:13:28.558519 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:13:28.758964 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:13:28.758920 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:13:28.932594 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:13:28.932559 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:13:31.108293 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:13:31.108254 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:13:31.490234 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:13:31.490196 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:13:33.946615 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:13:33.946578 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:13:34.012274 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:13:34.012218 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:13:35.912955 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:13:35.912919 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:13:36.028417 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:13:36.028360 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:13:38.703044 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:13:38.703013 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:13:40.865411 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:13:40.865383 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-rcn5q_86b07166-2f34-443b-9c61-86b7d950ad7b/console-operator/1.log" Jul 06 22:13:40.870904 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:13:40.870872 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-rcn5q_86b07166-2f34-443b-9c61-86b7d950ad7b/console-operator/1.log" Jul 06 22:13:40.876418 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:13:40.876393 2581 kubelet.go:1648] "Image garbage collection succeeded" Jul 06 22:13:51.459678 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:13:51.459645 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:13:51.618774 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:13:51.618741 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:14:03.062314 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:03.062267 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:14:05.679926 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:05.679891 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:14:06.984863 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:06.984829 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:14:07.792135 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:07.792100 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:14:17.980346 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:17.980308 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:14:18.063623 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:18.063594 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:14:22.457223 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:22.457186 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:14:22.550929 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:22.550893 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:14:23.031325 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:23.031290 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:14:23.276506 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:23.276472 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:14:23.421439 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:23.421352 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:14:23.426306 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:23.426284 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5c695b4674-zcn8n"] Jul 06 22:14:23.426626 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:23.426611 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bfb8f404-9338-404f-81f9-beb0a5b26d8a" containerName="registry" Jul 06 22:14:23.426697 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:23.426630 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="bfb8f404-9338-404f-81f9-beb0a5b26d8a" containerName="registry" Jul 06 22:14:23.426750 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:23.426720 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="bfb8f404-9338-404f-81f9-beb0a5b26d8a" containerName="registry" Jul 06 22:14:23.429555 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:23.429538 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5c695b4674-zcn8n" Jul 06 22:14:23.432390 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:23.432372 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-54jfp\"" Jul 06 22:14:23.432982 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:23.432960 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 06 22:14:23.433114 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:23.433088 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 06 22:14:23.433199 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:23.433182 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 06 22:14:23.433451 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:23.433435 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 06 22:14:23.450081 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:23.450048 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5c695b4674-zcn8n"] Jul 06 22:14:23.542126 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:23.542090 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/48cb8eea-ec98-410c-9b13-84352620f7cd-webhook-cert\") pod \"opendatahub-operator-controller-manager-5c695b4674-zcn8n\" (UID: \"48cb8eea-ec98-410c-9b13-84352620f7cd\") " pod="opendatahub/opendatahub-operator-controller-manager-5c695b4674-zcn8n" Jul 06 22:14:23.542478 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:23.542136 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gnmj2\" (UniqueName: \"kubernetes.io/projected/48cb8eea-ec98-410c-9b13-84352620f7cd-kube-api-access-gnmj2\") pod \"opendatahub-operator-controller-manager-5c695b4674-zcn8n\" (UID: \"48cb8eea-ec98-410c-9b13-84352620f7cd\") " pod="opendatahub/opendatahub-operator-controller-manager-5c695b4674-zcn8n" Jul 06 22:14:23.542478 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:23.542165 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/48cb8eea-ec98-410c-9b13-84352620f7cd-manifests\") pod \"opendatahub-operator-controller-manager-5c695b4674-zcn8n\" (UID: \"48cb8eea-ec98-410c-9b13-84352620f7cd\") " pod="opendatahub/opendatahub-operator-controller-manager-5c695b4674-zcn8n" Jul 06 22:14:23.542478 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:23.542280 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/48cb8eea-ec98-410c-9b13-84352620f7cd-tmp\") pod \"opendatahub-operator-controller-manager-5c695b4674-zcn8n\" (UID: \"48cb8eea-ec98-410c-9b13-84352620f7cd\") " pod="opendatahub/opendatahub-operator-controller-manager-5c695b4674-zcn8n" Jul 06 22:14:23.542478 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:23.542315 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/48cb8eea-ec98-410c-9b13-84352620f7cd-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5c695b4674-zcn8n\" (UID: \"48cb8eea-ec98-410c-9b13-84352620f7cd\") " pod="opendatahub/opendatahub-operator-controller-manager-5c695b4674-zcn8n" Jul 06 22:14:23.575713 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:23.575678 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:14:23.642714 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:23.642686 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/48cb8eea-ec98-410c-9b13-84352620f7cd-webhook-cert\") pod \"opendatahub-operator-controller-manager-5c695b4674-zcn8n\" (UID: \"48cb8eea-ec98-410c-9b13-84352620f7cd\") " pod="opendatahub/opendatahub-operator-controller-manager-5c695b4674-zcn8n" Jul 06 22:14:23.642828 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:23.642722 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gnmj2\" (UniqueName: \"kubernetes.io/projected/48cb8eea-ec98-410c-9b13-84352620f7cd-kube-api-access-gnmj2\") pod \"opendatahub-operator-controller-manager-5c695b4674-zcn8n\" (UID: \"48cb8eea-ec98-410c-9b13-84352620f7cd\") " pod="opendatahub/opendatahub-operator-controller-manager-5c695b4674-zcn8n" Jul 06 22:14:23.642828 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:23.642750 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/48cb8eea-ec98-410c-9b13-84352620f7cd-manifests\") pod \"opendatahub-operator-controller-manager-5c695b4674-zcn8n\" (UID: \"48cb8eea-ec98-410c-9b13-84352620f7cd\") " pod="opendatahub/opendatahub-operator-controller-manager-5c695b4674-zcn8n" Jul 06 22:14:23.642828 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:23.642787 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/48cb8eea-ec98-410c-9b13-84352620f7cd-tmp\") pod \"opendatahub-operator-controller-manager-5c695b4674-zcn8n\" (UID: \"48cb8eea-ec98-410c-9b13-84352620f7cd\") " pod="opendatahub/opendatahub-operator-controller-manager-5c695b4674-zcn8n" Jul 06 22:14:23.642828 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:23.642806 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/48cb8eea-ec98-410c-9b13-84352620f7cd-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5c695b4674-zcn8n\" (UID: \"48cb8eea-ec98-410c-9b13-84352620f7cd\") " pod="opendatahub/opendatahub-operator-controller-manager-5c695b4674-zcn8n" Jul 06 22:14:23.643368 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:23.643343 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/48cb8eea-ec98-410c-9b13-84352620f7cd-manifests\") pod \"opendatahub-operator-controller-manager-5c695b4674-zcn8n\" (UID: \"48cb8eea-ec98-410c-9b13-84352620f7cd\") " pod="opendatahub/opendatahub-operator-controller-manager-5c695b4674-zcn8n" Jul 06 22:14:23.645207 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:23.645181 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/48cb8eea-ec98-410c-9b13-84352620f7cd-tmp\") pod \"opendatahub-operator-controller-manager-5c695b4674-zcn8n\" (UID: \"48cb8eea-ec98-410c-9b13-84352620f7cd\") " pod="opendatahub/opendatahub-operator-controller-manager-5c695b4674-zcn8n" Jul 06 22:14:23.645335 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:23.645319 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/48cb8eea-ec98-410c-9b13-84352620f7cd-webhook-cert\") pod \"opendatahub-operator-controller-manager-5c695b4674-zcn8n\" (UID: \"48cb8eea-ec98-410c-9b13-84352620f7cd\") " pod="opendatahub/opendatahub-operator-controller-manager-5c695b4674-zcn8n" Jul 06 22:14:23.645380 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:23.645354 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/48cb8eea-ec98-410c-9b13-84352620f7cd-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5c695b4674-zcn8n\" (UID: \"48cb8eea-ec98-410c-9b13-84352620f7cd\") " pod="opendatahub/opendatahub-operator-controller-manager-5c695b4674-zcn8n" Jul 06 22:14:23.651789 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:23.651769 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gnmj2\" (UniqueName: \"kubernetes.io/projected/48cb8eea-ec98-410c-9b13-84352620f7cd-kube-api-access-gnmj2\") pod \"opendatahub-operator-controller-manager-5c695b4674-zcn8n\" (UID: \"48cb8eea-ec98-410c-9b13-84352620f7cd\") " pod="opendatahub/opendatahub-operator-controller-manager-5c695b4674-zcn8n" Jul 06 22:14:23.677044 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:23.677014 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:14:23.739957 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:23.739919 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5c695b4674-zcn8n" Jul 06 22:14:23.776461 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:23.776432 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:14:23.881336 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:23.881310 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5c695b4674-zcn8n"] Jul 06 22:14:23.883858 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:14:23.883818 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48cb8eea_ec98_410c_9b13_84352620f7cd.slice/crio-ad9f10e5393a47c1c3c6eba6c43c021ebe881cd1749b6a647b15a88255323dfd WatchSource:0}: Error finding container ad9f10e5393a47c1c3c6eba6c43c021ebe881cd1749b6a647b15a88255323dfd: Status 404 returned error can't find the container with id ad9f10e5393a47c1c3c6eba6c43c021ebe881cd1749b6a647b15a88255323dfd Jul 06 22:14:23.885386 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:23.885358 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:14:23.885761 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:23.885746 2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 22:14:23.991173 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:23.991083 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:14:24.570907 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:24.570862 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5c695b4674-zcn8n" event={"ID":"48cb8eea-ec98-410c-9b13-84352620f7cd","Type":"ContainerStarted","Data":"ad9f10e5393a47c1c3c6eba6c43c021ebe881cd1749b6a647b15a88255323dfd"} Jul 06 22:14:27.588786 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:27.588752 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5c695b4674-zcn8n" event={"ID":"48cb8eea-ec98-410c-9b13-84352620f7cd","Type":"ContainerStarted","Data":"96f756fa6e2a1b4ef5fae08189e7c2e1b05ce39e84c8fe6c52247f191a2141ec"} Jul 06 22:14:28.593481 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:28.593444 2581 generic.go:356] "Generic (PLEG): container finished" podID="48cb8eea-ec98-410c-9b13-84352620f7cd" containerID="96f756fa6e2a1b4ef5fae08189e7c2e1b05ce39e84c8fe6c52247f191a2141ec" exitCode=0 Jul 06 22:14:28.593915 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:28.593527 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5c695b4674-zcn8n" event={"ID":"48cb8eea-ec98-410c-9b13-84352620f7cd","Type":"ContainerDied","Data":"96f756fa6e2a1b4ef5fae08189e7c2e1b05ce39e84c8fe6c52247f191a2141ec"} Jul 06 22:14:29.598716 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:29.598681 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5c695b4674-zcn8n" event={"ID":"48cb8eea-ec98-410c-9b13-84352620f7cd","Type":"ContainerStarted","Data":"37b6a2f249f1d1782e0704e3dc30a1517ed00e3bd02090eacf63fa3caa9091f4"} Jul 06 22:14:29.599105 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:29.598794 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-5c695b4674-zcn8n" Jul 06 22:14:29.623437 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:29.623389 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-5c695b4674-zcn8n" podStartSLOduration=2.283741137 podStartE2EDuration="6.623373583s" totalImagesPullingTime="3.298962685s" totalInitContainerRuntime="1.040669761s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:14:23 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:14:29.622146572 +0000 UTC m=+349.180941911" watchObservedRunningTime="2026-07-06 22:14:29.623373583 +0000 UTC m=+349.182168903" Jul 06 22:14:40.605585 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:40.605507 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-5c695b4674-zcn8n" Jul 06 22:14:42.517177 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:42.517139 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:14:42.719673 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:42.719628 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:14:42.979402 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:42.979372 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:14:43.135249 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:43.135214 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-658774566b-59kk5"] Jul 06 22:14:43.144848 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:43.144825 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-658774566b-59kk5" Jul 06 22:14:43.148059 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:43.148034 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 06 22:14:43.148250 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:43.148233 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-rcbc4\"" Jul 06 22:14:43.148467 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:43.148192 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 06 22:14:43.149826 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:43.149804 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-658774566b-59kk5"] Jul 06 22:14:43.178482 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:43.178454 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9g7f5\" (UniqueName: \"kubernetes.io/projected/70ec6221-d9c4-449b-b469-8725e711cae5-kube-api-access-9g7f5\") pod \"kube-auth-proxy-658774566b-59kk5\" (UID: \"70ec6221-d9c4-449b-b469-8725e711cae5\") " pod="openshift-ingress/kube-auth-proxy-658774566b-59kk5" Jul 06 22:14:43.178616 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:43.178506 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/70ec6221-d9c4-449b-b469-8725e711cae5-tmp\") pod \"kube-auth-proxy-658774566b-59kk5\" (UID: \"70ec6221-d9c4-449b-b469-8725e711cae5\") " pod="openshift-ingress/kube-auth-proxy-658774566b-59kk5" Jul 06 22:14:43.178616 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:43.178591 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/70ec6221-d9c4-449b-b469-8725e711cae5-tls-certs\") pod \"kube-auth-proxy-658774566b-59kk5\" (UID: \"70ec6221-d9c4-449b-b469-8725e711cae5\") " pod="openshift-ingress/kube-auth-proxy-658774566b-59kk5" Jul 06 22:14:43.193668 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:43.193609 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:14:43.279822 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:43.279734 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/70ec6221-d9c4-449b-b469-8725e711cae5-tmp\") pod \"kube-auth-proxy-658774566b-59kk5\" (UID: \"70ec6221-d9c4-449b-b469-8725e711cae5\") " pod="openshift-ingress/kube-auth-proxy-658774566b-59kk5" Jul 06 22:14:43.279822 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:43.279778 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/70ec6221-d9c4-449b-b469-8725e711cae5-tls-certs\") pod \"kube-auth-proxy-658774566b-59kk5\" (UID: \"70ec6221-d9c4-449b-b469-8725e711cae5\") " pod="openshift-ingress/kube-auth-proxy-658774566b-59kk5" Jul 06 22:14:43.279822 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:43.279819 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9g7f5\" (UniqueName: \"kubernetes.io/projected/70ec6221-d9c4-449b-b469-8725e711cae5-kube-api-access-9g7f5\") pod \"kube-auth-proxy-658774566b-59kk5\" (UID: \"70ec6221-d9c4-449b-b469-8725e711cae5\") " pod="openshift-ingress/kube-auth-proxy-658774566b-59kk5" Jul 06 22:14:43.281973 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:43.281943 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/70ec6221-d9c4-449b-b469-8725e711cae5-tmp\") pod \"kube-auth-proxy-658774566b-59kk5\" (UID: \"70ec6221-d9c4-449b-b469-8725e711cae5\") " pod="openshift-ingress/kube-auth-proxy-658774566b-59kk5" Jul 06 22:14:43.282298 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:43.282277 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/70ec6221-d9c4-449b-b469-8725e711cae5-tls-certs\") pod \"kube-auth-proxy-658774566b-59kk5\" (UID: \"70ec6221-d9c4-449b-b469-8725e711cae5\") " pod="openshift-ingress/kube-auth-proxy-658774566b-59kk5" Jul 06 22:14:43.299887 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:43.299858 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9g7f5\" (UniqueName: \"kubernetes.io/projected/70ec6221-d9c4-449b-b469-8725e711cae5-kube-api-access-9g7f5\") pod \"kube-auth-proxy-658774566b-59kk5\" (UID: \"70ec6221-d9c4-449b-b469-8725e711cae5\") " pod="openshift-ingress/kube-auth-proxy-658774566b-59kk5" Jul 06 22:14:43.437451 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:43.437417 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-2xpwc"] Jul 06 22:14:43.440804 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:43.440782 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-2xpwc" Jul 06 22:14:43.444043 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:43.444022 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 06 22:14:43.444214 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:43.444193 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-cwbj4\"" Jul 06 22:14:43.455620 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:43.455601 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-658774566b-59kk5" Jul 06 22:14:43.463263 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:43.463234 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-2xpwc"] Jul 06 22:14:43.481981 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:43.481956 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/98c7acc6-8ae3-470e-b51b-8be2db428a7d-cert\") pod \"odh-model-controller-7998c67dc4-2xpwc\" (UID: \"98c7acc6-8ae3-470e-b51b-8be2db428a7d\") " pod="opendatahub/odh-model-controller-7998c67dc4-2xpwc" Jul 06 22:14:43.482134 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:43.482006 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v8bjb\" (UniqueName: \"kubernetes.io/projected/98c7acc6-8ae3-470e-b51b-8be2db428a7d-kube-api-access-v8bjb\") pod \"odh-model-controller-7998c67dc4-2xpwc\" (UID: \"98c7acc6-8ae3-470e-b51b-8be2db428a7d\") " pod="opendatahub/odh-model-controller-7998c67dc4-2xpwc" Jul 06 22:14:43.534052 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:43.533946 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:14:43.582514 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:43.582481 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/98c7acc6-8ae3-470e-b51b-8be2db428a7d-cert\") pod \"odh-model-controller-7998c67dc4-2xpwc\" (UID: \"98c7acc6-8ae3-470e-b51b-8be2db428a7d\") " pod="opendatahub/odh-model-controller-7998c67dc4-2xpwc" Jul 06 22:14:43.582650 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:43.582548 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v8bjb\" (UniqueName: \"kubernetes.io/projected/98c7acc6-8ae3-470e-b51b-8be2db428a7d-kube-api-access-v8bjb\") pod \"odh-model-controller-7998c67dc4-2xpwc\" (UID: \"98c7acc6-8ae3-470e-b51b-8be2db428a7d\") " pod="opendatahub/odh-model-controller-7998c67dc4-2xpwc" Jul 06 22:14:43.582650 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:14:43.582587 2581 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jul 06 22:14:43.582723 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:14:43.582658 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/98c7acc6-8ae3-470e-b51b-8be2db428a7d-cert podName:98c7acc6-8ae3-470e-b51b-8be2db428a7d nodeName:}" failed. No retries permitted until 2026-07-06 22:14:44.082636611 +0000 UTC m=+363.641431909 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/98c7acc6-8ae3-470e-b51b-8be2db428a7d-cert") pod "odh-model-controller-7998c67dc4-2xpwc" (UID: "98c7acc6-8ae3-470e-b51b-8be2db428a7d") : secret "odh-model-controller-webhook-cert" not found Jul 06 22:14:43.592823 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:43.592797 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-658774566b-59kk5"] Jul 06 22:14:43.595226 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:43.595209 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v8bjb\" (UniqueName: \"kubernetes.io/projected/98c7acc6-8ae3-470e-b51b-8be2db428a7d-kube-api-access-v8bjb\") pod \"odh-model-controller-7998c67dc4-2xpwc\" (UID: \"98c7acc6-8ae3-470e-b51b-8be2db428a7d\") " pod="opendatahub/odh-model-controller-7998c67dc4-2xpwc" Jul 06 22:14:43.596679 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:14:43.596654 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70ec6221_d9c4_449b_b469_8725e711cae5.slice/crio-824b67842af48744f62ddb3f7c8cb8b0cb380f37b1a94f5bc444787b0f3b7f32 WatchSource:0}: Error finding container 824b67842af48744f62ddb3f7c8cb8b0cb380f37b1a94f5bc444787b0f3b7f32: Status 404 returned error can't find the container with id 824b67842af48744f62ddb3f7c8cb8b0cb380f37b1a94f5bc444787b0f3b7f32 Jul 06 22:14:43.637721 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:43.637692 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:14:43.651636 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:43.651607 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-658774566b-59kk5" event={"ID":"70ec6221-d9c4-449b-b469-8725e711cae5","Type":"ContainerStarted","Data":"824b67842af48744f62ddb3f7c8cb8b0cb380f37b1a94f5bc444787b0f3b7f32"} Jul 06 22:14:43.768343 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:43.768311 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:14:43.870331 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:43.870239 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:14:44.086184 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:44.086147 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/98c7acc6-8ae3-470e-b51b-8be2db428a7d-cert\") pod \"odh-model-controller-7998c67dc4-2xpwc\" (UID: \"98c7acc6-8ae3-470e-b51b-8be2db428a7d\") " pod="opendatahub/odh-model-controller-7998c67dc4-2xpwc" Jul 06 22:14:44.088492 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:44.088464 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/98c7acc6-8ae3-470e-b51b-8be2db428a7d-cert\") pod \"odh-model-controller-7998c67dc4-2xpwc\" (UID: \"98c7acc6-8ae3-470e-b51b-8be2db428a7d\") " pod="opendatahub/odh-model-controller-7998c67dc4-2xpwc" Jul 06 22:14:44.351155 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:44.351116 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-2xpwc" Jul 06 22:14:44.387465 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:44.387424 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:14:44.456806 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:44.456747 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-rc8jv"] Jul 06 22:14:44.460719 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:44.460696 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-rc8jv" Jul 06 22:14:44.468490 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:44.468265 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 06 22:14:44.468490 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:44.468302 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-clphh\"" Jul 06 22:14:44.468490 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:44.468304 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-smpjr"] Jul 06 22:14:44.471823 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:44.471803 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-smpjr" Jul 06 22:14:44.474589 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:44.474423 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 06 22:14:44.474589 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:44.474575 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 06 22:14:44.475044 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:44.475025 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 06 22:14:44.475415 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:44.475389 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 06 22:14:44.476043 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:44.476021 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-q9wdv\"" Jul 06 22:14:44.488846 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:44.488413 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dbwlc\" (UniqueName: \"kubernetes.io/projected/99921ce0-902e-4f75-a484-30dc656427b5-kube-api-access-dbwlc\") pod \"istiod-openshift-gateway-7574459656-smpjr\" (UID: \"99921ce0-902e-4f75-a484-30dc656427b5\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-smpjr" Jul 06 22:14:44.488846 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:44.488454 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/99921ce0-902e-4f75-a484-30dc656427b5-cacerts\") pod \"istiod-openshift-gateway-7574459656-smpjr\" (UID: \"99921ce0-902e-4f75-a484-30dc656427b5\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-smpjr" Jul 06 22:14:44.488846 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:44.488479 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/99921ce0-902e-4f75-a484-30dc656427b5-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-smpjr\" (UID: \"99921ce0-902e-4f75-a484-30dc656427b5\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-smpjr" Jul 06 22:14:44.488846 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:44.488513 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xvc6z\" (UniqueName: \"kubernetes.io/projected/3f238cfe-21cc-46f6-94f7-60d2327e6af1-kube-api-access-xvc6z\") pod \"model-serving-api-54d89bd5fc-rc8jv\" (UID: \"3f238cfe-21cc-46f6-94f7-60d2327e6af1\") " pod="opendatahub/model-serving-api-54d89bd5fc-rc8jv" Jul 06 22:14:44.488846 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:44.488547 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/99921ce0-902e-4f75-a484-30dc656427b5-istio-token\") pod \"istiod-openshift-gateway-7574459656-smpjr\" (UID: \"99921ce0-902e-4f75-a484-30dc656427b5\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-smpjr" Jul 06 22:14:44.488846 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:44.488586 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/99921ce0-902e-4f75-a484-30dc656427b5-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-smpjr\" (UID: \"99921ce0-902e-4f75-a484-30dc656427b5\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-smpjr" Jul 06 22:14:44.488846 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:44.488629 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/99921ce0-902e-4f75-a484-30dc656427b5-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-smpjr\" (UID: \"99921ce0-902e-4f75-a484-30dc656427b5\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-smpjr" Jul 06 22:14:44.488846 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:44.488685 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/99921ce0-902e-4f75-a484-30dc656427b5-local-certs\") pod \"istiod-openshift-gateway-7574459656-smpjr\" (UID: \"99921ce0-902e-4f75-a484-30dc656427b5\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-smpjr" Jul 06 22:14:44.488846 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:44.488713 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3f238cfe-21cc-46f6-94f7-60d2327e6af1-tls-certs\") pod \"model-serving-api-54d89bd5fc-rc8jv\" (UID: \"3f238cfe-21cc-46f6-94f7-60d2327e6af1\") " pod="opendatahub/model-serving-api-54d89bd5fc-rc8jv" Jul 06 22:14:44.488846 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:44.487565 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-rc8jv"] Jul 06 22:14:44.490297 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:44.490277 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-smpjr"] Jul 06 22:14:44.523172 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:44.523147 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-2xpwc"] Jul 06 22:14:44.525192 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:44.525157 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:14:44.525848 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:14:44.525822 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98c7acc6_8ae3_470e_b51b_8be2db428a7d.slice/crio-22b9b8b62e4adc4b3a34c23a7a7ea44e33ae01d5c31932619925f041fea5a5eb WatchSource:0}: Error finding container 22b9b8b62e4adc4b3a34c23a7a7ea44e33ae01d5c31932619925f041fea5a5eb: Status 404 returned error can't find the container with id 22b9b8b62e4adc4b3a34c23a7a7ea44e33ae01d5c31932619925f041fea5a5eb Jul 06 22:14:44.589592 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:44.589546 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/99921ce0-902e-4f75-a484-30dc656427b5-cacerts\") pod \"istiod-openshift-gateway-7574459656-smpjr\" (UID: \"99921ce0-902e-4f75-a484-30dc656427b5\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-smpjr" Jul 06 22:14:44.589592 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:44.589589 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/99921ce0-902e-4f75-a484-30dc656427b5-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-smpjr\" (UID: \"99921ce0-902e-4f75-a484-30dc656427b5\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-smpjr" Jul 06 22:14:44.590122 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:44.589611 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xvc6z\" (UniqueName: \"kubernetes.io/projected/3f238cfe-21cc-46f6-94f7-60d2327e6af1-kube-api-access-xvc6z\") pod \"model-serving-api-54d89bd5fc-rc8jv\" (UID: \"3f238cfe-21cc-46f6-94f7-60d2327e6af1\") " pod="opendatahub/model-serving-api-54d89bd5fc-rc8jv" Jul 06 22:14:44.590122 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:44.589646 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/99921ce0-902e-4f75-a484-30dc656427b5-istio-token\") pod \"istiod-openshift-gateway-7574459656-smpjr\" (UID: \"99921ce0-902e-4f75-a484-30dc656427b5\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-smpjr" Jul 06 22:14:44.590122 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:44.589686 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/99921ce0-902e-4f75-a484-30dc656427b5-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-smpjr\" (UID: \"99921ce0-902e-4f75-a484-30dc656427b5\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-smpjr" Jul 06 22:14:44.590122 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:44.589726 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/99921ce0-902e-4f75-a484-30dc656427b5-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-smpjr\" (UID: \"99921ce0-902e-4f75-a484-30dc656427b5\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-smpjr" Jul 06 22:14:44.590122 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:44.589783 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/99921ce0-902e-4f75-a484-30dc656427b5-local-certs\") pod \"istiod-openshift-gateway-7574459656-smpjr\" (UID: \"99921ce0-902e-4f75-a484-30dc656427b5\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-smpjr" Jul 06 22:14:44.590122 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:44.589807 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3f238cfe-21cc-46f6-94f7-60d2327e6af1-tls-certs\") pod \"model-serving-api-54d89bd5fc-rc8jv\" (UID: \"3f238cfe-21cc-46f6-94f7-60d2327e6af1\") " pod="opendatahub/model-serving-api-54d89bd5fc-rc8jv" Jul 06 22:14:44.590122 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:44.589848 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dbwlc\" (UniqueName: \"kubernetes.io/projected/99921ce0-902e-4f75-a484-30dc656427b5-kube-api-access-dbwlc\") pod \"istiod-openshift-gateway-7574459656-smpjr\" (UID: \"99921ce0-902e-4f75-a484-30dc656427b5\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-smpjr" Jul 06 22:14:44.590504 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:44.590486 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/99921ce0-902e-4f75-a484-30dc656427b5-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-smpjr\" (UID: \"99921ce0-902e-4f75-a484-30dc656427b5\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-smpjr" Jul 06 22:14:44.592181 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:44.592158 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/99921ce0-902e-4f75-a484-30dc656427b5-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-smpjr\" (UID: \"99921ce0-902e-4f75-a484-30dc656427b5\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-smpjr" Jul 06 22:14:44.592181 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:44.592163 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/99921ce0-902e-4f75-a484-30dc656427b5-cacerts\") pod \"istiod-openshift-gateway-7574459656-smpjr\" (UID: \"99921ce0-902e-4f75-a484-30dc656427b5\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-smpjr" Jul 06 22:14:44.592362 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:44.592191 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/99921ce0-902e-4f75-a484-30dc656427b5-local-certs\") pod \"istiod-openshift-gateway-7574459656-smpjr\" (UID: \"99921ce0-902e-4f75-a484-30dc656427b5\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-smpjr" Jul 06 22:14:44.592533 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:44.592511 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/99921ce0-902e-4f75-a484-30dc656427b5-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-smpjr\" (UID: \"99921ce0-902e-4f75-a484-30dc656427b5\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-smpjr" Jul 06 22:14:44.592625 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:44.592607 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3f238cfe-21cc-46f6-94f7-60d2327e6af1-tls-certs\") pod \"model-serving-api-54d89bd5fc-rc8jv\" (UID: \"3f238cfe-21cc-46f6-94f7-60d2327e6af1\") " pod="opendatahub/model-serving-api-54d89bd5fc-rc8jv" Jul 06 22:14:44.598219 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:44.598192 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/99921ce0-902e-4f75-a484-30dc656427b5-istio-token\") pod \"istiod-openshift-gateway-7574459656-smpjr\" (UID: \"99921ce0-902e-4f75-a484-30dc656427b5\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-smpjr" Jul 06 22:14:44.598818 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:44.598794 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xvc6z\" (UniqueName: \"kubernetes.io/projected/3f238cfe-21cc-46f6-94f7-60d2327e6af1-kube-api-access-xvc6z\") pod \"model-serving-api-54d89bd5fc-rc8jv\" (UID: \"3f238cfe-21cc-46f6-94f7-60d2327e6af1\") " pod="opendatahub/model-serving-api-54d89bd5fc-rc8jv" Jul 06 22:14:44.599158 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:44.599132 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dbwlc\" (UniqueName: \"kubernetes.io/projected/99921ce0-902e-4f75-a484-30dc656427b5-kube-api-access-dbwlc\") pod \"istiod-openshift-gateway-7574459656-smpjr\" (UID: \"99921ce0-902e-4f75-a484-30dc656427b5\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-smpjr" Jul 06 22:14:44.619584 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:44.619521 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:14:44.657495 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:44.657468 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-2xpwc" event={"ID":"98c7acc6-8ae3-470e-b51b-8be2db428a7d","Type":"ContainerStarted","Data":"22b9b8b62e4adc4b3a34c23a7a7ea44e33ae01d5c31932619925f041fea5a5eb"} Jul 06 22:14:44.779560 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:44.779522 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-rc8jv" Jul 06 22:14:44.788994 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:44.788968 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-smpjr" Jul 06 22:14:44.947042 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:44.946972 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-rc8jv"] Jul 06 22:14:44.950883 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:14:44.950854 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f238cfe_21cc_46f6_94f7_60d2327e6af1.slice/crio-40b64964da1f86b2566f67618077ab16462d6b16c2a926d73c87aec52eaae5b7 WatchSource:0}: Error finding container 40b64964da1f86b2566f67618077ab16462d6b16c2a926d73c87aec52eaae5b7: Status 404 returned error can't find the container with id 40b64964da1f86b2566f67618077ab16462d6b16c2a926d73c87aec52eaae5b7 Jul 06 22:14:44.968124 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:44.968100 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-smpjr"] Jul 06 22:14:44.971240 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:14:44.971210 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99921ce0_902e_4f75_a484_30dc656427b5.slice/crio-64b0ce3eb46432306aaff45b65301f9064ae03c117820e71a1000ca8ec2d12b4 WatchSource:0}: Error finding container 64b0ce3eb46432306aaff45b65301f9064ae03c117820e71a1000ca8ec2d12b4: Status 404 returned error can't find the container with id 64b0ce3eb46432306aaff45b65301f9064ae03c117820e71a1000ca8ec2d12b4 Jul 06 22:14:45.665383 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:45.665320 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-smpjr" event={"ID":"99921ce0-902e-4f75-a484-30dc656427b5","Type":"ContainerStarted","Data":"64b0ce3eb46432306aaff45b65301f9064ae03c117820e71a1000ca8ec2d12b4"} Jul 06 22:14:45.666938 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:45.666908 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-rc8jv" event={"ID":"3f238cfe-21cc-46f6-94f7-60d2327e6af1","Type":"ContainerStarted","Data":"40b64964da1f86b2566f67618077ab16462d6b16c2a926d73c87aec52eaae5b7"} Jul 06 22:14:45.872160 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:45.872126 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:14:49.465423 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:49.465386 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 22:14:49.465807 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:49.465471 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 22:14:49.689041 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:49.688995 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-smpjr" event={"ID":"99921ce0-902e-4f75-a484-30dc656427b5","Type":"ContainerStarted","Data":"9cc6705df56a714f9c89619987b46cf3fcbf7dd01541e97f333d740f91ea9706"} Jul 06 22:14:49.689236 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:49.689136 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-smpjr" Jul 06 22:14:49.690593 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:49.690560 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-rc8jv" event={"ID":"3f238cfe-21cc-46f6-94f7-60d2327e6af1","Type":"ContainerStarted","Data":"96e3e09e69f8e53b40648bfadb3e4cbd26bf810b7a0a30148763ea1d1405323c"} Jul 06 22:14:49.690720 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:49.690637 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-rc8jv" Jul 06 22:14:49.692010 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:49.691981 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-658774566b-59kk5" event={"ID":"70ec6221-d9c4-449b-b469-8725e711cae5","Type":"ContainerStarted","Data":"2a003a7f222dce840f5b30fc8948a5be96296ffd3ffbd2e7bb254c9b7be5befc"} Jul 06 22:14:49.693414 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:49.693391 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-2xpwc" event={"ID":"98c7acc6-8ae3-470e-b51b-8be2db428a7d","Type":"ContainerStarted","Data":"f16acf8c0c97824a337d0a3e2f725bb6f1a3a1ac286196c188cc59ad0ded8320"} Jul 06 22:14:49.693548 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:49.693535 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-2xpwc" Jul 06 22:14:49.739813 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:49.739766 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-smpjr" podStartSLOduration=1.248289394 podStartE2EDuration="5.739752353s" totalImagesPullingTime="4.491462959s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:14:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:14:49.737719433 +0000 UTC m=+369.296514753" watchObservedRunningTime="2026-07-06 22:14:49.739752353 +0000 UTC m=+369.298547674" Jul 06 22:14:49.777681 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:49.777633 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-658774566b-59kk5" podStartSLOduration=0.980818414 podStartE2EDuration="6.77762306s" totalImagesPullingTime="5.796804646s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:14:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:14:49.775901596 +0000 UTC m=+369.334696915" watchObservedRunningTime="2026-07-06 22:14:49.77762306 +0000 UTC m=+369.336418387" Jul 06 22:14:49.808613 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:49.808554 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-2xpwc" podStartSLOduration=1.8705662790000002 podStartE2EDuration="6.808540106s" totalImagesPullingTime="4.937973827s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:14:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:14:49.806309436 +0000 UTC m=+369.365104756" watchObservedRunningTime="2026-07-06 22:14:49.808540106 +0000 UTC m=+369.367335426" Jul 06 22:14:49.850535 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:49.850473 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-rc8jv" podStartSLOduration=1.350719694 podStartE2EDuration="5.850451309s" totalImagesPullingTime="4.499731615s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:14:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:14:49.848208661 +0000 UTC m=+369.407004016" watchObservedRunningTime="2026-07-06 22:14:49.850451309 +0000 UTC m=+369.409246631" Jul 06 22:14:50.264488 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:50.264448 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:14:50.405978 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:50.405938 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:14:50.700037 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:50.699992 2581 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-smpjr container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 06 22:14:50.700407 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:50.700055 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-smpjr" podUID="99921ce0-902e-4f75-a484-30dc656427b5" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 22:14:51.317395 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:51.317360 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vcm7z"] Jul 06 22:14:51.321250 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:51.321226 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vcm7z" Jul 06 22:14:51.324216 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:51.324195 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-hwk97\"" Jul 06 22:14:51.332790 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:51.332768 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vcm7z"] Jul 06 22:14:51.348834 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:51.348804 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e046a164-38cc-40ae-a412-18ad8af5b2e3-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4vcm7z\" (UID: \"e046a164-38cc-40ae-a412-18ad8af5b2e3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vcm7z" Jul 06 22:14:51.348950 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:51.348841 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zdkpm\" (UniqueName: \"kubernetes.io/projected/e046a164-38cc-40ae-a412-18ad8af5b2e3-kube-api-access-zdkpm\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4vcm7z\" (UID: \"e046a164-38cc-40ae-a412-18ad8af5b2e3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vcm7z" Jul 06 22:14:51.348950 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:51.348879 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e046a164-38cc-40ae-a412-18ad8af5b2e3-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4vcm7z\" (UID: \"e046a164-38cc-40ae-a412-18ad8af5b2e3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vcm7z" Jul 06 22:14:51.348950 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:51.348911 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e046a164-38cc-40ae-a412-18ad8af5b2e3-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4vcm7z\" (UID: \"e046a164-38cc-40ae-a412-18ad8af5b2e3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vcm7z" Jul 06 22:14:51.348950 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:51.348928 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e046a164-38cc-40ae-a412-18ad8af5b2e3-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4vcm7z\" (UID: \"e046a164-38cc-40ae-a412-18ad8af5b2e3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vcm7z" Jul 06 22:14:51.349185 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:51.349006 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e046a164-38cc-40ae-a412-18ad8af5b2e3-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4vcm7z\" (UID: \"e046a164-38cc-40ae-a412-18ad8af5b2e3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vcm7z" Jul 06 22:14:51.349185 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:51.349040 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e046a164-38cc-40ae-a412-18ad8af5b2e3-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4vcm7z\" (UID: \"e046a164-38cc-40ae-a412-18ad8af5b2e3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vcm7z" Jul 06 22:14:51.349185 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:51.349110 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e046a164-38cc-40ae-a412-18ad8af5b2e3-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4vcm7z\" (UID: \"e046a164-38cc-40ae-a412-18ad8af5b2e3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vcm7z" Jul 06 22:14:51.349322 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:51.349189 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e046a164-38cc-40ae-a412-18ad8af5b2e3-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4vcm7z\" (UID: \"e046a164-38cc-40ae-a412-18ad8af5b2e3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vcm7z" Jul 06 22:14:51.356837 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:51.356779 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:14:51.450355 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:51.450317 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e046a164-38cc-40ae-a412-18ad8af5b2e3-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4vcm7z\" (UID: \"e046a164-38cc-40ae-a412-18ad8af5b2e3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vcm7z" Jul 06 22:14:51.450542 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:51.450362 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e046a164-38cc-40ae-a412-18ad8af5b2e3-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4vcm7z\" (UID: \"e046a164-38cc-40ae-a412-18ad8af5b2e3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vcm7z" Jul 06 22:14:51.450542 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:51.450395 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e046a164-38cc-40ae-a412-18ad8af5b2e3-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4vcm7z\" (UID: \"e046a164-38cc-40ae-a412-18ad8af5b2e3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vcm7z" Jul 06 22:14:51.450542 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:51.450422 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e046a164-38cc-40ae-a412-18ad8af5b2e3-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4vcm7z\" (UID: \"e046a164-38cc-40ae-a412-18ad8af5b2e3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vcm7z" Jul 06 22:14:51.450542 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:51.450460 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e046a164-38cc-40ae-a412-18ad8af5b2e3-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4vcm7z\" (UID: \"e046a164-38cc-40ae-a412-18ad8af5b2e3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vcm7z" Jul 06 22:14:51.450542 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:51.450497 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zdkpm\" (UniqueName: \"kubernetes.io/projected/e046a164-38cc-40ae-a412-18ad8af5b2e3-kube-api-access-zdkpm\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4vcm7z\" (UID: \"e046a164-38cc-40ae-a412-18ad8af5b2e3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vcm7z" Jul 06 22:14:51.450542 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:51.450532 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e046a164-38cc-40ae-a412-18ad8af5b2e3-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4vcm7z\" (UID: \"e046a164-38cc-40ae-a412-18ad8af5b2e3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vcm7z" Jul 06 22:14:51.450857 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:51.450575 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e046a164-38cc-40ae-a412-18ad8af5b2e3-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4vcm7z\" (UID: \"e046a164-38cc-40ae-a412-18ad8af5b2e3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vcm7z" Jul 06 22:14:51.450857 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:51.450601 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e046a164-38cc-40ae-a412-18ad8af5b2e3-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4vcm7z\" (UID: \"e046a164-38cc-40ae-a412-18ad8af5b2e3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vcm7z" Jul 06 22:14:51.450857 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:51.450746 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e046a164-38cc-40ae-a412-18ad8af5b2e3-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4vcm7z\" (UID: \"e046a164-38cc-40ae-a412-18ad8af5b2e3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vcm7z" Jul 06 22:14:51.451014 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:51.450866 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e046a164-38cc-40ae-a412-18ad8af5b2e3-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4vcm7z\" (UID: \"e046a164-38cc-40ae-a412-18ad8af5b2e3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vcm7z" Jul 06 22:14:51.451014 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:51.450946 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e046a164-38cc-40ae-a412-18ad8af5b2e3-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4vcm7z\" (UID: \"e046a164-38cc-40ae-a412-18ad8af5b2e3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vcm7z" Jul 06 22:14:51.451150 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:51.451116 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e046a164-38cc-40ae-a412-18ad8af5b2e3-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4vcm7z\" (UID: \"e046a164-38cc-40ae-a412-18ad8af5b2e3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vcm7z" Jul 06 22:14:51.451614 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:51.451591 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e046a164-38cc-40ae-a412-18ad8af5b2e3-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4vcm7z\" (UID: \"e046a164-38cc-40ae-a412-18ad8af5b2e3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vcm7z" Jul 06 22:14:51.452946 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:51.452923 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e046a164-38cc-40ae-a412-18ad8af5b2e3-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4vcm7z\" (UID: \"e046a164-38cc-40ae-a412-18ad8af5b2e3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vcm7z" Jul 06 22:14:51.453235 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:51.453218 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e046a164-38cc-40ae-a412-18ad8af5b2e3-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4vcm7z\" (UID: \"e046a164-38cc-40ae-a412-18ad8af5b2e3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vcm7z" Jul 06 22:14:51.459800 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:51.459770 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zdkpm\" (UniqueName: \"kubernetes.io/projected/e046a164-38cc-40ae-a412-18ad8af5b2e3-kube-api-access-zdkpm\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4vcm7z\" (UID: \"e046a164-38cc-40ae-a412-18ad8af5b2e3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vcm7z" Jul 06 22:14:51.460095 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:51.460052 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e046a164-38cc-40ae-a412-18ad8af5b2e3-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4vcm7z\" (UID: \"e046a164-38cc-40ae-a412-18ad8af5b2e3\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vcm7z" Jul 06 22:14:51.633380 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:51.633297 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vcm7z" Jul 06 22:14:51.684897 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:51.684856 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:14:51.792682 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:51.792658 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vcm7z"] Jul 06 22:14:51.794306 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:14:51.794278 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode046a164_38cc_40ae_a412_18ad8af5b2e3.slice/crio-b0a10debdd898874dc622d6a1a2ce71742660d173bcbba7a461accb044d2f7da WatchSource:0}: Error finding container b0a10debdd898874dc622d6a1a2ce71742660d173bcbba7a461accb044d2f7da: Status 404 returned error can't find the container with id b0a10debdd898874dc622d6a1a2ce71742660d173bcbba7a461accb044d2f7da Jul 06 22:14:51.824498 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:51.824460 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:14:52.708833 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:52.708799 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vcm7z" event={"ID":"e046a164-38cc-40ae-a412-18ad8af5b2e3","Type":"ContainerStarted","Data":"b0a10debdd898874dc622d6a1a2ce71742660d173bcbba7a461accb044d2f7da"} Jul 06 22:14:53.461688 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:53.461636 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:14:53.568839 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:53.568807 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:14:53.700001 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:53.699965 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-smpjr" Jul 06 22:14:54.145410 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:54.145371 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:14:54.236398 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:54.236361 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-wn2sx"] Jul 06 22:14:54.244786 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:54.243403 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-wn2sx" Jul 06 22:14:54.246493 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:54.246467 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-4swjt"] Jul 06 22:14:54.247571 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:54.247012 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 06 22:14:54.247571 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:54.247403 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-ng8jf\"" Jul 06 22:14:54.252122 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:54.251619 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4swjt" Jul 06 22:14:54.252742 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:54.252535 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-wn2sx"] Jul 06 22:14:54.254495 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:54.254474 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-btjbg\"" Jul 06 22:14:54.256115 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:54.254700 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 06 22:14:54.256115 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:54.255782 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:14:54.260186 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:54.260118 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-4swjt"] Jul 06 22:14:54.281768 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:54.281745 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5ad9d1ac-6623-4ff6-8c6d-6003ee007795-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-4swjt\" (UID: \"5ad9d1ac-6623-4ff6-8c6d-6003ee007795\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4swjt" Jul 06 22:14:54.281897 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:54.281795 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p25nh\" (UniqueName: \"kubernetes.io/projected/ab6dcafe-a3c5-41d3-92ae-55aafb915634-kube-api-access-p25nh\") pod \"kserve-controller-manager-d6f4b65b8-wn2sx\" (UID: \"ab6dcafe-a3c5-41d3-92ae-55aafb915634\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-wn2sx" Jul 06 22:14:54.281961 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:54.281916 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ab6dcafe-a3c5-41d3-92ae-55aafb915634-cert\") pod \"kserve-controller-manager-d6f4b65b8-wn2sx\" (UID: \"ab6dcafe-a3c5-41d3-92ae-55aafb915634\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-wn2sx" Jul 06 22:14:54.282044 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:54.282022 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pdght\" (UniqueName: \"kubernetes.io/projected/5ad9d1ac-6623-4ff6-8c6d-6003ee007795-kube-api-access-pdght\") pod \"llmisvc-controller-manager-777c6bd7d5-4swjt\" (UID: \"5ad9d1ac-6623-4ff6-8c6d-6003ee007795\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4swjt" Jul 06 22:14:54.382567 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:54.382523 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ab6dcafe-a3c5-41d3-92ae-55aafb915634-cert\") pod \"kserve-controller-manager-d6f4b65b8-wn2sx\" (UID: \"ab6dcafe-a3c5-41d3-92ae-55aafb915634\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-wn2sx" Jul 06 22:14:54.382744 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:54.382614 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pdght\" (UniqueName: \"kubernetes.io/projected/5ad9d1ac-6623-4ff6-8c6d-6003ee007795-kube-api-access-pdght\") pod \"llmisvc-controller-manager-777c6bd7d5-4swjt\" (UID: \"5ad9d1ac-6623-4ff6-8c6d-6003ee007795\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4swjt" Jul 06 22:14:54.382744 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:54.382647 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5ad9d1ac-6623-4ff6-8c6d-6003ee007795-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-4swjt\" (UID: \"5ad9d1ac-6623-4ff6-8c6d-6003ee007795\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4swjt" Jul 06 22:14:54.382744 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:54.382684 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p25nh\" (UniqueName: \"kubernetes.io/projected/ab6dcafe-a3c5-41d3-92ae-55aafb915634-kube-api-access-p25nh\") pod \"kserve-controller-manager-d6f4b65b8-wn2sx\" (UID: \"ab6dcafe-a3c5-41d3-92ae-55aafb915634\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-wn2sx" Jul 06 22:14:54.382744 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:14:54.382694 2581 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found Jul 06 22:14:54.382945 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:14:54.382770 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ab6dcafe-a3c5-41d3-92ae-55aafb915634-cert podName:ab6dcafe-a3c5-41d3-92ae-55aafb915634 nodeName:}" failed. No retries permitted until 2026-07-06 22:14:54.882747384 +0000 UTC m=+374.441542697 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/ab6dcafe-a3c5-41d3-92ae-55aafb915634-cert") pod "kserve-controller-manager-d6f4b65b8-wn2sx" (UID: "ab6dcafe-a3c5-41d3-92ae-55aafb915634") : secret "kserve-webhook-server-cert" not found Jul 06 22:14:54.382945 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:14:54.382813 2581 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found Jul 06 22:14:54.382945 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:14:54.382880 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5ad9d1ac-6623-4ff6-8c6d-6003ee007795-cert podName:5ad9d1ac-6623-4ff6-8c6d-6003ee007795 nodeName:}" failed. No retries permitted until 2026-07-06 22:14:54.882861656 +0000 UTC m=+374.441656959 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/5ad9d1ac-6623-4ff6-8c6d-6003ee007795-cert") pod "llmisvc-controller-manager-777c6bd7d5-4swjt" (UID: "5ad9d1ac-6623-4ff6-8c6d-6003ee007795") : secret "llmisvc-webhook-server-cert" not found Jul 06 22:14:54.397675 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:54.397608 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pdght\" (UniqueName: \"kubernetes.io/projected/5ad9d1ac-6623-4ff6-8c6d-6003ee007795-kube-api-access-pdght\") pod \"llmisvc-controller-manager-777c6bd7d5-4swjt\" (UID: \"5ad9d1ac-6623-4ff6-8c6d-6003ee007795\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4swjt" Jul 06 22:14:54.401877 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:54.401843 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p25nh\" (UniqueName: \"kubernetes.io/projected/ab6dcafe-a3c5-41d3-92ae-55aafb915634-kube-api-access-p25nh\") pod \"kserve-controller-manager-d6f4b65b8-wn2sx\" (UID: \"ab6dcafe-a3c5-41d3-92ae-55aafb915634\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-wn2sx" Jul 06 22:14:54.491539 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:54.491500 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:14:54.652198 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:54.652113 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:14:54.769038 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:54.769001 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 22:14:54.769122 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:54.769099 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 22:14:54.769170 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:54.769157 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 22:14:54.886980 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:54.886947 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5ad9d1ac-6623-4ff6-8c6d-6003ee007795-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-4swjt\" (UID: \"5ad9d1ac-6623-4ff6-8c6d-6003ee007795\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4swjt" Jul 06 22:14:54.887167 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:54.887008 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ab6dcafe-a3c5-41d3-92ae-55aafb915634-cert\") pod \"kserve-controller-manager-d6f4b65b8-wn2sx\" (UID: \"ab6dcafe-a3c5-41d3-92ae-55aafb915634\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-wn2sx" Jul 06 22:14:54.889541 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:54.889508 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ab6dcafe-a3c5-41d3-92ae-55aafb915634-cert\") pod \"kserve-controller-manager-d6f4b65b8-wn2sx\" (UID: \"ab6dcafe-a3c5-41d3-92ae-55aafb915634\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-wn2sx" Jul 06 22:14:54.889653 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:54.889521 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5ad9d1ac-6623-4ff6-8c6d-6003ee007795-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-4swjt\" (UID: \"5ad9d1ac-6623-4ff6-8c6d-6003ee007795\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4swjt" Jul 06 22:14:55.163675 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:55.163627 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-wn2sx" Jul 06 22:14:55.172816 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:55.172795 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4swjt" Jul 06 22:14:55.305718 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:55.305684 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-smpjr"] Jul 06 22:14:55.305929 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:55.305902 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-smpjr" podUID="99921ce0-902e-4f75-a484-30dc656427b5" containerName="discovery" containerID="cri-o://9cc6705df56a714f9c89619987b46cf3fcbf7dd01541e97f333d740f91ea9706" gracePeriod=30 Jul 06 22:14:55.324871 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:55.324836 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-wn2sx"] Jul 06 22:14:55.326051 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:14:55.326027 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab6dcafe_a3c5_41d3_92ae_55aafb915634.slice/crio-581581b22e0ffcbfb97b05f6ae26115b681241c29036b3941e4d9748e97199fc WatchSource:0}: Error finding container 581581b22e0ffcbfb97b05f6ae26115b681241c29036b3941e4d9748e97199fc: Status 404 returned error can't find the container with id 581581b22e0ffcbfb97b05f6ae26115b681241c29036b3941e4d9748e97199fc Jul 06 22:14:55.337589 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:55.337562 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-4swjt"] Jul 06 22:14:55.391879 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:14:55.391851 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod5ad9d1ac_6623_4ff6_8c6d_6003ee007795.slice/crio-0cadce755d2e8a59293bb36348d33c297060109d341ceb8ee6d929090bcb2e3e WatchSource:0}: Error finding container 0cadce755d2e8a59293bb36348d33c297060109d341ceb8ee6d929090bcb2e3e: Status 404 returned error can't find the container with id 0cadce755d2e8a59293bb36348d33c297060109d341ceb8ee6d929090bcb2e3e Jul 06 22:14:55.462389 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:55.462337 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:14:55.551092 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:55.551053 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-smpjr" Jul 06 22:14:55.557834 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:55.557809 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:14:55.592985 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:55.592954 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/99921ce0-902e-4f75-a484-30dc656427b5-istio-csr-dns-cert\") pod \"99921ce0-902e-4f75-a484-30dc656427b5\" (UID: \"99921ce0-902e-4f75-a484-30dc656427b5\") " Jul 06 22:14:55.593141 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:55.592998 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/99921ce0-902e-4f75-a484-30dc656427b5-istio-csr-ca-configmap\") pod \"99921ce0-902e-4f75-a484-30dc656427b5\" (UID: \"99921ce0-902e-4f75-a484-30dc656427b5\") " Jul 06 22:14:55.593213 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:55.593152 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/99921ce0-902e-4f75-a484-30dc656427b5-local-certs\") pod \"99921ce0-902e-4f75-a484-30dc656427b5\" (UID: \"99921ce0-902e-4f75-a484-30dc656427b5\") " Jul 06 22:14:55.593273 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:55.593233 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dbwlc\" (UniqueName: \"kubernetes.io/projected/99921ce0-902e-4f75-a484-30dc656427b5-kube-api-access-dbwlc\") pod \"99921ce0-902e-4f75-a484-30dc656427b5\" (UID: \"99921ce0-902e-4f75-a484-30dc656427b5\") " Jul 06 22:14:55.593323 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:55.593274 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/99921ce0-902e-4f75-a484-30dc656427b5-istio-token\") pod \"99921ce0-902e-4f75-a484-30dc656427b5\" (UID: \"99921ce0-902e-4f75-a484-30dc656427b5\") " Jul 06 22:14:55.593323 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:55.593305 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/99921ce0-902e-4f75-a484-30dc656427b5-istio-kubeconfig\") pod \"99921ce0-902e-4f75-a484-30dc656427b5\" (UID: \"99921ce0-902e-4f75-a484-30dc656427b5\") " Jul 06 22:14:55.593422 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:55.593336 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/99921ce0-902e-4f75-a484-30dc656427b5-cacerts\") pod \"99921ce0-902e-4f75-a484-30dc656427b5\" (UID: \"99921ce0-902e-4f75-a484-30dc656427b5\") " Jul 06 22:14:55.593422 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:55.593393 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/99921ce0-902e-4f75-a484-30dc656427b5-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "99921ce0-902e-4f75-a484-30dc656427b5" (UID: "99921ce0-902e-4f75-a484-30dc656427b5"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 22:14:55.593631 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:55.593610 2581 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/99921ce0-902e-4f75-a484-30dc656427b5-istio-csr-ca-configmap\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:14:55.595736 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:55.595694 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/99921ce0-902e-4f75-a484-30dc656427b5-istio-token" (OuterVolumeSpecName: "istio-token") pod "99921ce0-902e-4f75-a484-30dc656427b5" (UID: "99921ce0-902e-4f75-a484-30dc656427b5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 22:14:55.595736 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:55.595712 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/99921ce0-902e-4f75-a484-30dc656427b5-kube-api-access-dbwlc" (OuterVolumeSpecName: "kube-api-access-dbwlc") pod "99921ce0-902e-4f75-a484-30dc656427b5" (UID: "99921ce0-902e-4f75-a484-30dc656427b5"). InnerVolumeSpecName "kube-api-access-dbwlc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 22:14:55.595965 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:55.595766 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99921ce0-902e-4f75-a484-30dc656427b5-local-certs" (OuterVolumeSpecName: "local-certs") pod "99921ce0-902e-4f75-a484-30dc656427b5" (UID: "99921ce0-902e-4f75-a484-30dc656427b5"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:14:55.595965 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:55.595794 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/99921ce0-902e-4f75-a484-30dc656427b5-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "99921ce0-902e-4f75-a484-30dc656427b5" (UID: "99921ce0-902e-4f75-a484-30dc656427b5"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 22:14:55.595965 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:55.595837 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/99921ce0-902e-4f75-a484-30dc656427b5-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "99921ce0-902e-4f75-a484-30dc656427b5" (UID: "99921ce0-902e-4f75-a484-30dc656427b5"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 22:14:55.596418 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:55.596402 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/99921ce0-902e-4f75-a484-30dc656427b5-cacerts" (OuterVolumeSpecName: "cacerts") pod "99921ce0-902e-4f75-a484-30dc656427b5" (UID: "99921ce0-902e-4f75-a484-30dc656427b5"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 22:14:55.694787 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:55.694752 2581 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/99921ce0-902e-4f75-a484-30dc656427b5-istio-csr-dns-cert\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:14:55.694787 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:55.694784 2581 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/99921ce0-902e-4f75-a484-30dc656427b5-local-certs\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:14:55.694984 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:55.694797 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dbwlc\" (UniqueName: \"kubernetes.io/projected/99921ce0-902e-4f75-a484-30dc656427b5-kube-api-access-dbwlc\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:14:55.694984 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:55.694813 2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/99921ce0-902e-4f75-a484-30dc656427b5-istio-token\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:14:55.694984 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:55.694824 2581 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/99921ce0-902e-4f75-a484-30dc656427b5-istio-kubeconfig\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:14:55.694984 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:55.694833 2581 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/99921ce0-902e-4f75-a484-30dc656427b5-cacerts\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:14:55.711010 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:55.710967 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:14:55.729680 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:55.729642 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vcm7z" event={"ID":"e046a164-38cc-40ae-a412-18ad8af5b2e3","Type":"ContainerStarted","Data":"386b409127c29ff3ae215eb2089bcc1e2a63947a1f5fbf037cb58770fccb39a6"} Jul 06 22:14:55.730842 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:55.730815 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4swjt" event={"ID":"5ad9d1ac-6623-4ff6-8c6d-6003ee007795","Type":"ContainerStarted","Data":"0cadce755d2e8a59293bb36348d33c297060109d341ceb8ee6d929090bcb2e3e"} Jul 06 22:14:55.732117 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:55.732091 2581 generic.go:356] "Generic (PLEG): container finished" podID="99921ce0-902e-4f75-a484-30dc656427b5" containerID="9cc6705df56a714f9c89619987b46cf3fcbf7dd01541e97f333d740f91ea9706" exitCode=0 Jul 06 22:14:55.732237 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:55.732159 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-smpjr" event={"ID":"99921ce0-902e-4f75-a484-30dc656427b5","Type":"ContainerDied","Data":"9cc6705df56a714f9c89619987b46cf3fcbf7dd01541e97f333d740f91ea9706"} Jul 06 22:14:55.732237 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:55.732208 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-smpjr" event={"ID":"99921ce0-902e-4f75-a484-30dc656427b5","Type":"ContainerDied","Data":"64b0ce3eb46432306aaff45b65301f9064ae03c117820e71a1000ca8ec2d12b4"} Jul 06 22:14:55.732237 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:55.732209 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-smpjr" Jul 06 22:14:55.732386 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:55.732230 2581 scope.go:117] "RemoveContainer" containerID="9cc6705df56a714f9c89619987b46cf3fcbf7dd01541e97f333d740f91ea9706" Jul 06 22:14:55.733526 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:55.733500 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-wn2sx" event={"ID":"ab6dcafe-a3c5-41d3-92ae-55aafb915634","Type":"ContainerStarted","Data":"581581b22e0ffcbfb97b05f6ae26115b681241c29036b3941e4d9748e97199fc"} Jul 06 22:14:55.741465 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:55.741448 2581 scope.go:117] "RemoveContainer" containerID="9cc6705df56a714f9c89619987b46cf3fcbf7dd01541e97f333d740f91ea9706" Jul 06 22:14:55.741757 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:14:55.741735 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9cc6705df56a714f9c89619987b46cf3fcbf7dd01541e97f333d740f91ea9706\": container with ID starting with 9cc6705df56a714f9c89619987b46cf3fcbf7dd01541e97f333d740f91ea9706 not found: ID does not exist" containerID="9cc6705df56a714f9c89619987b46cf3fcbf7dd01541e97f333d740f91ea9706" Jul 06 22:14:55.741802 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:55.741763 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9cc6705df56a714f9c89619987b46cf3fcbf7dd01541e97f333d740f91ea9706"} err="failed to get container status \"9cc6705df56a714f9c89619987b46cf3fcbf7dd01541e97f333d740f91ea9706\": rpc error: code = NotFound desc = could not find container \"9cc6705df56a714f9c89619987b46cf3fcbf7dd01541e97f333d740f91ea9706\": container with ID starting with 9cc6705df56a714f9c89619987b46cf3fcbf7dd01541e97f333d740f91ea9706 not found: ID does not exist" Jul 06 22:14:55.780843 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:55.780790 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vcm7z" podStartSLOduration=1.809017691 podStartE2EDuration="4.78077293s" totalImagesPullingTime="2.971755239s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:14:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:14:55.779212256 +0000 UTC m=+375.338007579" watchObservedRunningTime="2026-07-06 22:14:55.78077293 +0000 UTC m=+375.339568250" Jul 06 22:14:55.797740 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:55.797710 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-smpjr"] Jul 06 22:14:55.802153 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:55.802126 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-smpjr"] Jul 06 22:14:56.634484 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:56.634441 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vcm7z" Jul 06 22:14:56.641887 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:56.641844 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vcm7z" Jul 06 22:14:56.739664 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:56.739637 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vcm7z" Jul 06 22:14:56.740910 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:56.740875 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4vcm7z" Jul 06 22:14:56.977963 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:56.977928 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="99921ce0-902e-4f75-a484-30dc656427b5" path="/var/lib/kubelet/pods/99921ce0-902e-4f75-a484-30dc656427b5/volumes" Jul 06 22:14:59.755433 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:59.755388 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-wn2sx" event={"ID":"ab6dcafe-a3c5-41d3-92ae-55aafb915634","Type":"ContainerStarted","Data":"4d9e583aa3ede4a5a6cc091a221012a5930605f4fae42c29b746a5185b2049b7"} Jul 06 22:14:59.755907 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:59.755542 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-wn2sx" Jul 06 22:14:59.756902 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:59.756874 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4swjt" event={"ID":"5ad9d1ac-6623-4ff6-8c6d-6003ee007795","Type":"ContainerStarted","Data":"7dabf75e730509c5e569dc4af6d008aaf6af0723883a8508a33b9b679b7705a6"} Jul 06 22:14:59.757051 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:59.756996 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4swjt" Jul 06 22:14:59.774147 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:59.774079 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-wn2sx" podStartSLOduration=2.109873036 podStartE2EDuration="5.774042644s" totalImagesPullingTime="3.664169608s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:14:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:14:59.772083782 +0000 UTC m=+379.330879104" watchObservedRunningTime="2026-07-06 22:14:59.774042644 +0000 UTC m=+379.332837966" Jul 06 22:14:59.790794 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:59.790742 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4swjt" podStartSLOduration=2.205040766 podStartE2EDuration="5.790724045s" totalImagesPullingTime="3.585683279s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:14:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:14:59.790086283 +0000 UTC m=+379.348881606" watchObservedRunningTime="2026-07-06 22:14:59.790724045 +0000 UTC m=+379.349519366" Jul 06 22:14:59.809639 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:59.809589 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:14:59.909640 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:14:59.909606 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:15:00.700445 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:15:00.700413 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-2xpwc" Jul 06 22:15:00.702499 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:15:00.702475 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-rc8jv" Jul 06 22:15:01.746108 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:15:01.746056 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:15:30.764249 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:15:30.764220 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4swjt" Jul 06 22:15:30.767144 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:15:30.767127 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-wn2sx" Jul 06 22:15:44.048888 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:15:44.048839 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:15:44.148207 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:15:44.148166 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:15:44.149113 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:15:44.149090 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-984zs"] Jul 06 22:15:44.149408 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:15:44.149397 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="99921ce0-902e-4f75-a484-30dc656427b5" containerName="discovery" Jul 06 22:15:44.149449 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:15:44.149409 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="99921ce0-902e-4f75-a484-30dc656427b5" containerName="discovery" Jul 06 22:15:44.149482 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:15:44.149465 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="99921ce0-902e-4f75-a484-30dc656427b5" containerName="discovery" Jul 06 22:15:44.153526 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:15:44.153502 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-984zs" Jul 06 22:15:44.156126 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:15:44.156101 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 06 22:15:44.156126 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:15:44.156111 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-qk9gc\"" Jul 06 22:15:44.157023 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:15:44.157006 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 06 22:15:44.162350 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:15:44.162056 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-984zs"] Jul 06 22:15:44.197649 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:15:44.197624 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n47kx\" (UniqueName: \"kubernetes.io/projected/812b077b-2dc2-49e6-8434-1ca50b71c2de-kube-api-access-n47kx\") pod \"kuadrant-operator-catalog-984zs\" (UID: \"812b077b-2dc2-49e6-8434-1ca50b71c2de\") " pod="kuadrant-system/kuadrant-operator-catalog-984zs" Jul 06 22:15:44.244026 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:15:44.243982 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:15:44.299008 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:15:44.298930 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n47kx\" (UniqueName: \"kubernetes.io/projected/812b077b-2dc2-49e6-8434-1ca50b71c2de-kube-api-access-n47kx\") pod \"kuadrant-operator-catalog-984zs\" (UID: \"812b077b-2dc2-49e6-8434-1ca50b71c2de\") " pod="kuadrant-system/kuadrant-operator-catalog-984zs" Jul 06 22:15:44.307500 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:15:44.307473 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n47kx\" (UniqueName: \"kubernetes.io/projected/812b077b-2dc2-49e6-8434-1ca50b71c2de-kube-api-access-n47kx\") pod \"kuadrant-operator-catalog-984zs\" (UID: \"812b077b-2dc2-49e6-8434-1ca50b71c2de\") " pod="kuadrant-system/kuadrant-operator-catalog-984zs" Jul 06 22:15:44.465367 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:15:44.465328 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-984zs" Jul 06 22:15:44.520971 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:15:44.520941 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-984zs"] Jul 06 22:15:44.593737 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:15:44.593716 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-984zs"] Jul 06 22:15:44.595524 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:15:44.595487 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod812b077b_2dc2_49e6_8434_1ca50b71c2de.slice/crio-335989e46c6c63af2d9326aa10c9ea10d05dc5783b4cb1e0aacb4e83f1fdc7d6 WatchSource:0}: Error finding container 335989e46c6c63af2d9326aa10c9ea10d05dc5783b4cb1e0aacb4e83f1fdc7d6: Status 404 returned error can't find the container with id 335989e46c6c63af2d9326aa10c9ea10d05dc5783b4cb1e0aacb4e83f1fdc7d6 Jul 06 22:15:44.730454 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:15:44.730421 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-lswds"] Jul 06 22:15:44.734724 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:15:44.734709 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-lswds" Jul 06 22:15:44.741270 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:15:44.741241 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-lswds"] Jul 06 22:15:44.802991 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:15:44.802961 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-42qxn\" (UniqueName: \"kubernetes.io/projected/75cd310a-3b61-4cd2-8156-7a0b787cd2cf-kube-api-access-42qxn\") pod \"kuadrant-operator-catalog-lswds\" (UID: \"75cd310a-3b61-4cd2-8156-7a0b787cd2cf\") " pod="kuadrant-system/kuadrant-operator-catalog-lswds" Jul 06 22:15:44.904474 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:15:44.904388 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-42qxn\" (UniqueName: \"kubernetes.io/projected/75cd310a-3b61-4cd2-8156-7a0b787cd2cf-kube-api-access-42qxn\") pod \"kuadrant-operator-catalog-lswds\" (UID: \"75cd310a-3b61-4cd2-8156-7a0b787cd2cf\") " pod="kuadrant-system/kuadrant-operator-catalog-lswds" Jul 06 22:15:44.913098 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:15:44.913053 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-42qxn\" (UniqueName: \"kubernetes.io/projected/75cd310a-3b61-4cd2-8156-7a0b787cd2cf-kube-api-access-42qxn\") pod \"kuadrant-operator-catalog-lswds\" (UID: \"75cd310a-3b61-4cd2-8156-7a0b787cd2cf\") " pod="kuadrant-system/kuadrant-operator-catalog-lswds" Jul 06 22:15:44.951130 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:15:44.951091 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-984zs" event={"ID":"812b077b-2dc2-49e6-8434-1ca50b71c2de","Type":"ContainerStarted","Data":"335989e46c6c63af2d9326aa10c9ea10d05dc5783b4cb1e0aacb4e83f1fdc7d6"} Jul 06 22:15:44.960741 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:15:44.960715 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:15:45.044466 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:15:45.044437 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-lswds" Jul 06 22:15:45.165765 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:15:45.165670 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:15:45.180039 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:15:45.179849 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-lswds"] Jul 06 22:15:45.183155 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:15:45.183124 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75cd310a_3b61_4cd2_8156_7a0b787cd2cf.slice/crio-2a58618ba5858b654af1054eaeed2f0252d5cb40fead3af7d437d964cc2d0b85 WatchSource:0}: Error finding container 2a58618ba5858b654af1054eaeed2f0252d5cb40fead3af7d437d964cc2d0b85: Status 404 returned error can't find the container with id 2a58618ba5858b654af1054eaeed2f0252d5cb40fead3af7d437d964cc2d0b85 Jul 06 22:15:45.957893 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:15:45.957850 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-lswds" event={"ID":"75cd310a-3b61-4cd2-8156-7a0b787cd2cf","Type":"ContainerStarted","Data":"2a58618ba5858b654af1054eaeed2f0252d5cb40fead3af7d437d964cc2d0b85"} Jul 06 22:15:46.963478 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:15:46.963437 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-lswds" event={"ID":"75cd310a-3b61-4cd2-8156-7a0b787cd2cf","Type":"ContainerStarted","Data":"4c7aedcd417e11754e907a6d43e8bab5bb5020dfba19299fad82ca97759906e8"} Jul 06 22:15:46.965083 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:15:46.965044 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-984zs" podUID="812b077b-2dc2-49e6-8434-1ca50b71c2de" containerName="registry-server" containerID="cri-o://2c829057f4ff26f4b706d77ff43d408290a1bfb84f61cb4d1bd1baea6861cdfb" gracePeriod=2 Jul 06 22:15:46.965216 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:15:46.965047 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-984zs" event={"ID":"812b077b-2dc2-49e6-8434-1ca50b71c2de","Type":"ContainerStarted","Data":"2c829057f4ff26f4b706d77ff43d408290a1bfb84f61cb4d1bd1baea6861cdfb"} Jul 06 22:15:46.981160 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:15:46.981056 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-lswds" podStartSLOduration=1.5443676800000001 podStartE2EDuration="2.981040576s" totalImagesPullingTime="1.436672896s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:15:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:15:46.978947478 +0000 UTC m=+426.537742822" watchObservedRunningTime="2026-07-06 22:15:46.981040576 +0000 UTC m=+426.539835897" Jul 06 22:15:46.995792 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:15:46.995748 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-984zs" podStartSLOduration=0.971737795 podStartE2EDuration="2.995733631s" totalImagesPullingTime="2.023995836s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:15:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:15:46.994182364 +0000 UTC m=+426.552977685" watchObservedRunningTime="2026-07-06 22:15:46.995733631 +0000 UTC m=+426.554528951" Jul 06 22:15:47.209354 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:15:47.209323 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-984zs" Jul 06 22:15:47.329102 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:15:47.329006 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n47kx\" (UniqueName: \"kubernetes.io/projected/812b077b-2dc2-49e6-8434-1ca50b71c2de-kube-api-access-n47kx\") pod \"812b077b-2dc2-49e6-8434-1ca50b71c2de\" (UID: \"812b077b-2dc2-49e6-8434-1ca50b71c2de\") " Jul 06 22:15:47.331150 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:15:47.331121 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/812b077b-2dc2-49e6-8434-1ca50b71c2de-kube-api-access-n47kx" (OuterVolumeSpecName: "kube-api-access-n47kx") pod "812b077b-2dc2-49e6-8434-1ca50b71c2de" (UID: "812b077b-2dc2-49e6-8434-1ca50b71c2de"). InnerVolumeSpecName "kube-api-access-n47kx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 22:15:47.430107 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:15:47.430057 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n47kx\" (UniqueName: \"kubernetes.io/projected/812b077b-2dc2-49e6-8434-1ca50b71c2de-kube-api-access-n47kx\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:15:47.970750 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:15:47.970716 2581 generic.go:356] "Generic (PLEG): container finished" podID="812b077b-2dc2-49e6-8434-1ca50b71c2de" containerID="2c829057f4ff26f4b706d77ff43d408290a1bfb84f61cb4d1bd1baea6861cdfb" exitCode=0 Jul 06 22:15:47.971198 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:15:47.970754 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-984zs" Jul 06 22:15:47.971198 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:15:47.970801 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-984zs" event={"ID":"812b077b-2dc2-49e6-8434-1ca50b71c2de","Type":"ContainerDied","Data":"2c829057f4ff26f4b706d77ff43d408290a1bfb84f61cb4d1bd1baea6861cdfb"} Jul 06 22:15:47.971198 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:15:47.970846 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-984zs" event={"ID":"812b077b-2dc2-49e6-8434-1ca50b71c2de","Type":"ContainerDied","Data":"335989e46c6c63af2d9326aa10c9ea10d05dc5783b4cb1e0aacb4e83f1fdc7d6"} Jul 06 22:15:47.971198 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:15:47.970868 2581 scope.go:117] "RemoveContainer" containerID="2c829057f4ff26f4b706d77ff43d408290a1bfb84f61cb4d1bd1baea6861cdfb" Jul 06 22:15:47.980199 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:15:47.980182 2581 scope.go:117] "RemoveContainer" containerID="2c829057f4ff26f4b706d77ff43d408290a1bfb84f61cb4d1bd1baea6861cdfb" Jul 06 22:15:47.980510 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:15:47.980466 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2c829057f4ff26f4b706d77ff43d408290a1bfb84f61cb4d1bd1baea6861cdfb\": container with ID starting with 2c829057f4ff26f4b706d77ff43d408290a1bfb84f61cb4d1bd1baea6861cdfb not found: ID does not exist" containerID="2c829057f4ff26f4b706d77ff43d408290a1bfb84f61cb4d1bd1baea6861cdfb" Jul 06 22:15:47.980584 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:15:47.980501 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c829057f4ff26f4b706d77ff43d408290a1bfb84f61cb4d1bd1baea6861cdfb"} err="failed to get container status \"2c829057f4ff26f4b706d77ff43d408290a1bfb84f61cb4d1bd1baea6861cdfb\": rpc error: code = NotFound desc = could not find container \"2c829057f4ff26f4b706d77ff43d408290a1bfb84f61cb4d1bd1baea6861cdfb\": container with ID starting with 2c829057f4ff26f4b706d77ff43d408290a1bfb84f61cb4d1bd1baea6861cdfb not found: ID does not exist" Jul 06 22:15:47.999624 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:15:47.999601 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-984zs"] Jul 06 22:15:48.005487 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:15:48.005468 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-984zs"] Jul 06 22:15:48.976275 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:15:48.976240 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="812b077b-2dc2-49e6-8434-1ca50b71c2de" path="/var/lib/kubelet/pods/812b077b-2dc2-49e6-8434-1ca50b71c2de/volumes" Jul 06 22:15:55.044652 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:15:55.044614 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-lswds" Jul 06 22:15:55.045205 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:15:55.044679 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-lswds" Jul 06 22:15:55.067126 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:15:55.067100 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-lswds" Jul 06 22:15:56.028542 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:15:56.028514 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-lswds" Jul 06 22:16:01.199399 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:01.199354 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:16:01.299487 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:01.299455 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:16:03.535390 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:03.535302 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:16:03.671240 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:03.671208 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:16:04.263907 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:04.263868 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:16:04.325225 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:04.325196 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:16:14.185515 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:14.185472 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:16:14.650838 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:14.650751 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:16:14.744556 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:14.744520 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:16:14.843762 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:14.843730 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:16:16.812205 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:16.812175 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-r8gcf"] Jul 06 22:16:16.812557 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:16.812515 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="812b077b-2dc2-49e6-8434-1ca50b71c2de" containerName="registry-server" Jul 06 22:16:16.812557 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:16.812526 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="812b077b-2dc2-49e6-8434-1ca50b71c2de" containerName="registry-server" Jul 06 22:16:16.812622 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:16.812581 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="812b077b-2dc2-49e6-8434-1ca50b71c2de" containerName="registry-server" Jul 06 22:16:16.815590 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:16.815573 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-r8gcf" Jul 06 22:16:16.818285 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:16.818259 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-48fwl\"" Jul 06 22:16:16.818598 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:16.818581 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 06 22:16:16.829531 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:16.829509 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-r8gcf"] Jul 06 22:16:16.967060 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:16.967020 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k9n69\" (UniqueName: \"kubernetes.io/projected/a79d66ac-7935-4124-b79c-355e0cb8b89d-kube-api-access-k9n69\") pod \"dns-operator-controller-manager-864dd78c4c-r8gcf\" (UID: \"a79d66ac-7935-4124-b79c-355e0cb8b89d\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-r8gcf" Jul 06 22:16:17.068444 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:17.068370 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k9n69\" (UniqueName: \"kubernetes.io/projected/a79d66ac-7935-4124-b79c-355e0cb8b89d-kube-api-access-k9n69\") pod \"dns-operator-controller-manager-864dd78c4c-r8gcf\" (UID: \"a79d66ac-7935-4124-b79c-355e0cb8b89d\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-r8gcf" Jul 06 22:16:17.081811 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:17.081780 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k9n69\" (UniqueName: \"kubernetes.io/projected/a79d66ac-7935-4124-b79c-355e0cb8b89d-kube-api-access-k9n69\") pod \"dns-operator-controller-manager-864dd78c4c-r8gcf\" (UID: \"a79d66ac-7935-4124-b79c-355e0cb8b89d\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-r8gcf" Jul 06 22:16:17.125429 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:17.125399 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-r8gcf" Jul 06 22:16:17.292819 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:17.292787 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-r8gcf"] Jul 06 22:16:17.296482 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:16:17.296457 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda79d66ac_7935_4124_b79c_355e0cb8b89d.slice/crio-5aea77b139903fca3bf259fc45cce6e817caacf7c86ffc9fd1d06bcac46ca706 WatchSource:0}: Error finding container 5aea77b139903fca3bf259fc45cce6e817caacf7c86ffc9fd1d06bcac46ca706: Status 404 returned error can't find the container with id 5aea77b139903fca3bf259fc45cce6e817caacf7c86ffc9fd1d06bcac46ca706 Jul 06 22:16:18.101114 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:18.101056 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-r8gcf" event={"ID":"a79d66ac-7935-4124-b79c-355e0cb8b89d","Type":"ContainerStarted","Data":"5aea77b139903fca3bf259fc45cce6e817caacf7c86ffc9fd1d06bcac46ca706"} Jul 06 22:16:18.379257 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:18.379189 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:16:20.111602 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:20.111560 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-r8gcf" event={"ID":"a79d66ac-7935-4124-b79c-355e0cb8b89d","Type":"ContainerStarted","Data":"188479f15026c6605cbdae7eede061205c6165cb4ec8eea30f42f1dbb85ce4a8"} Jul 06 22:16:20.112029 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:20.111670 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-r8gcf" Jul 06 22:16:20.135716 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:20.135666 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-r8gcf" podStartSLOduration=2.060442341 podStartE2EDuration="4.135650112s" totalImagesPullingTime="2.075207771s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:16:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:16:20.132878033 +0000 UTC m=+459.691673373" watchObservedRunningTime="2026-07-06 22:16:20.135650112 +0000 UTC m=+459.694445432" Jul 06 22:16:20.245390 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:20.245361 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-blqqq"] Jul 06 22:16:20.248823 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:20.248805 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-blqqq" Jul 06 22:16:20.251646 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:20.251623 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-tdbf8\"" Jul 06 22:16:20.260849 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:20.260826 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-blqqq"] Jul 06 22:16:20.396739 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:20.396653 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fmvb7\" (UniqueName: \"kubernetes.io/projected/31ff4402-9d57-4d89-96a8-4e0e078456de-kube-api-access-fmvb7\") pod \"limitador-operator-controller-manager-859d7b78bd-blqqq\" (UID: \"31ff4402-9d57-4d89-96a8-4e0e078456de\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-blqqq" Jul 06 22:16:20.497787 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:20.497749 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fmvb7\" (UniqueName: \"kubernetes.io/projected/31ff4402-9d57-4d89-96a8-4e0e078456de-kube-api-access-fmvb7\") pod \"limitador-operator-controller-manager-859d7b78bd-blqqq\" (UID: \"31ff4402-9d57-4d89-96a8-4e0e078456de\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-blqqq" Jul 06 22:16:20.510262 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:20.510230 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fmvb7\" (UniqueName: \"kubernetes.io/projected/31ff4402-9d57-4d89-96a8-4e0e078456de-kube-api-access-fmvb7\") pod \"limitador-operator-controller-manager-859d7b78bd-blqqq\" (UID: \"31ff4402-9d57-4d89-96a8-4e0e078456de\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-blqqq" Jul 06 22:16:20.559230 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:20.559197 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-blqqq" Jul 06 22:16:20.693613 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:20.693586 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-blqqq"] Jul 06 22:16:20.694608 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:16:20.694582 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31ff4402_9d57_4d89_96a8_4e0e078456de.slice/crio-dafd919c629c3c7397a63f6b2ec7f58e20a343c32c8b446037b913610a8a73df WatchSource:0}: Error finding container dafd919c629c3c7397a63f6b2ec7f58e20a343c32c8b446037b913610a8a73df: Status 404 returned error can't find the container with id dafd919c629c3c7397a63f6b2ec7f58e20a343c32c8b446037b913610a8a73df Jul 06 22:16:20.815620 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:20.815588 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:16:20.973624 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:20.973504 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:16:21.117671 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:21.117640 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-blqqq" event={"ID":"31ff4402-9d57-4d89-96a8-4e0e078456de","Type":"ContainerStarted","Data":"dafd919c629c3c7397a63f6b2ec7f58e20a343c32c8b446037b913610a8a73df"} Jul 06 22:16:22.210946 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:22.210906 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:16:23.129223 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:23.129186 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-blqqq" event={"ID":"31ff4402-9d57-4d89-96a8-4e0e078456de","Type":"ContainerStarted","Data":"17a678c3e6f49458e928aade4bf77f32cd12c528fdabf22a198b55deb6c98fe0"} Jul 06 22:16:23.129409 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:23.129292 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-blqqq" Jul 06 22:16:23.156221 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:23.156172 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-blqqq" podStartSLOduration=1.353499109 podStartE2EDuration="3.156157472s" totalImagesPullingTime="1.802658363s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:16:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:16:23.155033379 +0000 UTC m=+462.713828702" watchObservedRunningTime="2026-07-06 22:16:23.156157472 +0000 UTC m=+462.714952793" Jul 06 22:16:25.303393 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:25.303357 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4gcph"] Jul 06 22:16:25.306743 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:25.306725 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4gcph" Jul 06 22:16:25.310045 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:25.310021 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-cdlgd\"" Jul 06 22:16:25.326847 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:25.326809 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4gcph"] Jul 06 22:16:25.397140 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:25.397111 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:16:25.439705 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:25.439681 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/28395927-59c0-4608-9a08-59016c3c97da-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-4gcph\" (UID: \"28395927-59c0-4608-9a08-59016c3c97da\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4gcph" Jul 06 22:16:25.439841 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:25.439716 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wfgvt\" (UniqueName: \"kubernetes.io/projected/28395927-59c0-4608-9a08-59016c3c97da-kube-api-access-wfgvt\") pod \"kuadrant-operator-controller-manager-8684c9ff49-4gcph\" (UID: \"28395927-59c0-4608-9a08-59016c3c97da\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4gcph" Jul 06 22:16:25.540147 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:25.540108 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/28395927-59c0-4608-9a08-59016c3c97da-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-4gcph\" (UID: \"28395927-59c0-4608-9a08-59016c3c97da\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4gcph" Jul 06 22:16:25.540288 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:25.540156 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wfgvt\" (UniqueName: \"kubernetes.io/projected/28395927-59c0-4608-9a08-59016c3c97da-kube-api-access-wfgvt\") pod \"kuadrant-operator-controller-manager-8684c9ff49-4gcph\" (UID: \"28395927-59c0-4608-9a08-59016c3c97da\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4gcph" Jul 06 22:16:25.540511 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:25.540490 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/28395927-59c0-4608-9a08-59016c3c97da-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-4gcph\" (UID: \"28395927-59c0-4608-9a08-59016c3c97da\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4gcph" Jul 06 22:16:25.549674 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:25.549647 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wfgvt\" (UniqueName: \"kubernetes.io/projected/28395927-59c0-4608-9a08-59016c3c97da-kube-api-access-wfgvt\") pod \"kuadrant-operator-controller-manager-8684c9ff49-4gcph\" (UID: \"28395927-59c0-4608-9a08-59016c3c97da\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4gcph" Jul 06 22:16:25.616238 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:25.616167 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4gcph" Jul 06 22:16:25.760714 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:25.760682 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4gcph"] Jul 06 22:16:25.764698 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:16:25.764669 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28395927_59c0_4608_9a08_59016c3c97da.slice/crio-8e808d5dbf4809ab07b00c2e6d7dc318d147d719fe51cbed9ad012a4fe00f2aa WatchSource:0}: Error finding container 8e808d5dbf4809ab07b00c2e6d7dc318d147d719fe51cbed9ad012a4fe00f2aa: Status 404 returned error can't find the container with id 8e808d5dbf4809ab07b00c2e6d7dc318d147d719fe51cbed9ad012a4fe00f2aa Jul 06 22:16:25.801110 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:25.801052 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:16:26.030827 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:26.030792 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:16:26.142907 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:26.142868 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:16:26.146984 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:26.146953 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4gcph" event={"ID":"28395927-59c0-4608-9a08-59016c3c97da","Type":"ContainerStarted","Data":"8e808d5dbf4809ab07b00c2e6d7dc318d147d719fe51cbed9ad012a4fe00f2aa"} Jul 06 22:16:28.530798 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:28.530761 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-4m2sf"] Jul 06 22:16:28.534137 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:28.534118 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-4m2sf" Jul 06 22:16:28.537320 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:28.537301 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-wz46l\"" Jul 06 22:16:28.554396 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:28.554372 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-4m2sf"] Jul 06 22:16:28.667993 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:28.667955 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mc4c2\" (UniqueName: \"kubernetes.io/projected/cfda177d-012f-4dff-ae40-553adf9e1cdb-kube-api-access-mc4c2\") pod \"authorino-operator-75cc8d5cb7-4m2sf\" (UID: \"cfda177d-012f-4dff-ae40-553adf9e1cdb\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-4m2sf" Jul 06 22:16:28.768811 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:28.768775 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mc4c2\" (UniqueName: \"kubernetes.io/projected/cfda177d-012f-4dff-ae40-553adf9e1cdb-kube-api-access-mc4c2\") pod \"authorino-operator-75cc8d5cb7-4m2sf\" (UID: \"cfda177d-012f-4dff-ae40-553adf9e1cdb\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-4m2sf" Jul 06 22:16:28.791876 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:28.791810 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mc4c2\" (UniqueName: \"kubernetes.io/projected/cfda177d-012f-4dff-ae40-553adf9e1cdb-kube-api-access-mc4c2\") pod \"authorino-operator-75cc8d5cb7-4m2sf\" (UID: \"cfda177d-012f-4dff-ae40-553adf9e1cdb\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-4m2sf" Jul 06 22:16:28.844525 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:28.844493 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-4m2sf" Jul 06 22:16:28.982618 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:28.982577 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-4m2sf"] Jul 06 22:16:28.984913 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:16:28.984884 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcfda177d_012f_4dff_ae40_553adf9e1cdb.slice/crio-79eabae972c35be76509603d51f2e2c49f3d4b1db27a9f3530e52a808b3c30b9 WatchSource:0}: Error finding container 79eabae972c35be76509603d51f2e2c49f3d4b1db27a9f3530e52a808b3c30b9: Status 404 returned error can't find the container with id 79eabae972c35be76509603d51f2e2c49f3d4b1db27a9f3530e52a808b3c30b9 Jul 06 22:16:29.162366 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:29.162278 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-4m2sf" event={"ID":"cfda177d-012f-4dff-ae40-553adf9e1cdb","Type":"ContainerStarted","Data":"79eabae972c35be76509603d51f2e2c49f3d4b1db27a9f3530e52a808b3c30b9"} Jul 06 22:16:29.293983 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:29.293949 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:16:30.594615 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:30.594569 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:16:30.695192 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:30.695104 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:16:31.120216 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:31.120182 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-r8gcf" Jul 06 22:16:33.186236 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:33.186198 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-4m2sf" event={"ID":"cfda177d-012f-4dff-ae40-553adf9e1cdb","Type":"ContainerStarted","Data":"6306c20f25916d938a4bdd068bf03eaa37c36beff19c517f500b0b256c83ea4d"} Jul 06 22:16:33.186673 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:33.186403 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-4m2sf" Jul 06 22:16:33.187836 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:33.187814 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4gcph" event={"ID":"28395927-59c0-4608-9a08-59016c3c97da","Type":"ContainerStarted","Data":"c42fb521d5718cb319c8e9e01ec34a93f9722c6310ffcfea87982997d04b2829"} Jul 06 22:16:33.187951 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:33.187937 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4gcph" Jul 06 22:16:33.207444 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:33.207394 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-4m2sf" podStartSLOduration=1.440791862 podStartE2EDuration="5.207381399s" totalImagesPullingTime="3.766589537s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:16:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:16:33.20535136 +0000 UTC m=+472.764146694" watchObservedRunningTime="2026-07-06 22:16:33.207381399 +0000 UTC m=+472.766176719" Jul 06 22:16:33.226087 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:33.226016 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4gcph" podStartSLOduration=1.227862867 podStartE2EDuration="8.226002719s" totalImagesPullingTime="6.998139852s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:16:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:16:33.224802588 +0000 UTC m=+472.783597910" watchObservedRunningTime="2026-07-06 22:16:33.226002719 +0000 UTC m=+472.784798040" Jul 06 22:16:34.139341 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:34.139310 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-blqqq" Jul 06 22:16:34.431295 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:34.431268 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:16:34.679529 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:34.679497 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:16:35.772530 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:35.772497 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:16:36.175921 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:36.175886 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:16:37.787520 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:37.787465 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:16:37.887425 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:37.887384 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:16:44.195035 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:44.195003 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4gcph" Jul 06 22:16:44.195488 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:44.195091 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-4m2sf" Jul 06 22:16:45.616275 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:45.616241 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-j97rb"] Jul 06 22:16:45.620782 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:45.620763 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-j97rb" Jul 06 22:16:45.635077 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:45.635036 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-j97rb"] Jul 06 22:16:45.707168 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:45.707120 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fcef2cf3-14af-4e41-91a9-9b4bd1290383-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-j97rb\" (UID: \"fcef2cf3-14af-4e41-91a9-9b4bd1290383\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-j97rb" Jul 06 22:16:45.707347 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:45.707185 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c2nw8\" (UniqueName: \"kubernetes.io/projected/fcef2cf3-14af-4e41-91a9-9b4bd1290383-kube-api-access-c2nw8\") pod \"kuadrant-operator-controller-manager-55467d9f7-j97rb\" (UID: \"fcef2cf3-14af-4e41-91a9-9b4bd1290383\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-j97rb" Jul 06 22:16:45.808455 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:45.808415 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fcef2cf3-14af-4e41-91a9-9b4bd1290383-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-j97rb\" (UID: \"fcef2cf3-14af-4e41-91a9-9b4bd1290383\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-j97rb" Jul 06 22:16:45.808621 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:45.808466 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c2nw8\" (UniqueName: \"kubernetes.io/projected/fcef2cf3-14af-4e41-91a9-9b4bd1290383-kube-api-access-c2nw8\") pod \"kuadrant-operator-controller-manager-55467d9f7-j97rb\" (UID: \"fcef2cf3-14af-4e41-91a9-9b4bd1290383\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-j97rb" Jul 06 22:16:45.808798 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:45.808778 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fcef2cf3-14af-4e41-91a9-9b4bd1290383-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-j97rb\" (UID: \"fcef2cf3-14af-4e41-91a9-9b4bd1290383\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-j97rb" Jul 06 22:16:45.827359 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:45.827332 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c2nw8\" (UniqueName: \"kubernetes.io/projected/fcef2cf3-14af-4e41-91a9-9b4bd1290383-kube-api-access-c2nw8\") pod \"kuadrant-operator-controller-manager-55467d9f7-j97rb\" (UID: \"fcef2cf3-14af-4e41-91a9-9b4bd1290383\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-j97rb" Jul 06 22:16:45.920863 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:45.920772 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-j97rb"] Jul 06 22:16:45.921098 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:45.921079 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-j97rb" Jul 06 22:16:45.930295 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:45.930261 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-j97rb"] Jul 06 22:16:45.937585 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:45.937562 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4gcph"] Jul 06 22:16:45.937798 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:45.937742 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4gcph" podUID="28395927-59c0-4608-9a08-59016c3c97da" containerName="manager" containerID="cri-o://c42fb521d5718cb319c8e9e01ec34a93f9722c6310ffcfea87982997d04b2829" gracePeriod=2 Jul 06 22:16:45.942461 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:45.942400 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4gcph"] Jul 06 22:16:45.952039 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:45.952002 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-c7hv8"] Jul 06 22:16:45.953359 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:45.953203 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28395927-59c0-4608-9a08-59016c3c97da" containerName="manager" Jul 06 22:16:45.953359 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:45.953225 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="28395927-59c0-4608-9a08-59016c3c97da" containerName="manager" Jul 06 22:16:45.954059 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:45.953668 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="28395927-59c0-4608-9a08-59016c3c97da" containerName="manager" Jul 06 22:16:45.956764 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:45.956746 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-blqqq"] Jul 06 22:16:45.956888 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:45.956869 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-c7hv8" Jul 06 22:16:45.957121 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:45.957020 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-blqqq" podUID="31ff4402-9d57-4d89-96a8-4e0e078456de" containerName="manager" containerID="cri-o://17a678c3e6f49458e928aade4bf77f32cd12c528fdabf22a198b55deb6c98fe0" gracePeriod=2 Jul 06 22:16:45.959561 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:16:45.959364 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-4gcph\" is forbidden: User \"system:node:ip-10-0-137-101.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-101.ec2.internal' and this object" podUID="28395927-59c0-4608-9a08-59016c3c97da" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4gcph" Jul 06 22:16:45.965626 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:45.965604 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-blqqq"] Jul 06 22:16:45.967512 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:45.967486 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5hf5j"] Jul 06 22:16:45.967999 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:45.967980 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="31ff4402-9d57-4d89-96a8-4e0e078456de" containerName="manager" Jul 06 22:16:45.968200 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:45.968003 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="31ff4402-9d57-4d89-96a8-4e0e078456de" containerName="manager" Jul 06 22:16:45.968200 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:45.968142 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="31ff4402-9d57-4d89-96a8-4e0e078456de" containerName="manager" Jul 06 22:16:45.971633 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:45.971612 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-c7hv8"] Jul 06 22:16:45.971719 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:45.971690 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5hf5j" Jul 06 22:16:45.982091 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:45.982051 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5hf5j"] Jul 06 22:16:45.988180 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:16:45.988145 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-blqqq\" is forbidden: User \"system:node:ip-10-0-137-101.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-101.ec2.internal' and this object" podUID="31ff4402-9d57-4d89-96a8-4e0e078456de" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-blqqq" Jul 06 22:16:45.990799 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:16:45.990776 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-4gcph\" is forbidden: User \"system:node:ip-10-0-137-101.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-101.ec2.internal' and this object" podUID="28395927-59c0-4608-9a08-59016c3c97da" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4gcph" Jul 06 22:16:46.010285 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:46.010254 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/099f6a13-fbc6-40ac-ae8a-aee476b43e4a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-5hf5j\" (UID: \"099f6a13-fbc6-40ac-ae8a-aee476b43e4a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5hf5j" Jul 06 22:16:46.010419 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:46.010296 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/127848fa-e307-4d8f-b0d1-bcf9b7dc4897-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-c7hv8\" (UID: \"127848fa-e307-4d8f-b0d1-bcf9b7dc4897\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-c7hv8" Jul 06 22:16:46.010419 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:46.010321 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tq7zq\" (UniqueName: \"kubernetes.io/projected/127848fa-e307-4d8f-b0d1-bcf9b7dc4897-kube-api-access-tq7zq\") pod \"kuadrant-operator-controller-manager-55467d9f7-c7hv8\" (UID: \"127848fa-e307-4d8f-b0d1-bcf9b7dc4897\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-c7hv8" Jul 06 22:16:46.010419 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:46.010387 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4fbxb\" (UniqueName: \"kubernetes.io/projected/099f6a13-fbc6-40ac-ae8a-aee476b43e4a-kube-api-access-4fbxb\") pod \"kuadrant-operator-controller-manager-8684c9ff49-5hf5j\" (UID: \"099f6a13-fbc6-40ac-ae8a-aee476b43e4a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5hf5j" Jul 06 22:16:46.111611 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:46.111579 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/127848fa-e307-4d8f-b0d1-bcf9b7dc4897-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-c7hv8\" (UID: \"127848fa-e307-4d8f-b0d1-bcf9b7dc4897\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-c7hv8" Jul 06 22:16:46.111780 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:46.111619 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tq7zq\" (UniqueName: \"kubernetes.io/projected/127848fa-e307-4d8f-b0d1-bcf9b7dc4897-kube-api-access-tq7zq\") pod \"kuadrant-operator-controller-manager-55467d9f7-c7hv8\" (UID: \"127848fa-e307-4d8f-b0d1-bcf9b7dc4897\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-c7hv8" Jul 06 22:16:46.111780 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:46.111650 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4fbxb\" (UniqueName: \"kubernetes.io/projected/099f6a13-fbc6-40ac-ae8a-aee476b43e4a-kube-api-access-4fbxb\") pod \"kuadrant-operator-controller-manager-8684c9ff49-5hf5j\" (UID: \"099f6a13-fbc6-40ac-ae8a-aee476b43e4a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5hf5j" Jul 06 22:16:46.111780 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:46.111710 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/099f6a13-fbc6-40ac-ae8a-aee476b43e4a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-5hf5j\" (UID: \"099f6a13-fbc6-40ac-ae8a-aee476b43e4a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5hf5j" Jul 06 22:16:46.112036 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:46.112017 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/099f6a13-fbc6-40ac-ae8a-aee476b43e4a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-5hf5j\" (UID: \"099f6a13-fbc6-40ac-ae8a-aee476b43e4a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5hf5j" Jul 06 22:16:46.112036 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:46.112024 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/127848fa-e307-4d8f-b0d1-bcf9b7dc4897-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-c7hv8\" (UID: \"127848fa-e307-4d8f-b0d1-bcf9b7dc4897\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-c7hv8" Jul 06 22:16:46.120701 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:46.120671 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4fbxb\" (UniqueName: \"kubernetes.io/projected/099f6a13-fbc6-40ac-ae8a-aee476b43e4a-kube-api-access-4fbxb\") pod \"kuadrant-operator-controller-manager-8684c9ff49-5hf5j\" (UID: \"099f6a13-fbc6-40ac-ae8a-aee476b43e4a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5hf5j" Jul 06 22:16:46.123348 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:46.123320 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tq7zq\" (UniqueName: \"kubernetes.io/projected/127848fa-e307-4d8f-b0d1-bcf9b7dc4897-kube-api-access-tq7zq\") pod \"kuadrant-operator-controller-manager-55467d9f7-c7hv8\" (UID: \"127848fa-e307-4d8f-b0d1-bcf9b7dc4897\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-c7hv8" Jul 06 22:16:46.184112 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:46.184052 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4gcph" Jul 06 22:16:46.187124 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:16:46.187098 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-blqqq\" is forbidden: User \"system:node:ip-10-0-137-101.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-101.ec2.internal' and this object" podUID="31ff4402-9d57-4d89-96a8-4e0e078456de" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-blqqq" Jul 06 22:16:46.189345 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:16:46.189323 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-4gcph\" is forbidden: User \"system:node:ip-10-0-137-101.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-101.ec2.internal' and this object" podUID="28395927-59c0-4608-9a08-59016c3c97da" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4gcph" Jul 06 22:16:46.191531 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:46.191514 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-blqqq" Jul 06 22:16:46.193709 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:16:46.193683 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-4gcph\" is forbidden: User \"system:node:ip-10-0-137-101.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-101.ec2.internal' and this object" podUID="28395927-59c0-4608-9a08-59016c3c97da" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4gcph" Jul 06 22:16:46.195500 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:16:46.195481 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-blqqq\" is forbidden: User \"system:node:ip-10-0-137-101.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-101.ec2.internal' and this object" podUID="31ff4402-9d57-4d89-96a8-4e0e078456de" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-blqqq" Jul 06 22:16:46.212824 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:46.212799 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/28395927-59c0-4608-9a08-59016c3c97da-extensions-socket-volume\") pod \"28395927-59c0-4608-9a08-59016c3c97da\" (UID: \"28395927-59c0-4608-9a08-59016c3c97da\") " Jul 06 22:16:46.212934 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:46.212874 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wfgvt\" (UniqueName: \"kubernetes.io/projected/28395927-59c0-4608-9a08-59016c3c97da-kube-api-access-wfgvt\") pod \"28395927-59c0-4608-9a08-59016c3c97da\" (UID: \"28395927-59c0-4608-9a08-59016c3c97da\") " Jul 06 22:16:46.213151 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:46.213131 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28395927-59c0-4608-9a08-59016c3c97da-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "28395927-59c0-4608-9a08-59016c3c97da" (UID: "28395927-59c0-4608-9a08-59016c3c97da"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:16:46.214790 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:46.214770 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/28395927-59c0-4608-9a08-59016c3c97da-kube-api-access-wfgvt" (OuterVolumeSpecName: "kube-api-access-wfgvt") pod "28395927-59c0-4608-9a08-59016c3c97da" (UID: "28395927-59c0-4608-9a08-59016c3c97da"). InnerVolumeSpecName "kube-api-access-wfgvt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 22:16:46.246963 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:46.246934 2581 generic.go:356] "Generic (PLEG): container finished" podID="31ff4402-9d57-4d89-96a8-4e0e078456de" containerID="17a678c3e6f49458e928aade4bf77f32cd12c528fdabf22a198b55deb6c98fe0" exitCode=0 Jul 06 22:16:46.247159 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:46.246974 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-blqqq" Jul 06 22:16:46.247159 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:46.247033 2581 scope.go:117] "RemoveContainer" containerID="17a678c3e6f49458e928aade4bf77f32cd12c528fdabf22a198b55deb6c98fe0" Jul 06 22:16:46.248371 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:46.248353 2581 generic.go:356] "Generic (PLEG): container finished" podID="28395927-59c0-4608-9a08-59016c3c97da" containerID="c42fb521d5718cb319c8e9e01ec34a93f9722c6310ffcfea87982997d04b2829" exitCode=0 Jul 06 22:16:46.248453 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:46.248400 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4gcph" Jul 06 22:16:46.249570 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:16:46.249539 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-4gcph\" is forbidden: User \"system:node:ip-10-0-137-101.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-101.ec2.internal' and this object" podUID="28395927-59c0-4608-9a08-59016c3c97da" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4gcph" Jul 06 22:16:46.251829 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:16:46.251799 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-blqqq\" is forbidden: User \"system:node:ip-10-0-137-101.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-101.ec2.internal' and this object" podUID="31ff4402-9d57-4d89-96a8-4e0e078456de" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-blqqq" Jul 06 22:16:46.254452 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:16:46.254427 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-blqqq\" is forbidden: User \"system:node:ip-10-0-137-101.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-101.ec2.internal' and this object" podUID="31ff4402-9d57-4d89-96a8-4e0e078456de" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-blqqq" Jul 06 22:16:46.255535 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:46.255516 2581 scope.go:117] "RemoveContainer" containerID="17a678c3e6f49458e928aade4bf77f32cd12c528fdabf22a198b55deb6c98fe0" Jul 06 22:16:46.255915 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:16:46.255774 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"17a678c3e6f49458e928aade4bf77f32cd12c528fdabf22a198b55deb6c98fe0\": container with ID starting with 17a678c3e6f49458e928aade4bf77f32cd12c528fdabf22a198b55deb6c98fe0 not found: ID does not exist" containerID="17a678c3e6f49458e928aade4bf77f32cd12c528fdabf22a198b55deb6c98fe0" Jul 06 22:16:46.255915 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:46.255806 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"17a678c3e6f49458e928aade4bf77f32cd12c528fdabf22a198b55deb6c98fe0"} err="failed to get container status \"17a678c3e6f49458e928aade4bf77f32cd12c528fdabf22a198b55deb6c98fe0\": rpc error: code = NotFound desc = could not find container \"17a678c3e6f49458e928aade4bf77f32cd12c528fdabf22a198b55deb6c98fe0\": container with ID starting with 17a678c3e6f49458e928aade4bf77f32cd12c528fdabf22a198b55deb6c98fe0 not found: ID does not exist" Jul 06 22:16:46.255915 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:46.255823 2581 scope.go:117] "RemoveContainer" containerID="c42fb521d5718cb319c8e9e01ec34a93f9722c6310ffcfea87982997d04b2829" Jul 06 22:16:46.256578 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:16:46.256548 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-4gcph\" is forbidden: User \"system:node:ip-10-0-137-101.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-101.ec2.internal' and this object" podUID="28395927-59c0-4608-9a08-59016c3c97da" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4gcph" Jul 06 22:16:46.258836 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:16:46.258814 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-blqqq\" is forbidden: User \"system:node:ip-10-0-137-101.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-101.ec2.internal' and this object" podUID="31ff4402-9d57-4d89-96a8-4e0e078456de" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-blqqq" Jul 06 22:16:46.260919 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:16:46.260897 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-4gcph\" is forbidden: User \"system:node:ip-10-0-137-101.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-101.ec2.internal' and this object" podUID="28395927-59c0-4608-9a08-59016c3c97da" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4gcph" Jul 06 22:16:46.264039 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:46.264022 2581 scope.go:117] "RemoveContainer" containerID="c42fb521d5718cb319c8e9e01ec34a93f9722c6310ffcfea87982997d04b2829" Jul 06 22:16:46.264297 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:16:46.264280 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c42fb521d5718cb319c8e9e01ec34a93f9722c6310ffcfea87982997d04b2829\": container with ID starting with c42fb521d5718cb319c8e9e01ec34a93f9722c6310ffcfea87982997d04b2829 not found: ID does not exist" containerID="c42fb521d5718cb319c8e9e01ec34a93f9722c6310ffcfea87982997d04b2829" Jul 06 22:16:46.264357 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:46.264303 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c42fb521d5718cb319c8e9e01ec34a93f9722c6310ffcfea87982997d04b2829"} err="failed to get container status \"c42fb521d5718cb319c8e9e01ec34a93f9722c6310ffcfea87982997d04b2829\": rpc error: code = NotFound desc = could not find container \"c42fb521d5718cb319c8e9e01ec34a93f9722c6310ffcfea87982997d04b2829\": container with ID starting with c42fb521d5718cb319c8e9e01ec34a93f9722c6310ffcfea87982997d04b2829 not found: ID does not exist" Jul 06 22:16:46.281216 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:46.281189 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:16:46.313531 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:46.313503 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fmvb7\" (UniqueName: \"kubernetes.io/projected/31ff4402-9d57-4d89-96a8-4e0e078456de-kube-api-access-fmvb7\") pod \"31ff4402-9d57-4d89-96a8-4e0e078456de\" (UID: \"31ff4402-9d57-4d89-96a8-4e0e078456de\") " Jul 06 22:16:46.313680 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:46.313667 2581 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/28395927-59c0-4608-9a08-59016c3c97da-extensions-socket-volume\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:16:46.313726 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:46.313681 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wfgvt\" (UniqueName: \"kubernetes.io/projected/28395927-59c0-4608-9a08-59016c3c97da-kube-api-access-wfgvt\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:16:46.315507 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:46.315487 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/31ff4402-9d57-4d89-96a8-4e0e078456de-kube-api-access-fmvb7" (OuterVolumeSpecName: "kube-api-access-fmvb7") pod "31ff4402-9d57-4d89-96a8-4e0e078456de" (UID: "31ff4402-9d57-4d89-96a8-4e0e078456de"). InnerVolumeSpecName "kube-api-access-fmvb7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 22:16:46.364764 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:46.364736 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-c7hv8" Jul 06 22:16:46.370247 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:46.370232 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5hf5j" Jul 06 22:16:46.375492 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:46.375469 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:16:46.415347 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:46.415307 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fmvb7\" (UniqueName: \"kubernetes.io/projected/31ff4402-9d57-4d89-96a8-4e0e078456de-kube-api-access-fmvb7\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:16:46.470865 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:46.470828 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:16:46.511980 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:46.511873 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-c7hv8"] Jul 06 22:16:46.514458 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:16:46.514431 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod127848fa_e307_4d8f_b0d1_bcf9b7dc4897.slice/crio-aac7fda5f8edca88533f617048c6665d70391e68e65048fd65662ab1ba12a9f8 WatchSource:0}: Error finding container aac7fda5f8edca88533f617048c6665d70391e68e65048fd65662ab1ba12a9f8: Status 404 returned error can't find the container with id aac7fda5f8edca88533f617048c6665d70391e68e65048fd65662ab1ba12a9f8 Jul 06 22:16:46.552597 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:46.550817 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5hf5j"] Jul 06 22:16:46.566982 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:16:46.566953 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-4gcph\" is forbidden: User \"system:node:ip-10-0-137-101.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-101.ec2.internal' and this object" podUID="28395927-59c0-4608-9a08-59016c3c97da" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-4gcph" Jul 06 22:16:46.569049 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:16:46.569025 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-blqqq\" is forbidden: User \"system:node:ip-10-0-137-101.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-101.ec2.internal' and this object" podUID="31ff4402-9d57-4d89-96a8-4e0e078456de" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-blqqq" Jul 06 22:16:46.978960 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:46.978928 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="28395927-59c0-4608-9a08-59016c3c97da" path="/var/lib/kubelet/pods/28395927-59c0-4608-9a08-59016c3c97da/volumes" Jul 06 22:16:46.979345 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:46.979291 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="31ff4402-9d57-4d89-96a8-4e0e078456de" path="/var/lib/kubelet/pods/31ff4402-9d57-4d89-96a8-4e0e078456de/volumes" Jul 06 22:16:47.254624 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:47.254581 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5hf5j" event={"ID":"099f6a13-fbc6-40ac-ae8a-aee476b43e4a","Type":"ContainerStarted","Data":"50e5c0d05e179f7a8cdf088bd5ae0f991c830f8ae0954105ff38e2dcbae5e8b2"} Jul 06 22:16:47.254830 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:47.254650 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5hf5j" event={"ID":"099f6a13-fbc6-40ac-ae8a-aee476b43e4a","Type":"ContainerStarted","Data":"5535bc1a03741aa8af09f6b99115797945b1d4f9c17ac9e6ce292a38b1b1e72b"} Jul 06 22:16:47.254830 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:47.254778 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5hf5j" Jul 06 22:16:47.256191 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:47.256163 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-c7hv8" event={"ID":"127848fa-e307-4d8f-b0d1-bcf9b7dc4897","Type":"ContainerStarted","Data":"841905d736b0d21f1beab130bfbfa166481f8229cf99e7a35d9406a0a0c02f09"} Jul 06 22:16:47.256287 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:47.256211 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-c7hv8" event={"ID":"127848fa-e307-4d8f-b0d1-bcf9b7dc4897","Type":"ContainerStarted","Data":"aac7fda5f8edca88533f617048c6665d70391e68e65048fd65662ab1ba12a9f8"} Jul 06 22:16:47.256287 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:47.256233 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-c7hv8" Jul 06 22:16:47.279963 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:47.279922 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5hf5j" podStartSLOduration=2.279910093 podStartE2EDuration="2.279910093s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:16:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:16:47.278461571 +0000 UTC m=+486.837256889" watchObservedRunningTime="2026-07-06 22:16:47.279910093 +0000 UTC m=+486.838705414" Jul 06 22:16:47.306980 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:47.306937 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-c7hv8" podStartSLOduration=2.306924295 podStartE2EDuration="2.306924295s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:16:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:16:47.30502049 +0000 UTC m=+486.863815824" watchObservedRunningTime="2026-07-06 22:16:47.306924295 +0000 UTC m=+486.865719615" Jul 06 22:16:48.466098 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:16:48.466043 2581 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-55467d9f7-j97rb_kuadrant-system_fcef2cf3-14af-4e41-91a9-9b4bd1290383_0(70cab4c38ebc4e6cfeca387888434fe25f862f58d9e63829c320058753bab7c9): error adding pod kuadrant-system_kuadrant-operator-controller-manager-55467d9f7-j97rb to CNI network \"multus-cni-network\": plugin type=\"multus-shim\" name=\"multus-cni-network\" failed (add): CmdAdd (shim): failed to send CNI request: Post \"http://dummy/cni\": EOF: StdinData: {\"auxiliaryCNIChainName\":\"vendor-cni-chain\",\"binDir\":\"/var/lib/cni/bin\",\"clusterNetwork\":\"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf\",\"cniVersion\":\"0.3.1\",\"daemonSocketDir\":\"/run/multus/socket\",\"globalNamespaces\":\"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv\",\"logLevel\":\"verbose\",\"logToStderr\":true,\"name\":\"multus-cni-network\",\"namespaceIsolation\":true,\"type\":\"multus-shim\"}" Jul 06 22:16:48.466376 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:16:48.466125 2581 kuberuntime_sandbox.go:71] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-55467d9f7-j97rb_kuadrant-system_fcef2cf3-14af-4e41-91a9-9b4bd1290383_0(70cab4c38ebc4e6cfeca387888434fe25f862f58d9e63829c320058753bab7c9): error adding pod kuadrant-system_kuadrant-operator-controller-manager-55467d9f7-j97rb to CNI network \"multus-cni-network\": plugin type=\"multus-shim\" name=\"multus-cni-network\" failed (add): CmdAdd (shim): failed to send CNI request: Post \"http://dummy/cni\": EOF: StdinData: {\"auxiliaryCNIChainName\":\"vendor-cni-chain\",\"binDir\":\"/var/lib/cni/bin\",\"clusterNetwork\":\"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf\",\"cniVersion\":\"0.3.1\",\"daemonSocketDir\":\"/run/multus/socket\",\"globalNamespaces\":\"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv\",\"logLevel\":\"verbose\",\"logToStderr\":true,\"name\":\"multus-cni-network\",\"namespaceIsolation\":true,\"type\":\"multus-shim\"}" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-j97rb" Jul 06 22:16:48.873391 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:48.873301 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:16:48.974263 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:48.974215 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:16:49.267076 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:49.267046 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-j97rb" Jul 06 22:16:49.274897 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:49.274875 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-j97rb" Jul 06 22:16:49.277694 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:16:49.277673 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-j97rb\" is forbidden: User \"system:node:ip-10-0-137-101.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-101.ec2.internal' and this object" podUID="fcef2cf3-14af-4e41-91a9-9b4bd1290383" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-j97rb" Jul 06 22:16:49.340221 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:49.340191 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c2nw8\" (UniqueName: \"kubernetes.io/projected/fcef2cf3-14af-4e41-91a9-9b4bd1290383-kube-api-access-c2nw8\") pod \"fcef2cf3-14af-4e41-91a9-9b4bd1290383\" (UID: \"fcef2cf3-14af-4e41-91a9-9b4bd1290383\") " Jul 06 22:16:49.340309 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:49.340295 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fcef2cf3-14af-4e41-91a9-9b4bd1290383-extensions-socket-volume\") pod \"fcef2cf3-14af-4e41-91a9-9b4bd1290383\" (UID: \"fcef2cf3-14af-4e41-91a9-9b4bd1290383\") " Jul 06 22:16:49.340552 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:49.340530 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fcef2cf3-14af-4e41-91a9-9b4bd1290383-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "fcef2cf3-14af-4e41-91a9-9b4bd1290383" (UID: "fcef2cf3-14af-4e41-91a9-9b4bd1290383"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:16:49.342393 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:49.342366 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fcef2cf3-14af-4e41-91a9-9b4bd1290383-kube-api-access-c2nw8" (OuterVolumeSpecName: "kube-api-access-c2nw8") pod "fcef2cf3-14af-4e41-91a9-9b4bd1290383" (UID: "fcef2cf3-14af-4e41-91a9-9b4bd1290383"). InnerVolumeSpecName "kube-api-access-c2nw8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 22:16:49.441925 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:49.441891 2581 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fcef2cf3-14af-4e41-91a9-9b4bd1290383-extensions-socket-volume\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:16:49.441925 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:49.441919 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c2nw8\" (UniqueName: \"kubernetes.io/projected/fcef2cf3-14af-4e41-91a9-9b4bd1290383-kube-api-access-c2nw8\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:16:50.271306 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:50.271271 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-j97rb" Jul 06 22:16:50.273378 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:16:50.273346 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-j97rb\" is forbidden: User \"system:node:ip-10-0-137-101.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-101.ec2.internal' and this object" podUID="fcef2cf3-14af-4e41-91a9-9b4bd1290383" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-j97rb" Jul 06 22:16:50.279283 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:16:50.279255 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-j97rb\" is forbidden: User \"system:node:ip-10-0-137-101.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-101.ec2.internal' and this object" podUID="fcef2cf3-14af-4e41-91a9-9b4bd1290383" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-j97rb" Jul 06 22:16:50.974265 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:16:50.974227 2581 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-j97rb\" is forbidden: User \"system:node:ip-10-0-137-101.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-101.ec2.internal' and this object" podUID="fcef2cf3-14af-4e41-91a9-9b4bd1290383" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-j97rb" Jul 06 22:16:50.975825 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:50.975801 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fcef2cf3-14af-4e41-91a9-9b4bd1290383" path="/var/lib/kubelet/pods/fcef2cf3-14af-4e41-91a9-9b4bd1290383/volumes" Jul 06 22:16:51.744124 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:51.744083 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:16:51.844846 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:51.844814 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:16:55.251933 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:55.251892 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:16:55.352239 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:55.352201 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:16:58.265636 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:58.265604 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5hf5j" Jul 06 22:16:58.266024 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:58.265680 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-c7hv8" Jul 06 22:16:58.341797 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:58.341769 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5hf5j"] Jul 06 22:16:58.341981 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:58.341957 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5hf5j" podUID="099f6a13-fbc6-40ac-ae8a-aee476b43e4a" containerName="manager" containerID="cri-o://50e5c0d05e179f7a8cdf088bd5ae0f991c830f8ae0954105ff38e2dcbae5e8b2" gracePeriod=10 Jul 06 22:16:58.591161 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:58.591131 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5hf5j" Jul 06 22:16:58.614916 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:58.614889 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-njfz2"] Jul 06 22:16:58.615312 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:58.615296 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="099f6a13-fbc6-40ac-ae8a-aee476b43e4a" containerName="manager" Jul 06 22:16:58.615404 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:58.615315 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="099f6a13-fbc6-40ac-ae8a-aee476b43e4a" containerName="manager" Jul 06 22:16:58.615404 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:58.615397 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="099f6a13-fbc6-40ac-ae8a-aee476b43e4a" containerName="manager" Jul 06 22:16:58.618985 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:58.618964 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-njfz2" Jul 06 22:16:58.629918 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:58.629892 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-njfz2"] Jul 06 22:16:58.721212 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:58.721185 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/099f6a13-fbc6-40ac-ae8a-aee476b43e4a-extensions-socket-volume\") pod \"099f6a13-fbc6-40ac-ae8a-aee476b43e4a\" (UID: \"099f6a13-fbc6-40ac-ae8a-aee476b43e4a\") " Jul 06 22:16:58.721379 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:58.721223 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4fbxb\" (UniqueName: \"kubernetes.io/projected/099f6a13-fbc6-40ac-ae8a-aee476b43e4a-kube-api-access-4fbxb\") pod \"099f6a13-fbc6-40ac-ae8a-aee476b43e4a\" (UID: \"099f6a13-fbc6-40ac-ae8a-aee476b43e4a\") " Jul 06 22:16:58.721471 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:58.721449 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7993f081-3f17-4868-bfc3-285e366f68fb-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-njfz2\" (UID: \"7993f081-3f17-4868-bfc3-285e366f68fb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-njfz2" Jul 06 22:16:58.721531 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:58.721512 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xvwfj\" (UniqueName: \"kubernetes.io/projected/7993f081-3f17-4868-bfc3-285e366f68fb-kube-api-access-xvwfj\") pod \"kuadrant-operator-controller-manager-86878c769f-njfz2\" (UID: \"7993f081-3f17-4868-bfc3-285e366f68fb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-njfz2" Jul 06 22:16:58.721630 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:58.721612 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/099f6a13-fbc6-40ac-ae8a-aee476b43e4a-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "099f6a13-fbc6-40ac-ae8a-aee476b43e4a" (UID: "099f6a13-fbc6-40ac-ae8a-aee476b43e4a"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:16:58.723260 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:58.723230 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/099f6a13-fbc6-40ac-ae8a-aee476b43e4a-kube-api-access-4fbxb" (OuterVolumeSpecName: "kube-api-access-4fbxb") pod "099f6a13-fbc6-40ac-ae8a-aee476b43e4a" (UID: "099f6a13-fbc6-40ac-ae8a-aee476b43e4a"). InnerVolumeSpecName "kube-api-access-4fbxb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 22:16:58.822601 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:58.822522 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7993f081-3f17-4868-bfc3-285e366f68fb-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-njfz2\" (UID: \"7993f081-3f17-4868-bfc3-285e366f68fb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-njfz2" Jul 06 22:16:58.822601 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:58.822567 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xvwfj\" (UniqueName: \"kubernetes.io/projected/7993f081-3f17-4868-bfc3-285e366f68fb-kube-api-access-xvwfj\") pod \"kuadrant-operator-controller-manager-86878c769f-njfz2\" (UID: \"7993f081-3f17-4868-bfc3-285e366f68fb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-njfz2" Jul 06 22:16:58.822601 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:58.822602 2581 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/099f6a13-fbc6-40ac-ae8a-aee476b43e4a-extensions-socket-volume\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:16:58.822831 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:58.822624 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4fbxb\" (UniqueName: \"kubernetes.io/projected/099f6a13-fbc6-40ac-ae8a-aee476b43e4a-kube-api-access-4fbxb\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:16:58.822900 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:58.822879 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7993f081-3f17-4868-bfc3-285e366f68fb-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-njfz2\" (UID: \"7993f081-3f17-4868-bfc3-285e366f68fb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-njfz2" Jul 06 22:16:58.831463 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:58.831436 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xvwfj\" (UniqueName: \"kubernetes.io/projected/7993f081-3f17-4868-bfc3-285e366f68fb-kube-api-access-xvwfj\") pod \"kuadrant-operator-controller-manager-86878c769f-njfz2\" (UID: \"7993f081-3f17-4868-bfc3-285e366f68fb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-njfz2" Jul 06 22:16:58.931542 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:58.931503 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-njfz2" Jul 06 22:16:59.079664 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:59.079586 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-njfz2"] Jul 06 22:16:59.083350 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:16:59.083326 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7993f081_3f17_4868_bfc3_285e366f68fb.slice/crio-e7e1d7565f3fe00aa6c41c3f8e2200115702ad22a213c66403d1454a6a39fd93 WatchSource:0}: Error finding container e7e1d7565f3fe00aa6c41c3f8e2200115702ad22a213c66403d1454a6a39fd93: Status 404 returned error can't find the container with id e7e1d7565f3fe00aa6c41c3f8e2200115702ad22a213c66403d1454a6a39fd93 Jul 06 22:16:59.313671 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:59.313634 2581 generic.go:356] "Generic (PLEG): container finished" podID="099f6a13-fbc6-40ac-ae8a-aee476b43e4a" containerID="50e5c0d05e179f7a8cdf088bd5ae0f991c830f8ae0954105ff38e2dcbae5e8b2" exitCode=0 Jul 06 22:16:59.314176 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:59.313679 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5hf5j" Jul 06 22:16:59.314176 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:59.313717 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5hf5j" event={"ID":"099f6a13-fbc6-40ac-ae8a-aee476b43e4a","Type":"ContainerDied","Data":"50e5c0d05e179f7a8cdf088bd5ae0f991c830f8ae0954105ff38e2dcbae5e8b2"} Jul 06 22:16:59.314176 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:59.313776 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5hf5j" event={"ID":"099f6a13-fbc6-40ac-ae8a-aee476b43e4a","Type":"ContainerDied","Data":"5535bc1a03741aa8af09f6b99115797945b1d4f9c17ac9e6ce292a38b1b1e72b"} Jul 06 22:16:59.314176 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:59.313804 2581 scope.go:117] "RemoveContainer" containerID="50e5c0d05e179f7a8cdf088bd5ae0f991c830f8ae0954105ff38e2dcbae5e8b2" Jul 06 22:16:59.315416 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:59.315392 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-njfz2" event={"ID":"7993f081-3f17-4868-bfc3-285e366f68fb","Type":"ContainerStarted","Data":"a0b2ee72665a221b0e813a841f8848eeb780bcd2c1cc9dda49ef41ba6dbb0c51"} Jul 06 22:16:59.315510 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:59.315434 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-njfz2" event={"ID":"7993f081-3f17-4868-bfc3-285e366f68fb","Type":"ContainerStarted","Data":"e7e1d7565f3fe00aa6c41c3f8e2200115702ad22a213c66403d1454a6a39fd93"} Jul 06 22:16:59.315560 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:59.315523 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-njfz2" Jul 06 22:16:59.322346 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:59.322324 2581 scope.go:117] "RemoveContainer" containerID="50e5c0d05e179f7a8cdf088bd5ae0f991c830f8ae0954105ff38e2dcbae5e8b2" Jul 06 22:16:59.322579 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:16:59.322559 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"50e5c0d05e179f7a8cdf088bd5ae0f991c830f8ae0954105ff38e2dcbae5e8b2\": container with ID starting with 50e5c0d05e179f7a8cdf088bd5ae0f991c830f8ae0954105ff38e2dcbae5e8b2 not found: ID does not exist" containerID="50e5c0d05e179f7a8cdf088bd5ae0f991c830f8ae0954105ff38e2dcbae5e8b2" Jul 06 22:16:59.322652 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:59.322586 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"50e5c0d05e179f7a8cdf088bd5ae0f991c830f8ae0954105ff38e2dcbae5e8b2"} err="failed to get container status \"50e5c0d05e179f7a8cdf088bd5ae0f991c830f8ae0954105ff38e2dcbae5e8b2\": rpc error: code = NotFound desc = could not find container \"50e5c0d05e179f7a8cdf088bd5ae0f991c830f8ae0954105ff38e2dcbae5e8b2\": container with ID starting with 50e5c0d05e179f7a8cdf088bd5ae0f991c830f8ae0954105ff38e2dcbae5e8b2 not found: ID does not exist" Jul 06 22:16:59.339429 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:59.339322 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-njfz2" podStartSLOduration=1.339307451 podStartE2EDuration="1.339307451s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:16:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:16:59.337567121 +0000 UTC m=+498.896362442" watchObservedRunningTime="2026-07-06 22:16:59.339307451 +0000 UTC m=+498.898102771" Jul 06 22:16:59.351440 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:59.351407 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5hf5j"] Jul 06 22:16:59.359061 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:16:59.359031 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-5hf5j"] Jul 06 22:17:00.976209 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:00.976171 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="099f6a13-fbc6-40ac-ae8a-aee476b43e4a" path="/var/lib/kubelet/pods/099f6a13-fbc6-40ac-ae8a-aee476b43e4a/volumes" Jul 06 22:17:02.683589 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:02.683555 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:17:02.783955 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:02.783912 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:17:04.090468 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:04.090430 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:17:04.189603 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:04.189573 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:17:06.690915 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:06.690877 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:17:06.790560 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:06.790524 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:17:10.000953 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:10.000912 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:17:10.099541 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:10.099499 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:17:10.325277 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:10.325196 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-njfz2" Jul 06 22:17:10.378103 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:10.378056 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-c7hv8"] Jul 06 22:17:10.378343 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:10.378316 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-c7hv8" podUID="127848fa-e307-4d8f-b0d1-bcf9b7dc4897" containerName="manager" containerID="cri-o://841905d736b0d21f1beab130bfbfa166481f8229cf99e7a35d9406a0a0c02f09" gracePeriod=10 Jul 06 22:17:10.625040 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:10.625014 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-c7hv8" Jul 06 22:17:10.734787 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:10.734755 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/127848fa-e307-4d8f-b0d1-bcf9b7dc4897-extensions-socket-volume\") pod \"127848fa-e307-4d8f-b0d1-bcf9b7dc4897\" (UID: \"127848fa-e307-4d8f-b0d1-bcf9b7dc4897\") " Jul 06 22:17:10.734977 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:10.734834 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tq7zq\" (UniqueName: \"kubernetes.io/projected/127848fa-e307-4d8f-b0d1-bcf9b7dc4897-kube-api-access-tq7zq\") pod \"127848fa-e307-4d8f-b0d1-bcf9b7dc4897\" (UID: \"127848fa-e307-4d8f-b0d1-bcf9b7dc4897\") " Jul 06 22:17:10.735233 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:10.735205 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/127848fa-e307-4d8f-b0d1-bcf9b7dc4897-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "127848fa-e307-4d8f-b0d1-bcf9b7dc4897" (UID: "127848fa-e307-4d8f-b0d1-bcf9b7dc4897"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:17:10.736864 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:10.736841 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/127848fa-e307-4d8f-b0d1-bcf9b7dc4897-kube-api-access-tq7zq" (OuterVolumeSpecName: "kube-api-access-tq7zq") pod "127848fa-e307-4d8f-b0d1-bcf9b7dc4897" (UID: "127848fa-e307-4d8f-b0d1-bcf9b7dc4897"). InnerVolumeSpecName "kube-api-access-tq7zq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 22:17:10.835946 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:10.835916 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tq7zq\" (UniqueName: \"kubernetes.io/projected/127848fa-e307-4d8f-b0d1-bcf9b7dc4897-kube-api-access-tq7zq\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:17:10.835946 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:10.835943 2581 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/127848fa-e307-4d8f-b0d1-bcf9b7dc4897-extensions-socket-volume\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:17:11.379528 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:11.379446 2581 generic.go:356] "Generic (PLEG): container finished" podID="127848fa-e307-4d8f-b0d1-bcf9b7dc4897" containerID="841905d736b0d21f1beab130bfbfa166481f8229cf99e7a35d9406a0a0c02f09" exitCode=0 Jul 06 22:17:11.379528 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:11.379505 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-c7hv8" Jul 06 22:17:11.379941 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:11.379525 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-c7hv8" event={"ID":"127848fa-e307-4d8f-b0d1-bcf9b7dc4897","Type":"ContainerDied","Data":"841905d736b0d21f1beab130bfbfa166481f8229cf99e7a35d9406a0a0c02f09"} Jul 06 22:17:11.379941 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:11.379591 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-c7hv8" event={"ID":"127848fa-e307-4d8f-b0d1-bcf9b7dc4897","Type":"ContainerDied","Data":"aac7fda5f8edca88533f617048c6665d70391e68e65048fd65662ab1ba12a9f8"} Jul 06 22:17:11.379941 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:11.379617 2581 scope.go:117] "RemoveContainer" containerID="841905d736b0d21f1beab130bfbfa166481f8229cf99e7a35d9406a0a0c02f09" Jul 06 22:17:11.389963 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:11.389941 2581 scope.go:117] "RemoveContainer" containerID="841905d736b0d21f1beab130bfbfa166481f8229cf99e7a35d9406a0a0c02f09" Jul 06 22:17:11.390471 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:17:11.390454 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"841905d736b0d21f1beab130bfbfa166481f8229cf99e7a35d9406a0a0c02f09\": container with ID starting with 841905d736b0d21f1beab130bfbfa166481f8229cf99e7a35d9406a0a0c02f09 not found: ID does not exist" containerID="841905d736b0d21f1beab130bfbfa166481f8229cf99e7a35d9406a0a0c02f09" Jul 06 22:17:11.390556 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:11.390479 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"841905d736b0d21f1beab130bfbfa166481f8229cf99e7a35d9406a0a0c02f09"} err="failed to get container status \"841905d736b0d21f1beab130bfbfa166481f8229cf99e7a35d9406a0a0c02f09\": rpc error: code = NotFound desc = could not find container \"841905d736b0d21f1beab130bfbfa166481f8229cf99e7a35d9406a0a0c02f09\": container with ID starting with 841905d736b0d21f1beab130bfbfa166481f8229cf99e7a35d9406a0a0c02f09 not found: ID does not exist" Jul 06 22:17:11.400588 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:11.400566 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-c7hv8"] Jul 06 22:17:11.408714 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:11.408688 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-c7hv8"] Jul 06 22:17:12.976608 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:12.976572 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="127848fa-e307-4d8f-b0d1-bcf9b7dc4897" path="/var/lib/kubelet/pods/127848fa-e307-4d8f-b0d1-bcf9b7dc4897/volumes" Jul 06 22:17:13.481284 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:13.481252 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:17:13.581869 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:13.581815 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:17:14.667338 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:14.667306 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mbpp9"] Jul 06 22:17:14.667693 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:14.667653 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="127848fa-e307-4d8f-b0d1-bcf9b7dc4897" containerName="manager" Jul 06 22:17:14.667693 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:14.667664 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="127848fa-e307-4d8f-b0d1-bcf9b7dc4897" containerName="manager" Jul 06 22:17:14.667766 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:14.667736 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="127848fa-e307-4d8f-b0d1-bcf9b7dc4897" containerName="manager" Jul 06 22:17:14.671338 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:14.671317 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mbpp9" Jul 06 22:17:14.674222 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:14.674202 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-fmkhr\"" Jul 06 22:17:14.683055 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:14.683030 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mbpp9"] Jul 06 22:17:14.695972 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:14.695944 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:17:14.772811 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:14.772775 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0fd612d1-61ff-411a-a137-dded5a45a55e-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-mbpp9\" (UID: \"0fd612d1-61ff-411a-a137-dded5a45a55e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mbpp9" Jul 06 22:17:14.773001 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:14.772860 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0fd612d1-61ff-411a-a137-dded5a45a55e-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-mbpp9\" (UID: \"0fd612d1-61ff-411a-a137-dded5a45a55e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mbpp9" Jul 06 22:17:14.773001 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:14.772907 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0fd612d1-61ff-411a-a137-dded5a45a55e-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-mbpp9\" (UID: \"0fd612d1-61ff-411a-a137-dded5a45a55e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mbpp9" Jul 06 22:17:14.773001 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:14.772968 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mllb4\" (UniqueName: \"kubernetes.io/projected/0fd612d1-61ff-411a-a137-dded5a45a55e-kube-api-access-mllb4\") pod \"maas-default-gateway-openshift-default-56654dcf7-mbpp9\" (UID: \"0fd612d1-61ff-411a-a137-dded5a45a55e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mbpp9" Jul 06 22:17:14.773001 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:14.772997 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0fd612d1-61ff-411a-a137-dded5a45a55e-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-mbpp9\" (UID: \"0fd612d1-61ff-411a-a137-dded5a45a55e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mbpp9" Jul 06 22:17:14.773240 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:14.773024 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0fd612d1-61ff-411a-a137-dded5a45a55e-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-mbpp9\" (UID: \"0fd612d1-61ff-411a-a137-dded5a45a55e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mbpp9" Jul 06 22:17:14.773240 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:14.773057 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0fd612d1-61ff-411a-a137-dded5a45a55e-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-mbpp9\" (UID: \"0fd612d1-61ff-411a-a137-dded5a45a55e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mbpp9" Jul 06 22:17:14.773240 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:14.773102 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0fd612d1-61ff-411a-a137-dded5a45a55e-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-mbpp9\" (UID: \"0fd612d1-61ff-411a-a137-dded5a45a55e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mbpp9" Jul 06 22:17:14.773240 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:14.773147 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0fd612d1-61ff-411a-a137-dded5a45a55e-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-mbpp9\" (UID: \"0fd612d1-61ff-411a-a137-dded5a45a55e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mbpp9" Jul 06 22:17:14.874174 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:14.874141 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0fd612d1-61ff-411a-a137-dded5a45a55e-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-mbpp9\" (UID: \"0fd612d1-61ff-411a-a137-dded5a45a55e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mbpp9" Jul 06 22:17:14.874353 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:14.874186 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0fd612d1-61ff-411a-a137-dded5a45a55e-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-mbpp9\" (UID: \"0fd612d1-61ff-411a-a137-dded5a45a55e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mbpp9" Jul 06 22:17:14.874353 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:14.874201 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0fd612d1-61ff-411a-a137-dded5a45a55e-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-mbpp9\" (UID: \"0fd612d1-61ff-411a-a137-dded5a45a55e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mbpp9" Jul 06 22:17:14.874353 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:14.874243 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0fd612d1-61ff-411a-a137-dded5a45a55e-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-mbpp9\" (UID: \"0fd612d1-61ff-411a-a137-dded5a45a55e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mbpp9" Jul 06 22:17:14.874353 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:14.874272 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0fd612d1-61ff-411a-a137-dded5a45a55e-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-mbpp9\" (UID: \"0fd612d1-61ff-411a-a137-dded5a45a55e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mbpp9" Jul 06 22:17:14.874353 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:14.874330 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0fd612d1-61ff-411a-a137-dded5a45a55e-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-mbpp9\" (UID: \"0fd612d1-61ff-411a-a137-dded5a45a55e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mbpp9" Jul 06 22:17:14.874595 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:14.874368 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0fd612d1-61ff-411a-a137-dded5a45a55e-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-mbpp9\" (UID: \"0fd612d1-61ff-411a-a137-dded5a45a55e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mbpp9" Jul 06 22:17:14.874595 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:14.874405 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mllb4\" (UniqueName: \"kubernetes.io/projected/0fd612d1-61ff-411a-a137-dded5a45a55e-kube-api-access-mllb4\") pod \"maas-default-gateway-openshift-default-56654dcf7-mbpp9\" (UID: \"0fd612d1-61ff-411a-a137-dded5a45a55e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mbpp9" Jul 06 22:17:14.874595 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:14.874434 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0fd612d1-61ff-411a-a137-dded5a45a55e-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-mbpp9\" (UID: \"0fd612d1-61ff-411a-a137-dded5a45a55e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mbpp9" Jul 06 22:17:14.874777 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:14.874752 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0fd612d1-61ff-411a-a137-dded5a45a55e-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-mbpp9\" (UID: \"0fd612d1-61ff-411a-a137-dded5a45a55e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mbpp9" Jul 06 22:17:14.874835 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:14.874822 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0fd612d1-61ff-411a-a137-dded5a45a55e-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-mbpp9\" (UID: \"0fd612d1-61ff-411a-a137-dded5a45a55e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mbpp9" Jul 06 22:17:14.874872 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:14.874831 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0fd612d1-61ff-411a-a137-dded5a45a55e-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-mbpp9\" (UID: \"0fd612d1-61ff-411a-a137-dded5a45a55e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mbpp9" Jul 06 22:17:14.874983 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:14.874963 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0fd612d1-61ff-411a-a137-dded5a45a55e-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-mbpp9\" (UID: \"0fd612d1-61ff-411a-a137-dded5a45a55e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mbpp9" Jul 06 22:17:14.875176 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:14.875147 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0fd612d1-61ff-411a-a137-dded5a45a55e-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-mbpp9\" (UID: \"0fd612d1-61ff-411a-a137-dded5a45a55e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mbpp9" Jul 06 22:17:14.876654 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:14.876633 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0fd612d1-61ff-411a-a137-dded5a45a55e-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-mbpp9\" (UID: \"0fd612d1-61ff-411a-a137-dded5a45a55e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mbpp9" Jul 06 22:17:14.876864 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:14.876844 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0fd612d1-61ff-411a-a137-dded5a45a55e-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-mbpp9\" (UID: \"0fd612d1-61ff-411a-a137-dded5a45a55e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mbpp9" Jul 06 22:17:14.883377 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:14.883350 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mllb4\" (UniqueName: \"kubernetes.io/projected/0fd612d1-61ff-411a-a137-dded5a45a55e-kube-api-access-mllb4\") pod \"maas-default-gateway-openshift-default-56654dcf7-mbpp9\" (UID: \"0fd612d1-61ff-411a-a137-dded5a45a55e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mbpp9" Jul 06 22:17:14.883597 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:14.883576 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0fd612d1-61ff-411a-a137-dded5a45a55e-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-mbpp9\" (UID: \"0fd612d1-61ff-411a-a137-dded5a45a55e\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mbpp9" Jul 06 22:17:14.981699 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:14.981672 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mbpp9" Jul 06 22:17:15.125406 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:15.125383 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mbpp9"] Jul 06 22:17:15.128243 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:17:15.128202 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0fd612d1_61ff_411a_a137_dded5a45a55e.slice/crio-f35ba085930efd4a8e0c04e3a6e52a94ae8f3ac4ee9694cc4bd136d00a95a839 WatchSource:0}: Error finding container f35ba085930efd4a8e0c04e3a6e52a94ae8f3ac4ee9694cc4bd136d00a95a839: Status 404 returned error can't find the container with id f35ba085930efd4a8e0c04e3a6e52a94ae8f3ac4ee9694cc4bd136d00a95a839 Jul 06 22:17:15.130737 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:15.130701 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 22:17:15.130853 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:15.130784 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 22:17:15.130853 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:15.130829 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 22:17:15.258552 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:15.258466 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:17:15.401106 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:15.401057 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mbpp9" event={"ID":"0fd612d1-61ff-411a-a137-dded5a45a55e","Type":"ContainerStarted","Data":"57bf09705b5fa8cd733473969d130d79b1f460b306959f3cf8959f7a01322b67"} Jul 06 22:17:15.401290 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:15.401126 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mbpp9" event={"ID":"0fd612d1-61ff-411a-a137-dded5a45a55e","Type":"ContainerStarted","Data":"f35ba085930efd4a8e0c04e3a6e52a94ae8f3ac4ee9694cc4bd136d00a95a839"} Jul 06 22:17:15.422788 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:15.422743 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mbpp9" podStartSLOduration=1.422726779 podStartE2EDuration="1.422726779s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:17:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:17:15.421801663 +0000 UTC m=+514.980596983" watchObservedRunningTime="2026-07-06 22:17:15.422726779 +0000 UTC m=+514.981522097" Jul 06 22:17:15.982794 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:15.982760 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mbpp9" Jul 06 22:17:16.986563 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:16.986534 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mbpp9" Jul 06 22:17:17.412181 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:17.412101 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mbpp9" Jul 06 22:17:17.412980 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:17.412961 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mbpp9" Jul 06 22:17:23.492207 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:23.492155 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:17:23.592189 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:23.592157 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:17:24.088948 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:24.088914 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:17:24.204885 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:24.204847 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:17:25.384773 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:25.384718 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:17:25.486828 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:25.486798 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:17:31.222534 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:31.222486 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-ctprr"] Jul 06 22:17:31.223237 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:31.223202 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:17:31.227190 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:31.227174 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-ctprr" Jul 06 22:17:31.229627 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:31.229607 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 06 22:17:31.229627 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:31.229625 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-ppdw5\"" Jul 06 22:17:31.238901 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:31.238870 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-ctprr"] Jul 06 22:17:31.295460 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:31.295424 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/953846ad-e88e-4107-b372-7ed88b928f1c-config-file\") pod \"limitador-limitador-64c94fd64f-ctprr\" (UID: \"953846ad-e88e-4107-b372-7ed88b928f1c\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-ctprr" Jul 06 22:17:31.295607 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:31.295558 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wwd4v\" (UniqueName: \"kubernetes.io/projected/953846ad-e88e-4107-b372-7ed88b928f1c-kube-api-access-wwd4v\") pod \"limitador-limitador-64c94fd64f-ctprr\" (UID: \"953846ad-e88e-4107-b372-7ed88b928f1c\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-ctprr" Jul 06 22:17:31.320510 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:31.320476 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-ctprr"] Jul 06 22:17:31.348367 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:31.348332 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:17:31.396846 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:31.396810 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/953846ad-e88e-4107-b372-7ed88b928f1c-config-file\") pod \"limitador-limitador-64c94fd64f-ctprr\" (UID: \"953846ad-e88e-4107-b372-7ed88b928f1c\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-ctprr" Jul 06 22:17:31.397027 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:31.396894 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wwd4v\" (UniqueName: \"kubernetes.io/projected/953846ad-e88e-4107-b372-7ed88b928f1c-kube-api-access-wwd4v\") pod \"limitador-limitador-64c94fd64f-ctprr\" (UID: \"953846ad-e88e-4107-b372-7ed88b928f1c\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-ctprr" Jul 06 22:17:31.397467 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:31.397443 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/953846ad-e88e-4107-b372-7ed88b928f1c-config-file\") pod \"limitador-limitador-64c94fd64f-ctprr\" (UID: \"953846ad-e88e-4107-b372-7ed88b928f1c\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-ctprr" Jul 06 22:17:31.408209 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:31.408190 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wwd4v\" (UniqueName: \"kubernetes.io/projected/953846ad-e88e-4107-b372-7ed88b928f1c-kube-api-access-wwd4v\") pod \"limitador-limitador-64c94fd64f-ctprr\" (UID: \"953846ad-e88e-4107-b372-7ed88b928f1c\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-ctprr" Jul 06 22:17:31.446169 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:31.446137 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:17:31.539629 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:31.539539 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-ctprr" Jul 06 22:17:31.675598 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:31.675527 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-ctprr"] Jul 06 22:17:31.677791 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:17:31.677763 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod953846ad_e88e_4107_b372_7ed88b928f1c.slice/crio-76fb01a6d253485284f06eed15c471463eeff08bb86713b24832ff90890defc6 WatchSource:0}: Error finding container 76fb01a6d253485284f06eed15c471463eeff08bb86713b24832ff90890defc6: Status 404 returned error can't find the container with id 76fb01a6d253485284f06eed15c471463eeff08bb86713b24832ff90890defc6 Jul 06 22:17:32.056243 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:32.056207 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-nj6lz"] Jul 06 22:17:32.060851 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:32.060828 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-nj6lz" Jul 06 22:17:32.063546 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:32.063521 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-sl6cf\"" Jul 06 22:17:32.064585 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:32.064562 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-nj6lz"] Jul 06 22:17:32.092476 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:32.092443 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:17:32.102463 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:32.102437 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-96nqk\" (UniqueName: \"kubernetes.io/projected/ba68b784-595d-4461-b2b9-45533a5cfe62-kube-api-access-96nqk\") pod \"authorino-58cd955675-nj6lz\" (UID: \"ba68b784-595d-4461-b2b9-45533a5cfe62\") " pod="kuadrant-system/authorino-58cd955675-nj6lz" Jul 06 22:17:32.161873 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:32.161834 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:17:32.203090 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:32.203035 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-96nqk\" (UniqueName: \"kubernetes.io/projected/ba68b784-595d-4461-b2b9-45533a5cfe62-kube-api-access-96nqk\") pod \"authorino-58cd955675-nj6lz\" (UID: \"ba68b784-595d-4461-b2b9-45533a5cfe62\") " pod="kuadrant-system/authorino-58cd955675-nj6lz" Jul 06 22:17:32.211242 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:32.211219 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-96nqk\" (UniqueName: \"kubernetes.io/projected/ba68b784-595d-4461-b2b9-45533a5cfe62-kube-api-access-96nqk\") pod \"authorino-58cd955675-nj6lz\" (UID: \"ba68b784-595d-4461-b2b9-45533a5cfe62\") " pod="kuadrant-system/authorino-58cd955675-nj6lz" Jul 06 22:17:32.371877 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:32.371793 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-nj6lz" Jul 06 22:17:32.534135 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:32.534001 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-ctprr" event={"ID":"953846ad-e88e-4107-b372-7ed88b928f1c","Type":"ContainerStarted","Data":"76fb01a6d253485284f06eed15c471463eeff08bb86713b24832ff90890defc6"} Jul 06 22:17:32.610225 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:32.610159 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-nj6lz"] Jul 06 22:17:32.613158 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:17:32.613111 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podba68b784_595d_4461_b2b9_45533a5cfe62.slice/crio-8ffa8663effa960a5e42fdc254e7626295189d8e95450aef889016f19b9d97d7 WatchSource:0}: Error finding container 8ffa8663effa960a5e42fdc254e7626295189d8e95450aef889016f19b9d97d7: Status 404 returned error can't find the container with id 8ffa8663effa960a5e42fdc254e7626295189d8e95450aef889016f19b9d97d7 Jul 06 22:17:33.544512 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:33.544305 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-nj6lz" event={"ID":"ba68b784-595d-4461-b2b9-45533a5cfe62","Type":"ContainerStarted","Data":"8ffa8663effa960a5e42fdc254e7626295189d8e95450aef889016f19b9d97d7"} Jul 06 22:17:36.561395 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:36.561297 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-ctprr" event={"ID":"953846ad-e88e-4107-b372-7ed88b928f1c","Type":"ContainerStarted","Data":"d383154796f6ac93b3c80fb590861a30b265aa72eb095ec9e413e155c3f42ead"} Jul 06 22:17:36.561395 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:36.561371 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-ctprr" Jul 06 22:17:36.562820 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:36.562794 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-nj6lz" event={"ID":"ba68b784-595d-4461-b2b9-45533a5cfe62","Type":"ContainerStarted","Data":"cd2eb32e0703e31b58968ab7d8de8de692970121cca60a9aa6146b0e33d9354a"} Jul 06 22:17:36.607614 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:36.607556 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-ctprr" podStartSLOduration=1.067019588 podStartE2EDuration="5.607538111s" totalImagesPullingTime="4.540518523s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:17:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:17:36.58163991 +0000 UTC m=+536.140435311" watchObservedRunningTime="2026-07-06 22:17:36.607538111 +0000 UTC m=+536.166333431" Jul 06 22:17:36.607815 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:36.607646 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-nj6lz" podStartSLOduration=0.944979766 podStartE2EDuration="4.607641755s" totalImagesPullingTime="3.662661989s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:17:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:17:36.607465855 +0000 UTC m=+536.166261198" watchObservedRunningTime="2026-07-06 22:17:36.607641755 +0000 UTC m=+536.166437078" Jul 06 22:17:36.934679 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:36.934650 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-nj6lz"] Jul 06 22:17:38.571457 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:38.571423 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-nj6lz" podUID="ba68b784-595d-4461-b2b9-45533a5cfe62" containerName="authorino" containerID="cri-o://cd2eb32e0703e31b58968ab7d8de8de692970121cca60a9aa6146b0e33d9354a" gracePeriod=30 Jul 06 22:17:38.810594 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:38.810568 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-nj6lz" Jul 06 22:17:38.864254 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:38.864176 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-96nqk\" (UniqueName: \"kubernetes.io/projected/ba68b784-595d-4461-b2b9-45533a5cfe62-kube-api-access-96nqk\") pod \"ba68b784-595d-4461-b2b9-45533a5cfe62\" (UID: \"ba68b784-595d-4461-b2b9-45533a5cfe62\") " Jul 06 22:17:38.866215 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:38.866191 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ba68b784-595d-4461-b2b9-45533a5cfe62-kube-api-access-96nqk" (OuterVolumeSpecName: "kube-api-access-96nqk") pod "ba68b784-595d-4461-b2b9-45533a5cfe62" (UID: "ba68b784-595d-4461-b2b9-45533a5cfe62"). InnerVolumeSpecName "kube-api-access-96nqk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 22:17:38.965472 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:38.965431 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-96nqk\" (UniqueName: \"kubernetes.io/projected/ba68b784-595d-4461-b2b9-45533a5cfe62-kube-api-access-96nqk\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:17:39.577996 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:39.577959 2581 generic.go:356] "Generic (PLEG): container finished" podID="ba68b784-595d-4461-b2b9-45533a5cfe62" containerID="cd2eb32e0703e31b58968ab7d8de8de692970121cca60a9aa6146b0e33d9354a" exitCode=0 Jul 06 22:17:39.578489 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:39.578015 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-nj6lz" Jul 06 22:17:39.578489 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:39.578013 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-nj6lz" event={"ID":"ba68b784-595d-4461-b2b9-45533a5cfe62","Type":"ContainerDied","Data":"cd2eb32e0703e31b58968ab7d8de8de692970121cca60a9aa6146b0e33d9354a"} Jul 06 22:17:39.578489 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:39.578145 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-nj6lz" event={"ID":"ba68b784-595d-4461-b2b9-45533a5cfe62","Type":"ContainerDied","Data":"8ffa8663effa960a5e42fdc254e7626295189d8e95450aef889016f19b9d97d7"} Jul 06 22:17:39.578489 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:39.578182 2581 scope.go:117] "RemoveContainer" containerID="cd2eb32e0703e31b58968ab7d8de8de692970121cca60a9aa6146b0e33d9354a" Jul 06 22:17:39.586467 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:39.586452 2581 scope.go:117] "RemoveContainer" containerID="cd2eb32e0703e31b58968ab7d8de8de692970121cca60a9aa6146b0e33d9354a" Jul 06 22:17:39.586713 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:17:39.586690 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cd2eb32e0703e31b58968ab7d8de8de692970121cca60a9aa6146b0e33d9354a\": container with ID starting with cd2eb32e0703e31b58968ab7d8de8de692970121cca60a9aa6146b0e33d9354a not found: ID does not exist" containerID="cd2eb32e0703e31b58968ab7d8de8de692970121cca60a9aa6146b0e33d9354a" Jul 06 22:17:39.586783 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:39.586724 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cd2eb32e0703e31b58968ab7d8de8de692970121cca60a9aa6146b0e33d9354a"} err="failed to get container status \"cd2eb32e0703e31b58968ab7d8de8de692970121cca60a9aa6146b0e33d9354a\": rpc error: code = NotFound desc = could not find container \"cd2eb32e0703e31b58968ab7d8de8de692970121cca60a9aa6146b0e33d9354a\": container with ID starting with cd2eb32e0703e31b58968ab7d8de8de692970121cca60a9aa6146b0e33d9354a not found: ID does not exist" Jul 06 22:17:39.596829 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:39.596804 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-nj6lz"] Jul 06 22:17:39.599799 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:39.599775 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-nj6lz"] Jul 06 22:17:40.018599 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:40.018567 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:17:40.092717 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:40.092686 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:17:40.977658 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:40.977629 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ba68b784-595d-4461-b2b9-45533a5cfe62" path="/var/lib/kubelet/pods/ba68b784-595d-4461-b2b9-45533a5cfe62/volumes" Jul 06 22:17:45.981976 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:45.981941 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-ctprr"] Jul 06 22:17:45.982454 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:45.982129 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-ctprr" podUID="953846ad-e88e-4107-b372-7ed88b928f1c" containerName="limitador" containerID="cri-o://d383154796f6ac93b3c80fb590861a30b265aa72eb095ec9e413e155c3f42ead" gracePeriod=30 Jul 06 22:17:45.982835 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:45.982768 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-ctprr" Jul 06 22:17:46.533432 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:46.533409 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-ctprr" Jul 06 22:17:46.610550 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:46.610473 2581 generic.go:356] "Generic (PLEG): container finished" podID="953846ad-e88e-4107-b372-7ed88b928f1c" containerID="d383154796f6ac93b3c80fb590861a30b265aa72eb095ec9e413e155c3f42ead" exitCode=0 Jul 06 22:17:46.610550 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:46.610519 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-ctprr" Jul 06 22:17:46.610731 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:46.610560 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-ctprr" event={"ID":"953846ad-e88e-4107-b372-7ed88b928f1c","Type":"ContainerDied","Data":"d383154796f6ac93b3c80fb590861a30b265aa72eb095ec9e413e155c3f42ead"} Jul 06 22:17:46.610731 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:46.610616 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-ctprr" event={"ID":"953846ad-e88e-4107-b372-7ed88b928f1c","Type":"ContainerDied","Data":"76fb01a6d253485284f06eed15c471463eeff08bb86713b24832ff90890defc6"} Jul 06 22:17:46.610731 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:46.610638 2581 scope.go:117] "RemoveContainer" containerID="d383154796f6ac93b3c80fb590861a30b265aa72eb095ec9e413e155c3f42ead" Jul 06 22:17:46.618531 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:46.618506 2581 scope.go:117] "RemoveContainer" containerID="d383154796f6ac93b3c80fb590861a30b265aa72eb095ec9e413e155c3f42ead" Jul 06 22:17:46.618800 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:17:46.618784 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d383154796f6ac93b3c80fb590861a30b265aa72eb095ec9e413e155c3f42ead\": container with ID starting with d383154796f6ac93b3c80fb590861a30b265aa72eb095ec9e413e155c3f42ead not found: ID does not exist" containerID="d383154796f6ac93b3c80fb590861a30b265aa72eb095ec9e413e155c3f42ead" Jul 06 22:17:46.618836 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:46.618809 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d383154796f6ac93b3c80fb590861a30b265aa72eb095ec9e413e155c3f42ead"} err="failed to get container status \"d383154796f6ac93b3c80fb590861a30b265aa72eb095ec9e413e155c3f42ead\": rpc error: code = NotFound desc = could not find container \"d383154796f6ac93b3c80fb590861a30b265aa72eb095ec9e413e155c3f42ead\": container with ID starting with d383154796f6ac93b3c80fb590861a30b265aa72eb095ec9e413e155c3f42ead not found: ID does not exist" Jul 06 22:17:46.629102 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:46.629077 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/953846ad-e88e-4107-b372-7ed88b928f1c-config-file\") pod \"953846ad-e88e-4107-b372-7ed88b928f1c\" (UID: \"953846ad-e88e-4107-b372-7ed88b928f1c\") " Jul 06 22:17:46.629202 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:46.629190 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wwd4v\" (UniqueName: \"kubernetes.io/projected/953846ad-e88e-4107-b372-7ed88b928f1c-kube-api-access-wwd4v\") pod \"953846ad-e88e-4107-b372-7ed88b928f1c\" (UID: \"953846ad-e88e-4107-b372-7ed88b928f1c\") " Jul 06 22:17:46.629327 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:46.629304 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/953846ad-e88e-4107-b372-7ed88b928f1c-config-file" (OuterVolumeSpecName: "config-file") pod "953846ad-e88e-4107-b372-7ed88b928f1c" (UID: "953846ad-e88e-4107-b372-7ed88b928f1c"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 22:17:46.631105 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:46.631084 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/953846ad-e88e-4107-b372-7ed88b928f1c-kube-api-access-wwd4v" (OuterVolumeSpecName: "kube-api-access-wwd4v") pod "953846ad-e88e-4107-b372-7ed88b928f1c" (UID: "953846ad-e88e-4107-b372-7ed88b928f1c"). InnerVolumeSpecName "kube-api-access-wwd4v". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 22:17:46.730627 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:46.730590 2581 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/953846ad-e88e-4107-b372-7ed88b928f1c-config-file\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:17:46.730627 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:46.730617 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wwd4v\" (UniqueName: \"kubernetes.io/projected/953846ad-e88e-4107-b372-7ed88b928f1c-kube-api-access-wwd4v\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:17:46.931631 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:46.931595 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-ctprr"] Jul 06 22:17:46.937267 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:46.937242 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-ctprr"] Jul 06 22:17:46.975767 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:46.975741 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="953846ad-e88e-4107-b372-7ed88b928f1c" path="/var/lib/kubelet/pods/953846ad-e88e-4107-b372-7ed88b928f1c/volumes" Jul 06 22:17:48.312286 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:48.312253 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/postgres-69fb4594cf-mt2gt"] Jul 06 22:17:48.312645 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:48.312613 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ba68b784-595d-4461-b2b9-45533a5cfe62" containerName="authorino" Jul 06 22:17:48.312645 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:48.312626 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="ba68b784-595d-4461-b2b9-45533a5cfe62" containerName="authorino" Jul 06 22:17:48.312645 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:48.312646 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="953846ad-e88e-4107-b372-7ed88b928f1c" containerName="limitador" Jul 06 22:17:48.312741 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:48.312652 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="953846ad-e88e-4107-b372-7ed88b928f1c" containerName="limitador" Jul 06 22:17:48.312741 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:48.312711 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="953846ad-e88e-4107-b372-7ed88b928f1c" containerName="limitador" Jul 06 22:17:48.312741 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:48.312721 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="ba68b784-595d-4461-b2b9-45533a5cfe62" containerName="authorino" Jul 06 22:17:48.316786 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:48.316766 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-mt2gt" Jul 06 22:17:48.319281 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:48.319260 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"postgres-creds\"" Jul 06 22:17:48.319378 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:48.319260 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-kl66q\"" Jul 06 22:17:48.324795 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:48.324772 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-mt2gt"] Jul 06 22:17:48.351740 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:48.351706 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:17:48.450695 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:48.450666 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/07104706-fbc7-4f5a-9b3f-e15ba908f46f-data\") pod \"postgres-69fb4594cf-mt2gt\" (UID: \"07104706-fbc7-4f5a-9b3f-e15ba908f46f\") " pod="opendatahub/postgres-69fb4594cf-mt2gt" Jul 06 22:17:48.450845 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:48.450733 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v2hw5\" (UniqueName: \"kubernetes.io/projected/07104706-fbc7-4f5a-9b3f-e15ba908f46f-kube-api-access-v2hw5\") pod \"postgres-69fb4594cf-mt2gt\" (UID: \"07104706-fbc7-4f5a-9b3f-e15ba908f46f\") " pod="opendatahub/postgres-69fb4594cf-mt2gt" Jul 06 22:17:48.551366 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:48.551334 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v2hw5\" (UniqueName: \"kubernetes.io/projected/07104706-fbc7-4f5a-9b3f-e15ba908f46f-kube-api-access-v2hw5\") pod \"postgres-69fb4594cf-mt2gt\" (UID: \"07104706-fbc7-4f5a-9b3f-e15ba908f46f\") " pod="opendatahub/postgres-69fb4594cf-mt2gt" Jul 06 22:17:48.551535 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:48.551397 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/07104706-fbc7-4f5a-9b3f-e15ba908f46f-data\") pod \"postgres-69fb4594cf-mt2gt\" (UID: \"07104706-fbc7-4f5a-9b3f-e15ba908f46f\") " pod="opendatahub/postgres-69fb4594cf-mt2gt" Jul 06 22:17:48.551753 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:48.551730 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/07104706-fbc7-4f5a-9b3f-e15ba908f46f-data\") pod \"postgres-69fb4594cf-mt2gt\" (UID: \"07104706-fbc7-4f5a-9b3f-e15ba908f46f\") " pod="opendatahub/postgres-69fb4594cf-mt2gt" Jul 06 22:17:48.563254 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:48.563191 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v2hw5\" (UniqueName: \"kubernetes.io/projected/07104706-fbc7-4f5a-9b3f-e15ba908f46f-kube-api-access-v2hw5\") pod \"postgres-69fb4594cf-mt2gt\" (UID: \"07104706-fbc7-4f5a-9b3f-e15ba908f46f\") " pod="opendatahub/postgres-69fb4594cf-mt2gt" Jul 06 22:17:48.627597 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:48.627561 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-mt2gt" Jul 06 22:17:48.758832 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:48.758806 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-mt2gt"] Jul 06 22:17:48.760988 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:17:48.760963 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07104706_fbc7_4f5a_9b3f_e15ba908f46f.slice/crio-c7d8c38f9fb4a7fe21559044dc9ddb0f940da1337a6f033e1f36cc6f3f034941 WatchSource:0}: Error finding container c7d8c38f9fb4a7fe21559044dc9ddb0f940da1337a6f033e1f36cc6f3f034941: Status 404 returned error can't find the container with id c7d8c38f9fb4a7fe21559044dc9ddb0f940da1337a6f033e1f36cc6f3f034941 Jul 06 22:17:49.629235 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:49.629198 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-mt2gt" event={"ID":"07104706-fbc7-4f5a-9b3f-e15ba908f46f","Type":"ContainerStarted","Data":"c7d8c38f9fb4a7fe21559044dc9ddb0f940da1337a6f033e1f36cc6f3f034941"} Jul 06 22:17:56.666965 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:56.666914 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-mt2gt" event={"ID":"07104706-fbc7-4f5a-9b3f-e15ba908f46f","Type":"ContainerStarted","Data":"2597d894b13556f74062b6a6b018ea676ef2928c7c1ba258d1770f1ac57fc727"} Jul 06 22:17:56.667388 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:56.667001 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/postgres-69fb4594cf-mt2gt" Jul 06 22:17:56.689753 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:56.689669 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/postgres-69fb4594cf-mt2gt" podStartSLOduration=1.015069804 podStartE2EDuration="8.689654368s" totalImagesPullingTime="7.674584564s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:17:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:17:56.687249544 +0000 UTC m=+556.246044868" watchObservedRunningTime="2026-07-06 22:17:56.689654368 +0000 UTC m=+556.248449688" Jul 06 22:17:58.398927 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:17:58.398890 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:18:02.701091 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:02.701037 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/postgres-69fb4594cf-mt2gt" Jul 06 22:18:03.040225 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:03.040121 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:18:03.272484 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:03.272452 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-t5sng"] Jul 06 22:18:03.278575 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:03.278553 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-t5sng" Jul 06 22:18:03.283404 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:03.283384 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-sl6cf\"" Jul 06 22:18:03.287244 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:03.287216 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-t5sng"] Jul 06 22:18:03.315170 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:03.315084 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:18:03.394935 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:03.394903 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pp92r\" (UniqueName: \"kubernetes.io/projected/1a855feb-f662-4c0f-8aac-4129bf10d75b-kube-api-access-pp92r\") pod \"authorino-5d4b67c446-t5sng\" (UID: \"1a855feb-f662-4c0f-8aac-4129bf10d75b\") " pod="kuadrant-system/authorino-5d4b67c446-t5sng" Jul 06 22:18:03.467386 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:03.467350 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-t5sng"] Jul 06 22:18:03.467685 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:18:03.467662 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-pp92r], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-5d4b67c446-t5sng" podUID="1a855feb-f662-4c0f-8aac-4129bf10d75b" Jul 06 22:18:03.495779 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:03.495750 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pp92r\" (UniqueName: \"kubernetes.io/projected/1a855feb-f662-4c0f-8aac-4129bf10d75b-kube-api-access-pp92r\") pod \"authorino-5d4b67c446-t5sng\" (UID: \"1a855feb-f662-4c0f-8aac-4129bf10d75b\") " pod="kuadrant-system/authorino-5d4b67c446-t5sng" Jul 06 22:18:03.499800 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:03.499777 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-76877994-vwn5b"] Jul 06 22:18:03.502993 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:03.502974 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-vwn5b" Jul 06 22:18:03.505448 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:03.505432 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 06 22:18:03.509459 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:03.509437 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pp92r\" (UniqueName: \"kubernetes.io/projected/1a855feb-f662-4c0f-8aac-4129bf10d75b-kube-api-access-pp92r\") pod \"authorino-5d4b67c446-t5sng\" (UID: \"1a855feb-f662-4c0f-8aac-4129bf10d75b\") " pod="kuadrant-system/authorino-5d4b67c446-t5sng" Jul 06 22:18:03.513420 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:03.513395 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-76877994-vwn5b"] Jul 06 22:18:03.537187 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:03.537156 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-76877994-vwn5b"] Jul 06 22:18:03.537432 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:18:03.537413 2581 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-mxcsn tls-cert], unattached volumes=[], failed to process volumes=[kube-api-access-mxcsn tls-cert]: context canceled" pod="kuadrant-system/authorino-76877994-vwn5b" podUID="c21a1b10-c53b-4e5a-bf64-30b481b246b8" Jul 06 22:18:03.559745 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:03.559716 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-59fc789756-4zh9d"] Jul 06 22:18:03.563021 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:03.563003 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59fc789756-4zh9d" Jul 06 22:18:03.570673 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:03.570652 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59fc789756-4zh9d"] Jul 06 22:18:03.696734 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:03.696703 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6j496\" (UniqueName: \"kubernetes.io/projected/0d712e25-5eb7-446d-8199-242e4ca84067-kube-api-access-6j496\") pod \"authorino-59fc789756-4zh9d\" (UID: \"0d712e25-5eb7-446d-8199-242e4ca84067\") " pod="kuadrant-system/authorino-59fc789756-4zh9d" Jul 06 22:18:03.696734 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:03.696742 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0d712e25-5eb7-446d-8199-242e4ca84067-tls-cert\") pod \"authorino-59fc789756-4zh9d\" (UID: \"0d712e25-5eb7-446d-8199-242e4ca84067\") " pod="kuadrant-system/authorino-59fc789756-4zh9d" Jul 06 22:18:03.696950 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:03.696765 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c21a1b10-c53b-4e5a-bf64-30b481b246b8-tls-cert\") pod \"authorino-76877994-vwn5b\" (UID: \"c21a1b10-c53b-4e5a-bf64-30b481b246b8\") " pod="kuadrant-system/authorino-76877994-vwn5b" Jul 06 22:18:03.696950 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:03.696825 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mxcsn\" (UniqueName: \"kubernetes.io/projected/c21a1b10-c53b-4e5a-bf64-30b481b246b8-kube-api-access-mxcsn\") pod \"authorino-76877994-vwn5b\" (UID: \"c21a1b10-c53b-4e5a-bf64-30b481b246b8\") " pod="kuadrant-system/authorino-76877994-vwn5b" Jul 06 22:18:03.699116 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:03.699094 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-vwn5b" Jul 06 22:18:03.699243 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:03.699095 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-t5sng" Jul 06 22:18:03.705660 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:03.705643 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-vwn5b" Jul 06 22:18:03.708650 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:03.708633 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-t5sng" Jul 06 22:18:03.798030 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:03.797999 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6j496\" (UniqueName: \"kubernetes.io/projected/0d712e25-5eb7-446d-8199-242e4ca84067-kube-api-access-6j496\") pod \"authorino-59fc789756-4zh9d\" (UID: \"0d712e25-5eb7-446d-8199-242e4ca84067\") " pod="kuadrant-system/authorino-59fc789756-4zh9d" Jul 06 22:18:03.798237 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:03.798038 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0d712e25-5eb7-446d-8199-242e4ca84067-tls-cert\") pod \"authorino-59fc789756-4zh9d\" (UID: \"0d712e25-5eb7-446d-8199-242e4ca84067\") " pod="kuadrant-system/authorino-59fc789756-4zh9d" Jul 06 22:18:03.798237 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:03.798093 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c21a1b10-c53b-4e5a-bf64-30b481b246b8-tls-cert\") pod \"authorino-76877994-vwn5b\" (UID: \"c21a1b10-c53b-4e5a-bf64-30b481b246b8\") " pod="kuadrant-system/authorino-76877994-vwn5b" Jul 06 22:18:03.798237 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:03.798128 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mxcsn\" (UniqueName: \"kubernetes.io/projected/c21a1b10-c53b-4e5a-bf64-30b481b246b8-kube-api-access-mxcsn\") pod \"authorino-76877994-vwn5b\" (UID: \"c21a1b10-c53b-4e5a-bf64-30b481b246b8\") " pod="kuadrant-system/authorino-76877994-vwn5b" Jul 06 22:18:03.800698 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:03.800671 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c21a1b10-c53b-4e5a-bf64-30b481b246b8-tls-cert\") pod \"authorino-76877994-vwn5b\" (UID: \"c21a1b10-c53b-4e5a-bf64-30b481b246b8\") " pod="kuadrant-system/authorino-76877994-vwn5b" Jul 06 22:18:03.800698 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:03.800682 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0d712e25-5eb7-446d-8199-242e4ca84067-tls-cert\") pod \"authorino-59fc789756-4zh9d\" (UID: \"0d712e25-5eb7-446d-8199-242e4ca84067\") " pod="kuadrant-system/authorino-59fc789756-4zh9d" Jul 06 22:18:03.806185 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:03.806163 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mxcsn\" (UniqueName: \"kubernetes.io/projected/c21a1b10-c53b-4e5a-bf64-30b481b246b8-kube-api-access-mxcsn\") pod \"authorino-76877994-vwn5b\" (UID: \"c21a1b10-c53b-4e5a-bf64-30b481b246b8\") " pod="kuadrant-system/authorino-76877994-vwn5b" Jul 06 22:18:03.806412 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:03.806391 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6j496\" (UniqueName: \"kubernetes.io/projected/0d712e25-5eb7-446d-8199-242e4ca84067-kube-api-access-6j496\") pod \"authorino-59fc789756-4zh9d\" (UID: \"0d712e25-5eb7-446d-8199-242e4ca84067\") " pod="kuadrant-system/authorino-59fc789756-4zh9d" Jul 06 22:18:03.872984 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:03.872905 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59fc789756-4zh9d" Jul 06 22:18:03.899271 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:03.899240 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c21a1b10-c53b-4e5a-bf64-30b481b246b8-tls-cert\") pod \"c21a1b10-c53b-4e5a-bf64-30b481b246b8\" (UID: \"c21a1b10-c53b-4e5a-bf64-30b481b246b8\") " Jul 06 22:18:03.899450 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:03.899436 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pp92r\" (UniqueName: \"kubernetes.io/projected/1a855feb-f662-4c0f-8aac-4129bf10d75b-kube-api-access-pp92r\") pod \"1a855feb-f662-4c0f-8aac-4129bf10d75b\" (UID: \"1a855feb-f662-4c0f-8aac-4129bf10d75b\") " Jul 06 22:18:03.901272 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:03.901241 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c21a1b10-c53b-4e5a-bf64-30b481b246b8-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "c21a1b10-c53b-4e5a-bf64-30b481b246b8" (UID: "c21a1b10-c53b-4e5a-bf64-30b481b246b8"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 22:18:03.901365 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:03.901313 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1a855feb-f662-4c0f-8aac-4129bf10d75b-kube-api-access-pp92r" (OuterVolumeSpecName: "kube-api-access-pp92r") pod "1a855feb-f662-4c0f-8aac-4129bf10d75b" (UID: "1a855feb-f662-4c0f-8aac-4129bf10d75b"). InnerVolumeSpecName "kube-api-access-pp92r". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 22:18:04.000882 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:04.000846 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mxcsn\" (UniqueName: \"kubernetes.io/projected/c21a1b10-c53b-4e5a-bf64-30b481b246b8-kube-api-access-mxcsn\") pod \"c21a1b10-c53b-4e5a-bf64-30b481b246b8\" (UID: \"c21a1b10-c53b-4e5a-bf64-30b481b246b8\") " Jul 06 22:18:04.001060 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:04.001050 2581 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c21a1b10-c53b-4e5a-bf64-30b481b246b8-tls-cert\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:18:04.001153 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:04.001078 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pp92r\" (UniqueName: \"kubernetes.io/projected/1a855feb-f662-4c0f-8aac-4129bf10d75b-kube-api-access-pp92r\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:18:04.003237 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:04.003206 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c21a1b10-c53b-4e5a-bf64-30b481b246b8-kube-api-access-mxcsn" (OuterVolumeSpecName: "kube-api-access-mxcsn") pod "c21a1b10-c53b-4e5a-bf64-30b481b246b8" (UID: "c21a1b10-c53b-4e5a-bf64-30b481b246b8"). InnerVolumeSpecName "kube-api-access-mxcsn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 22:18:04.004400 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:04.004379 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59fc789756-4zh9d"] Jul 06 22:18:04.006141 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:18:04.006118 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0d712e25_5eb7_446d_8199_242e4ca84067.slice/crio-090d81e20c0f7b22228daef81ef4c8dbf8110fc32ec85f2c182a4af4f0964743 WatchSource:0}: Error finding container 090d81e20c0f7b22228daef81ef4c8dbf8110fc32ec85f2c182a4af4f0964743: Status 404 returned error can't find the container with id 090d81e20c0f7b22228daef81ef4c8dbf8110fc32ec85f2c182a4af4f0964743 Jul 06 22:18:04.102399 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:04.102363 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mxcsn\" (UniqueName: \"kubernetes.io/projected/c21a1b10-c53b-4e5a-bf64-30b481b246b8-kube-api-access-mxcsn\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:18:04.705593 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:04.705488 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59fc789756-4zh9d" event={"ID":"0d712e25-5eb7-446d-8199-242e4ca84067","Type":"ContainerStarted","Data":"2cc4c395295839cb969a9798c84e0cc872f7dc46d0936bafb752f52ff442c734"} Jul 06 22:18:04.705593 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:04.705537 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59fc789756-4zh9d" event={"ID":"0d712e25-5eb7-446d-8199-242e4ca84067","Type":"ContainerStarted","Data":"090d81e20c0f7b22228daef81ef4c8dbf8110fc32ec85f2c182a4af4f0964743"} Jul 06 22:18:04.705593 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:04.705590 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-vwn5b" Jul 06 22:18:04.706147 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:04.705677 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-t5sng" Jul 06 22:18:04.741420 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:04.741366 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-59fc789756-4zh9d" podStartSLOduration=1.296724375 podStartE2EDuration="1.741350163s" totalImagesPullingTime="444.625788ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:18:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:18:04.739744649 +0000 UTC m=+564.298539971" watchObservedRunningTime="2026-07-06 22:18:04.741350163 +0000 UTC m=+564.300145484" Jul 06 22:18:04.766508 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:04.766436 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-76877994-vwn5b"] Jul 06 22:18:04.778702 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:04.778671 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-76877994-vwn5b"] Jul 06 22:18:04.802412 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:04.802380 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-t5sng"] Jul 06 22:18:04.807415 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:04.807389 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-t5sng"] Jul 06 22:18:04.976174 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:04.976095 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1a855feb-f662-4c0f-8aac-4129bf10d75b" path="/var/lib/kubelet/pods/1a855feb-f662-4c0f-8aac-4129bf10d75b/volumes" Jul 06 22:18:04.976360 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:04.976348 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c21a1b10-c53b-4e5a-bf64-30b481b246b8" path="/var/lib/kubelet/pods/c21a1b10-c53b-4e5a-bf64-30b481b246b8/volumes" Jul 06 22:18:09.177251 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:09.177205 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:18:09.202049 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:09.202018 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-664bd5f95-vlxsv"] Jul 06 22:18:09.205485 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:09.205467 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-664bd5f95-vlxsv" Jul 06 22:18:09.209335 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:09.209304 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-z45sk\"" Jul 06 22:18:09.209499 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:09.209380 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 06 22:18:09.209499 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:09.209443 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 06 22:18:09.224186 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:09.224156 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-664bd5f95-vlxsv"] Jul 06 22:18:09.239960 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:09.239929 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c30dc356-7539-488c-af45-cd3acd7d6994-webhook-cert\") pod \"maas-controller-664bd5f95-vlxsv\" (UID: \"c30dc356-7539-488c-af45-cd3acd7d6994\") " pod="opendatahub/maas-controller-664bd5f95-vlxsv" Jul 06 22:18:09.240143 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:09.239986 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cmrmh\" (UniqueName: \"kubernetes.io/projected/c30dc356-7539-488c-af45-cd3acd7d6994-kube-api-access-cmrmh\") pod \"maas-controller-664bd5f95-vlxsv\" (UID: \"c30dc356-7539-488c-af45-cd3acd7d6994\") " pod="opendatahub/maas-controller-664bd5f95-vlxsv" Jul 06 22:18:09.286197 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:09.286164 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:18:09.340542 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:09.340499 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c30dc356-7539-488c-af45-cd3acd7d6994-webhook-cert\") pod \"maas-controller-664bd5f95-vlxsv\" (UID: \"c30dc356-7539-488c-af45-cd3acd7d6994\") " pod="opendatahub/maas-controller-664bd5f95-vlxsv" Jul 06 22:18:09.340705 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:09.340565 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cmrmh\" (UniqueName: \"kubernetes.io/projected/c30dc356-7539-488c-af45-cd3acd7d6994-kube-api-access-cmrmh\") pod \"maas-controller-664bd5f95-vlxsv\" (UID: \"c30dc356-7539-488c-af45-cd3acd7d6994\") " pod="opendatahub/maas-controller-664bd5f95-vlxsv" Jul 06 22:18:09.342995 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:09.342965 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c30dc356-7539-488c-af45-cd3acd7d6994-webhook-cert\") pod \"maas-controller-664bd5f95-vlxsv\" (UID: \"c30dc356-7539-488c-af45-cd3acd7d6994\") " pod="opendatahub/maas-controller-664bd5f95-vlxsv" Jul 06 22:18:09.349317 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:09.349291 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cmrmh\" (UniqueName: \"kubernetes.io/projected/c30dc356-7539-488c-af45-cd3acd7d6994-kube-api-access-cmrmh\") pod \"maas-controller-664bd5f95-vlxsv\" (UID: \"c30dc356-7539-488c-af45-cd3acd7d6994\") " pod="opendatahub/maas-controller-664bd5f95-vlxsv" Jul 06 22:18:09.376614 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:09.376577 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:18:09.380207 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:09.380184 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5fb54f555-x797t"] Jul 06 22:18:09.383587 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:09.383572 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5fb54f555-x797t" Jul 06 22:18:09.392312 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:09.392292 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5fb54f555-x797t"] Jul 06 22:18:09.441472 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:09.441395 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/82ae67d0-8e90-48d5-bad6-6193f3991b61-webhook-cert\") pod \"maas-controller-5fb54f555-x797t\" (UID: \"82ae67d0-8e90-48d5-bad6-6193f3991b61\") " pod="opendatahub/maas-controller-5fb54f555-x797t" Jul 06 22:18:09.441584 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:09.441477 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hwvf2\" (UniqueName: \"kubernetes.io/projected/82ae67d0-8e90-48d5-bad6-6193f3991b61-kube-api-access-hwvf2\") pod \"maas-controller-5fb54f555-x797t\" (UID: \"82ae67d0-8e90-48d5-bad6-6193f3991b61\") " pod="opendatahub/maas-controller-5fb54f555-x797t" Jul 06 22:18:09.516758 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:09.516724 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-664bd5f95-vlxsv" Jul 06 22:18:09.542014 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:09.541987 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hwvf2\" (UniqueName: \"kubernetes.io/projected/82ae67d0-8e90-48d5-bad6-6193f3991b61-kube-api-access-hwvf2\") pod \"maas-controller-5fb54f555-x797t\" (UID: \"82ae67d0-8e90-48d5-bad6-6193f3991b61\") " pod="opendatahub/maas-controller-5fb54f555-x797t" Jul 06 22:18:09.542162 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:09.542046 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/82ae67d0-8e90-48d5-bad6-6193f3991b61-webhook-cert\") pod \"maas-controller-5fb54f555-x797t\" (UID: \"82ae67d0-8e90-48d5-bad6-6193f3991b61\") " pod="opendatahub/maas-controller-5fb54f555-x797t" Jul 06 22:18:09.544475 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:09.544453 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/82ae67d0-8e90-48d5-bad6-6193f3991b61-webhook-cert\") pod \"maas-controller-5fb54f555-x797t\" (UID: \"82ae67d0-8e90-48d5-bad6-6193f3991b61\") " pod="opendatahub/maas-controller-5fb54f555-x797t" Jul 06 22:18:09.550696 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:09.550671 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hwvf2\" (UniqueName: \"kubernetes.io/projected/82ae67d0-8e90-48d5-bad6-6193f3991b61-kube-api-access-hwvf2\") pod \"maas-controller-5fb54f555-x797t\" (UID: \"82ae67d0-8e90-48d5-bad6-6193f3991b61\") " pod="opendatahub/maas-controller-5fb54f555-x797t" Jul 06 22:18:09.649235 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:09.649166 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-664bd5f95-vlxsv"] Jul 06 22:18:09.651468 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:18:09.651441 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc30dc356_7539_488c_af45_cd3acd7d6994.slice/crio-94eed347008e0c7d38cb25e94775383a6fa57f5e6b6cd79d745b9f78de591175 WatchSource:0}: Error finding container 94eed347008e0c7d38cb25e94775383a6fa57f5e6b6cd79d745b9f78de591175: Status 404 returned error can't find the container with id 94eed347008e0c7d38cb25e94775383a6fa57f5e6b6cd79d745b9f78de591175 Jul 06 22:18:09.695401 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:09.695322 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5fb54f555-x797t" Jul 06 22:18:09.731315 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:09.731283 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-664bd5f95-vlxsv" event={"ID":"c30dc356-7539-488c-af45-cd3acd7d6994","Type":"ContainerStarted","Data":"94eed347008e0c7d38cb25e94775383a6fa57f5e6b6cd79d745b9f78de591175"} Jul 06 22:18:09.836124 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:09.836017 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5fb54f555-x797t"] Jul 06 22:18:09.838283 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:18:09.838258 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82ae67d0_8e90_48d5_bad6_6193f3991b61.slice/crio-2fba67abf4903ce3a1233a545aae1e06eb293a1187de529f30416e89679d84cd WatchSource:0}: Error finding container 2fba67abf4903ce3a1233a545aae1e06eb293a1187de529f30416e89679d84cd: Status 404 returned error can't find the container with id 2fba67abf4903ce3a1233a545aae1e06eb293a1187de529f30416e89679d84cd Jul 06 22:18:10.742209 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:10.742100 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5fb54f555-x797t" event={"ID":"82ae67d0-8e90-48d5-bad6-6193f3991b61","Type":"ContainerStarted","Data":"2fba67abf4903ce3a1233a545aae1e06eb293a1187de529f30416e89679d84cd"} Jul 06 22:18:12.735320 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:12.735172 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:18:12.753145 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:12.753102 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5fb54f555-x797t" event={"ID":"82ae67d0-8e90-48d5-bad6-6193f3991b61","Type":"ContainerStarted","Data":"82b5e3940dbb902e5f08b5a044fba92fb0119d74a5f65f4b5e6fbf5dd6a930a2"} Jul 06 22:18:12.753305 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:12.753218 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5fb54f555-x797t" Jul 06 22:18:12.754521 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:12.754501 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-664bd5f95-vlxsv" event={"ID":"c30dc356-7539-488c-af45-cd3acd7d6994","Type":"ContainerStarted","Data":"c08dbc2c085728ad119aff2def4dd16289cf7517d909f8ced6938fa83a45841e"} Jul 06 22:18:12.754614 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:12.754552 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-664bd5f95-vlxsv" Jul 06 22:18:12.772721 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:12.772681 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5fb54f555-x797t" podStartSLOduration=1.114091377 podStartE2EDuration="3.772667421s" totalImagesPullingTime="2.658576044s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:18:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:18:12.770578952 +0000 UTC m=+572.329374272" watchObservedRunningTime="2026-07-06 22:18:12.772667421 +0000 UTC m=+572.331462747" Jul 06 22:18:12.805921 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:12.805859 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-664bd5f95-vlxsv" podStartSLOduration=0.950354792 podStartE2EDuration="3.805844369s" totalImagesPullingTime="2.855489577s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:18:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:18:12.804503454 +0000 UTC m=+572.363298776" watchObservedRunningTime="2026-07-06 22:18:12.805844369 +0000 UTC m=+572.364639689" Jul 06 22:18:12.876180 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:12.876130 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:18:23.764464 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:23.764425 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-664bd5f95-vlxsv" Jul 06 22:18:23.764933 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:23.764510 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5fb54f555-x797t" Jul 06 22:18:23.820921 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:23.820883 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-664bd5f95-vlxsv"] Jul 06 22:18:23.821127 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:23.821101 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-664bd5f95-vlxsv" podUID="c30dc356-7539-488c-af45-cd3acd7d6994" containerName="manager" containerID="cri-o://c08dbc2c085728ad119aff2def4dd16289cf7517d909f8ced6938fa83a45841e" gracePeriod=30 Jul 06 22:18:24.069950 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:24.069927 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-664bd5f95-vlxsv" Jul 06 22:18:24.171025 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:24.170988 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cmrmh\" (UniqueName: \"kubernetes.io/projected/c30dc356-7539-488c-af45-cd3acd7d6994-kube-api-access-cmrmh\") pod \"c30dc356-7539-488c-af45-cd3acd7d6994\" (UID: \"c30dc356-7539-488c-af45-cd3acd7d6994\") " Jul 06 22:18:24.171224 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:24.171036 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c30dc356-7539-488c-af45-cd3acd7d6994-webhook-cert\") pod \"c30dc356-7539-488c-af45-cd3acd7d6994\" (UID: \"c30dc356-7539-488c-af45-cd3acd7d6994\") " Jul 06 22:18:24.173312 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:24.173278 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c30dc356-7539-488c-af45-cd3acd7d6994-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "c30dc356-7539-488c-af45-cd3acd7d6994" (UID: "c30dc356-7539-488c-af45-cd3acd7d6994"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 22:18:24.173312 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:24.173298 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c30dc356-7539-488c-af45-cd3acd7d6994-kube-api-access-cmrmh" (OuterVolumeSpecName: "kube-api-access-cmrmh") pod "c30dc356-7539-488c-af45-cd3acd7d6994" (UID: "c30dc356-7539-488c-af45-cd3acd7d6994"). InnerVolumeSpecName "kube-api-access-cmrmh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 22:18:24.272567 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:24.272530 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cmrmh\" (UniqueName: \"kubernetes.io/projected/c30dc356-7539-488c-af45-cd3acd7d6994-kube-api-access-cmrmh\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:18:24.272567 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:24.272560 2581 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c30dc356-7539-488c-af45-cd3acd7d6994-webhook-cert\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:18:24.812707 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:24.812671 2581 generic.go:356] "Generic (PLEG): container finished" podID="c30dc356-7539-488c-af45-cd3acd7d6994" containerID="c08dbc2c085728ad119aff2def4dd16289cf7517d909f8ced6938fa83a45841e" exitCode=0 Jul 06 22:18:24.813144 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:24.812720 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-664bd5f95-vlxsv" Jul 06 22:18:24.813144 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:24.812751 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-664bd5f95-vlxsv" event={"ID":"c30dc356-7539-488c-af45-cd3acd7d6994","Type":"ContainerDied","Data":"c08dbc2c085728ad119aff2def4dd16289cf7517d909f8ced6938fa83a45841e"} Jul 06 22:18:24.813144 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:24.812805 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-664bd5f95-vlxsv" event={"ID":"c30dc356-7539-488c-af45-cd3acd7d6994","Type":"ContainerDied","Data":"94eed347008e0c7d38cb25e94775383a6fa57f5e6b6cd79d745b9f78de591175"} Jul 06 22:18:24.813144 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:24.812832 2581 scope.go:117] "RemoveContainer" containerID="c08dbc2c085728ad119aff2def4dd16289cf7517d909f8ced6938fa83a45841e" Jul 06 22:18:24.822188 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:24.821979 2581 scope.go:117] "RemoveContainer" containerID="c08dbc2c085728ad119aff2def4dd16289cf7517d909f8ced6938fa83a45841e" Jul 06 22:18:24.822275 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:18:24.822258 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c08dbc2c085728ad119aff2def4dd16289cf7517d909f8ced6938fa83a45841e\": container with ID starting with c08dbc2c085728ad119aff2def4dd16289cf7517d909f8ced6938fa83a45841e not found: ID does not exist" containerID="c08dbc2c085728ad119aff2def4dd16289cf7517d909f8ced6938fa83a45841e" Jul 06 22:18:24.822311 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:24.822282 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c08dbc2c085728ad119aff2def4dd16289cf7517d909f8ced6938fa83a45841e"} err="failed to get container status \"c08dbc2c085728ad119aff2def4dd16289cf7517d909f8ced6938fa83a45841e\": rpc error: code = NotFound desc = could not find container \"c08dbc2c085728ad119aff2def4dd16289cf7517d909f8ced6938fa83a45841e\": container with ID starting with c08dbc2c085728ad119aff2def4dd16289cf7517d909f8ced6938fa83a45841e not found: ID does not exist" Jul 06 22:18:24.833974 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:24.833949 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-664bd5f95-vlxsv"] Jul 06 22:18:24.838018 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:24.837997 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-664bd5f95-vlxsv"] Jul 06 22:18:24.976482 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:24.976448 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c30dc356-7539-488c-af45-cd3acd7d6994" path="/var/lib/kubelet/pods/c30dc356-7539-488c-af45-cd3acd7d6994/volumes" Jul 06 22:18:40.902079 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:40.902038 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-rcn5q_86b07166-2f34-443b-9c61-86b7d950ad7b/console-operator/1.log" Jul 06 22:18:40.902587 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:40.902189 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-rcn5q_86b07166-2f34-443b-9c61-86b7d950ad7b/console-operator/1.log" Jul 06 22:18:44.949548 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:44.949511 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:18:45.527515 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:45.527473 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:18:45.797698 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:45.797611 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:18:45.975890 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:45.975838 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:18:46.073426 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:46.073306 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:18:46.119937 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:46.119902 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-5mvrk"] Jul 06 22:18:46.120410 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:46.120391 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c30dc356-7539-488c-af45-cd3acd7d6994" containerName="manager" Jul 06 22:18:46.120410 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:46.120410 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="c30dc356-7539-488c-af45-cd3acd7d6994" containerName="manager" Jul 06 22:18:46.120535 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:46.120465 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="c30dc356-7539-488c-af45-cd3acd7d6994" containerName="manager" Jul 06 22:18:46.126488 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:46.126470 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-5mvrk" Jul 06 22:18:46.132425 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:46.132406 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 06 22:18:46.132693 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:46.132678 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-pgq6f\"" Jul 06 22:18:46.142280 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:46.142230 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-5mvrk"] Jul 06 22:18:46.198127 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:46.198095 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-9548w"] Jul 06 22:18:46.201617 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:46.201596 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-9548w" Jul 06 22:18:46.215085 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:46.215038 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-9548w"] Jul 06 22:18:46.265485 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:46.265449 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5xlcd\" (UniqueName: \"kubernetes.io/projected/68adbb52-4e6d-4fc0-bda7-271551f24007-kube-api-access-5xlcd\") pod \"payload-processing-58f86c7798-5mvrk\" (UID: \"68adbb52-4e6d-4fc0-bda7-271551f24007\") " pod="openshift-ingress/payload-processing-58f86c7798-5mvrk" Jul 06 22:18:46.265672 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:46.265500 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/68adbb52-4e6d-4fc0-bda7-271551f24007-plugins-config-volume\") pod \"payload-processing-58f86c7798-5mvrk\" (UID: \"68adbb52-4e6d-4fc0-bda7-271551f24007\") " pod="openshift-ingress/payload-processing-58f86c7798-5mvrk" Jul 06 22:18:46.366521 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:46.366421 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/85da56b6-db6b-4aa3-9478-0954a40c4105-plugins-config-volume\") pod \"payload-pre-processing-7964457696-9548w\" (UID: \"85da56b6-db6b-4aa3-9478-0954a40c4105\") " pod="openshift-ingress/payload-pre-processing-7964457696-9548w" Jul 06 22:18:46.366521 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:46.366469 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5xlcd\" (UniqueName: \"kubernetes.io/projected/68adbb52-4e6d-4fc0-bda7-271551f24007-kube-api-access-5xlcd\") pod \"payload-processing-58f86c7798-5mvrk\" (UID: \"68adbb52-4e6d-4fc0-bda7-271551f24007\") " pod="openshift-ingress/payload-processing-58f86c7798-5mvrk" Jul 06 22:18:46.366759 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:46.366547 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/68adbb52-4e6d-4fc0-bda7-271551f24007-plugins-config-volume\") pod \"payload-processing-58f86c7798-5mvrk\" (UID: \"68adbb52-4e6d-4fc0-bda7-271551f24007\") " pod="openshift-ingress/payload-processing-58f86c7798-5mvrk" Jul 06 22:18:46.366759 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:46.366616 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6dscg\" (UniqueName: \"kubernetes.io/projected/85da56b6-db6b-4aa3-9478-0954a40c4105-kube-api-access-6dscg\") pod \"payload-pre-processing-7964457696-9548w\" (UID: \"85da56b6-db6b-4aa3-9478-0954a40c4105\") " pod="openshift-ingress/payload-pre-processing-7964457696-9548w" Jul 06 22:18:46.367527 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:46.367499 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/68adbb52-4e6d-4fc0-bda7-271551f24007-plugins-config-volume\") pod \"payload-processing-58f86c7798-5mvrk\" (UID: \"68adbb52-4e6d-4fc0-bda7-271551f24007\") " pod="openshift-ingress/payload-processing-58f86c7798-5mvrk" Jul 06 22:18:46.411727 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:46.411694 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5xlcd\" (UniqueName: \"kubernetes.io/projected/68adbb52-4e6d-4fc0-bda7-271551f24007-kube-api-access-5xlcd\") pod \"payload-processing-58f86c7798-5mvrk\" (UID: \"68adbb52-4e6d-4fc0-bda7-271551f24007\") " pod="openshift-ingress/payload-processing-58f86c7798-5mvrk" Jul 06 22:18:46.436538 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:46.436511 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-5mvrk" Jul 06 22:18:46.474089 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:46.470514 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/85da56b6-db6b-4aa3-9478-0954a40c4105-plugins-config-volume\") pod \"payload-pre-processing-7964457696-9548w\" (UID: \"85da56b6-db6b-4aa3-9478-0954a40c4105\") " pod="openshift-ingress/payload-pre-processing-7964457696-9548w" Jul 06 22:18:46.474089 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:46.470635 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6dscg\" (UniqueName: \"kubernetes.io/projected/85da56b6-db6b-4aa3-9478-0954a40c4105-kube-api-access-6dscg\") pod \"payload-pre-processing-7964457696-9548w\" (UID: \"85da56b6-db6b-4aa3-9478-0954a40c4105\") " pod="openshift-ingress/payload-pre-processing-7964457696-9548w" Jul 06 22:18:46.474089 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:46.471830 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/85da56b6-db6b-4aa3-9478-0954a40c4105-plugins-config-volume\") pod \"payload-pre-processing-7964457696-9548w\" (UID: \"85da56b6-db6b-4aa3-9478-0954a40c4105\") " pod="openshift-ingress/payload-pre-processing-7964457696-9548w" Jul 06 22:18:46.480060 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:46.480030 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6dscg\" (UniqueName: \"kubernetes.io/projected/85da56b6-db6b-4aa3-9478-0954a40c4105-kube-api-access-6dscg\") pod \"payload-pre-processing-7964457696-9548w\" (UID: \"85da56b6-db6b-4aa3-9478-0954a40c4105\") " pod="openshift-ingress/payload-pre-processing-7964457696-9548w" Jul 06 22:18:46.513041 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:46.513005 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-9548w" Jul 06 22:18:46.583653 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:46.583622 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-5mvrk"] Jul 06 22:18:46.587645 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:18:46.587608 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68adbb52_4e6d_4fc0_bda7_271551f24007.slice/crio-761aaa05e7f86c08f456732967b3c677d9045067c37d0426427dfd4837e3a9b7 WatchSource:0}: Error finding container 761aaa05e7f86c08f456732967b3c677d9045067c37d0426427dfd4837e3a9b7: Status 404 returned error can't find the container with id 761aaa05e7f86c08f456732967b3c677d9045067c37d0426427dfd4837e3a9b7 Jul 06 22:18:46.657516 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:46.657491 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-9548w"] Jul 06 22:18:46.660276 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:18:46.660248 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85da56b6_db6b_4aa3_9478_0954a40c4105.slice/crio-a219b6af94e34ce9324a18e440cd65c422057fbf2d1f368b27f5c0484860ff90 WatchSource:0}: Error finding container a219b6af94e34ce9324a18e440cd65c422057fbf2d1f368b27f5c0484860ff90: Status 404 returned error can't find the container with id a219b6af94e34ce9324a18e440cd65c422057fbf2d1f368b27f5c0484860ff90 Jul 06 22:18:46.919049 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:46.918957 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-9548w" event={"ID":"85da56b6-db6b-4aa3-9478-0954a40c4105","Type":"ContainerStarted","Data":"a219b6af94e34ce9324a18e440cd65c422057fbf2d1f368b27f5c0484860ff90"} Jul 06 22:18:46.919997 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:46.919975 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-5mvrk" event={"ID":"68adbb52-4e6d-4fc0-bda7-271551f24007","Type":"ContainerStarted","Data":"761aaa05e7f86c08f456732967b3c677d9045067c37d0426427dfd4837e3a9b7"} Jul 06 22:18:48.932212 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:48.932173 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-9548w" event={"ID":"85da56b6-db6b-4aa3-9478-0954a40c4105","Type":"ContainerStarted","Data":"fe1776ed180d3b53910bd6c22c144ad1aa830d4d8ef8c6aff0d1997547bea050"} Jul 06 22:18:48.932676 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:48.932279 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7964457696-9548w" Jul 06 22:18:48.939760 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:48.939725 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-5mvrk" event={"ID":"68adbb52-4e6d-4fc0-bda7-271551f24007","Type":"ContainerStarted","Data":"01b0ec5203eb7fec6c7de2792cc1bd8a185f2526c12ad3399345a1b2af822f11"} Jul 06 22:18:48.939917 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:48.939906 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-58f86c7798-5mvrk" Jul 06 22:18:48.952845 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:48.952791 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7964457696-9548w" podStartSLOduration=1.414776365 podStartE2EDuration="2.952749244s" totalImagesPullingTime="1.537972879s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:18:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:18:48.951377566 +0000 UTC m=+608.510172889" watchObservedRunningTime="2026-07-06 22:18:48.952749244 +0000 UTC m=+608.511544567" Jul 06 22:18:48.968541 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:48.968492 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-58f86c7798-5mvrk" podStartSLOduration=1.355210657 podStartE2EDuration="2.96847823s" totalImagesPullingTime="1.613267573s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:18:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:18:48.96785766 +0000 UTC m=+608.526652974" watchObservedRunningTime="2026-07-06 22:18:48.96847823 +0000 UTC m=+608.527273552" Jul 06 22:18:53.971899 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:53.971868 2581 generic.go:356] "Generic (PLEG): container finished" podID="0fd612d1-61ff-411a-a137-dded5a45a55e" containerID="57bf09705b5fa8cd733473969d130d79b1f460b306959f3cf8959f7a01322b67" exitCode=0 Jul 06 22:18:53.972325 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:53.971941 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mbpp9" event={"ID":"0fd612d1-61ff-411a-a137-dded5a45a55e","Type":"ContainerDied","Data":"57bf09705b5fa8cd733473969d130d79b1f460b306959f3cf8959f7a01322b67"} Jul 06 22:18:53.972441 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:53.972424 2581 scope.go:117] "RemoveContainer" containerID="57bf09705b5fa8cd733473969d130d79b1f460b306959f3cf8959f7a01322b67" Jul 06 22:18:53.973488 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:53.973459 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 22:18:53.973602 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:53.973542 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 22:18:53.973602 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:53.973590 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 22:18:54.978445 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:54.978406 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mbpp9" event={"ID":"0fd612d1-61ff-411a-a137-dded5a45a55e","Type":"ContainerStarted","Data":"23aa3ad4461c7ce99c9a903cb0120321b7a10c13b17a2e19a3690a12fc8e3271"} Jul 06 22:18:54.981752 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:54.981735 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mbpp9" Jul 06 22:18:55.413800 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:55.413709 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:18:55.516384 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:55.516344 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:18:55.982146 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:55.982115 2581 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-mbpp9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" start-of-body= Jul 06 22:18:55.982526 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:55.982165 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mbpp9" podUID="0fd612d1-61ff-411a-a137-dded5a45a55e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" Jul 06 22:18:56.982988 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:56.982960 2581 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-mbpp9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" start-of-body= Jul 06 22:18:56.983441 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:56.983412 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mbpp9" podUID="0fd612d1-61ff-411a-a137-dded5a45a55e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" Jul 06 22:18:57.982431 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:57.982388 2581 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-mbpp9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" start-of-body= Jul 06 22:18:57.982640 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:57.982461 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mbpp9" podUID="0fd612d1-61ff-411a-a137-dded5a45a55e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.39:15021/healthz/ready\": dial tcp 10.132.0.39:15021: connect: connection refused" Jul 06 22:18:58.989173 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:58.989145 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mbpp9" Jul 06 22:18:58.989551 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:58.989376 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mbpp9" Jul 06 22:18:58.990127 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:58.990108 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-mbpp9" Jul 06 22:18:59.949754 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:59.949719 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7964457696-9548w" Jul 06 22:18:59.949926 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:18:59.949794 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-58f86c7798-5mvrk" Jul 06 22:19:04.303137 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:19:04.303049 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5ghnm"] Jul 06 22:19:04.307340 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:19:04.307321 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5ghnm" Jul 06 22:19:04.309987 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:19:04.309964 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 06 22:19:04.310990 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:19:04.310970 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-kjlnd\"" Jul 06 22:19:04.311108 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:19:04.311015 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 06 22:19:04.311108 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:19:04.311014 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jul 06 22:19:04.317011 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:19:04.316989 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5ghnm"] Jul 06 22:19:04.323059 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:19:04.323031 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n6jlg\" (UniqueName: \"kubernetes.io/projected/10fce2a3-efad-4bc4-8fc6-8fec4a808c92-kube-api-access-n6jlg\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5ghnm\" (UID: \"10fce2a3-efad-4bc4-8fc6-8fec4a808c92\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5ghnm" Jul 06 22:19:04.323178 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:19:04.323137 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/10fce2a3-efad-4bc4-8fc6-8fec4a808c92-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5ghnm\" (UID: \"10fce2a3-efad-4bc4-8fc6-8fec4a808c92\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5ghnm" Jul 06 22:19:04.323222 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:19:04.323207 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/10fce2a3-efad-4bc4-8fc6-8fec4a808c92-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5ghnm\" (UID: \"10fce2a3-efad-4bc4-8fc6-8fec4a808c92\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5ghnm" Jul 06 22:19:04.323262 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:19:04.323231 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/10fce2a3-efad-4bc4-8fc6-8fec4a808c92-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5ghnm\" (UID: \"10fce2a3-efad-4bc4-8fc6-8fec4a808c92\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5ghnm" Jul 06 22:19:04.323315 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:19:04.323260 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/10fce2a3-efad-4bc4-8fc6-8fec4a808c92-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5ghnm\" (UID: \"10fce2a3-efad-4bc4-8fc6-8fec4a808c92\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5ghnm" Jul 06 22:19:04.323402 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:19:04.323378 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/10fce2a3-efad-4bc4-8fc6-8fec4a808c92-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5ghnm\" (UID: \"10fce2a3-efad-4bc4-8fc6-8fec4a808c92\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5ghnm" Jul 06 22:19:04.330656 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:19:04.330631 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:19:04.424708 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:19:04.424671 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/10fce2a3-efad-4bc4-8fc6-8fec4a808c92-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5ghnm\" (UID: \"10fce2a3-efad-4bc4-8fc6-8fec4a808c92\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5ghnm" Jul 06 22:19:04.424901 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:19:04.424757 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/10fce2a3-efad-4bc4-8fc6-8fec4a808c92-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5ghnm\" (UID: \"10fce2a3-efad-4bc4-8fc6-8fec4a808c92\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5ghnm" Jul 06 22:19:04.424901 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:19:04.424793 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n6jlg\" (UniqueName: \"kubernetes.io/projected/10fce2a3-efad-4bc4-8fc6-8fec4a808c92-kube-api-access-n6jlg\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5ghnm\" (UID: \"10fce2a3-efad-4bc4-8fc6-8fec4a808c92\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5ghnm" Jul 06 22:19:04.425017 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:19:04.424930 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/10fce2a3-efad-4bc4-8fc6-8fec4a808c92-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5ghnm\" (UID: \"10fce2a3-efad-4bc4-8fc6-8fec4a808c92\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5ghnm" Jul 06 22:19:04.425017 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:19:04.425000 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/10fce2a3-efad-4bc4-8fc6-8fec4a808c92-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5ghnm\" (UID: \"10fce2a3-efad-4bc4-8fc6-8fec4a808c92\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5ghnm" Jul 06 22:19:04.425168 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:19:04.425027 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/10fce2a3-efad-4bc4-8fc6-8fec4a808c92-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5ghnm\" (UID: \"10fce2a3-efad-4bc4-8fc6-8fec4a808c92\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5ghnm" Jul 06 22:19:04.425168 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:19:04.425138 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/10fce2a3-efad-4bc4-8fc6-8fec4a808c92-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5ghnm\" (UID: \"10fce2a3-efad-4bc4-8fc6-8fec4a808c92\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5ghnm" Jul 06 22:19:04.425168 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:19:04.425155 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/10fce2a3-efad-4bc4-8fc6-8fec4a808c92-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5ghnm\" (UID: \"10fce2a3-efad-4bc4-8fc6-8fec4a808c92\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5ghnm" Jul 06 22:19:04.425353 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:19:04.425329 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/10fce2a3-efad-4bc4-8fc6-8fec4a808c92-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5ghnm\" (UID: \"10fce2a3-efad-4bc4-8fc6-8fec4a808c92\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5ghnm" Jul 06 22:19:04.427253 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:19:04.427224 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/10fce2a3-efad-4bc4-8fc6-8fec4a808c92-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5ghnm\" (UID: \"10fce2a3-efad-4bc4-8fc6-8fec4a808c92\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5ghnm" Jul 06 22:19:04.427385 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:19:04.427369 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/10fce2a3-efad-4bc4-8fc6-8fec4a808c92-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5ghnm\" (UID: \"10fce2a3-efad-4bc4-8fc6-8fec4a808c92\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5ghnm" Jul 06 22:19:04.435673 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:19:04.435642 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n6jlg\" (UniqueName: \"kubernetes.io/projected/10fce2a3-efad-4bc4-8fc6-8fec4a808c92-kube-api-access-n6jlg\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5ghnm\" (UID: \"10fce2a3-efad-4bc4-8fc6-8fec4a808c92\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5ghnm" Jul 06 22:19:04.618729 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:19:04.618647 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5ghnm" Jul 06 22:19:04.762874 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:19:04.762842 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5ghnm"] Jul 06 22:19:04.764502 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:19:04.764476 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10fce2a3_efad_4bc4_8fc6_8fec4a808c92.slice/crio-e24563bd2e918b855d173eb8b1decb0a1cfbd09d640ab4dc3a8e88ff50807af2 WatchSource:0}: Error finding container e24563bd2e918b855d173eb8b1decb0a1cfbd09d640ab4dc3a8e88ff50807af2: Status 404 returned error can't find the container with id e24563bd2e918b855d173eb8b1decb0a1cfbd09d640ab4dc3a8e88ff50807af2 Jul 06 22:19:05.030960 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:19:05.030914 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5ghnm" event={"ID":"10fce2a3-efad-4bc4-8fc6-8fec4a808c92","Type":"ContainerStarted","Data":"e24563bd2e918b855d173eb8b1decb0a1cfbd09d640ab4dc3a8e88ff50807af2"} Jul 06 22:19:07.041833 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:19:07.041799 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5ghnm" event={"ID":"10fce2a3-efad-4bc4-8fc6-8fec4a808c92","Type":"ContainerStarted","Data":"5fc2d46c6bfe18ddf2641b3d8df5ef11d32c9c6f639f61b2a2af90242a659ad8"} Jul 06 22:19:07.061614 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:19:07.061573 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5ghnm" podStartSLOduration=1.760081339 podStartE2EDuration="3.061558467s" totalImagesPullingTime="1.301477128s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:19:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:19:07.060297657 +0000 UTC m=+626.619092978" watchObservedRunningTime="2026-07-06 22:19:07.061558467 +0000 UTC m=+626.620353800" Jul 06 22:19:11.501838 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:19:11.501803 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-7rzhc"] Jul 06 22:19:11.505918 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:19:11.505892 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7rzhc" Jul 06 22:19:11.508852 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:19:11.508830 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 06 22:19:11.516161 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:19:11.516141 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-7rzhc"] Jul 06 22:19:11.533113 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:19:11.533055 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:19:11.586813 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:19:11.586781 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b9536688-2588-4410-9d32-ca359b04815e-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-7rzhc\" (UID: \"b9536688-2588-4410-9d32-ca359b04815e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7rzhc" Jul 06 22:19:11.586959 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:19:11.586819 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b9536688-2588-4410-9d32-ca359b04815e-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-7rzhc\" (UID: \"b9536688-2588-4410-9d32-ca359b04815e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7rzhc" Jul 06 22:19:11.586959 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:19:11.586895 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-59mg4\" (UniqueName: \"kubernetes.io/projected/b9536688-2588-4410-9d32-ca359b04815e-kube-api-access-59mg4\") pod \"e2e-distinct-simulated-kserve-59975d69f9-7rzhc\" (UID: \"b9536688-2588-4410-9d32-ca359b04815e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7rzhc" Jul 06 22:19:11.586959 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:19:11.586929 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b9536688-2588-4410-9d32-ca359b04815e-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-7rzhc\" (UID: \"b9536688-2588-4410-9d32-ca359b04815e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7rzhc" Jul 06 22:19:11.586959 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:19:11.586945 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b9536688-2588-4410-9d32-ca359b04815e-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-7rzhc\" (UID: \"b9536688-2588-4410-9d32-ca359b04815e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7rzhc" Jul 06 22:19:11.587139 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:19:11.586966 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b9536688-2588-4410-9d32-ca359b04815e-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-7rzhc\" (UID: \"b9536688-2588-4410-9d32-ca359b04815e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7rzhc" Jul 06 22:19:11.687935 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:19:11.687896 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-59mg4\" (UniqueName: \"kubernetes.io/projected/b9536688-2588-4410-9d32-ca359b04815e-kube-api-access-59mg4\") pod \"e2e-distinct-simulated-kserve-59975d69f9-7rzhc\" (UID: \"b9536688-2588-4410-9d32-ca359b04815e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7rzhc" Jul 06 22:19:11.688161 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:19:11.687944 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b9536688-2588-4410-9d32-ca359b04815e-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-7rzhc\" (UID: \"b9536688-2588-4410-9d32-ca359b04815e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7rzhc" Jul 06 22:19:11.688161 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:19:11.687968 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b9536688-2588-4410-9d32-ca359b04815e-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-7rzhc\" (UID: \"b9536688-2588-4410-9d32-ca359b04815e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7rzhc" Jul 06 22:19:11.688161 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:19:11.687997 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b9536688-2588-4410-9d32-ca359b04815e-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-7rzhc\" (UID: \"b9536688-2588-4410-9d32-ca359b04815e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7rzhc" Jul 06 22:19:11.688161 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:19:11.688092 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b9536688-2588-4410-9d32-ca359b04815e-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-7rzhc\" (UID: \"b9536688-2588-4410-9d32-ca359b04815e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7rzhc" Jul 06 22:19:11.688161 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:19:11.688119 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b9536688-2588-4410-9d32-ca359b04815e-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-7rzhc\" (UID: \"b9536688-2588-4410-9d32-ca359b04815e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7rzhc" Jul 06 22:19:11.688460 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:19:11.688362 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b9536688-2588-4410-9d32-ca359b04815e-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-7rzhc\" (UID: \"b9536688-2588-4410-9d32-ca359b04815e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7rzhc" Jul 06 22:19:11.688460 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:19:11.688450 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b9536688-2588-4410-9d32-ca359b04815e-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-7rzhc\" (UID: \"b9536688-2588-4410-9d32-ca359b04815e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7rzhc" Jul 06 22:19:11.688721 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:19:11.688701 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b9536688-2588-4410-9d32-ca359b04815e-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-7rzhc\" (UID: \"b9536688-2588-4410-9d32-ca359b04815e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7rzhc" Jul 06 22:19:11.690448 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:19:11.690426 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b9536688-2588-4410-9d32-ca359b04815e-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-7rzhc\" (UID: \"b9536688-2588-4410-9d32-ca359b04815e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7rzhc" Jul 06 22:19:11.690632 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:19:11.690615 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b9536688-2588-4410-9d32-ca359b04815e-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-7rzhc\" (UID: \"b9536688-2588-4410-9d32-ca359b04815e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7rzhc" Jul 06 22:19:11.696348 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:19:11.696326 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-59mg4\" (UniqueName: \"kubernetes.io/projected/b9536688-2588-4410-9d32-ca359b04815e-kube-api-access-59mg4\") pod \"e2e-distinct-simulated-kserve-59975d69f9-7rzhc\" (UID: \"b9536688-2588-4410-9d32-ca359b04815e\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7rzhc" Jul 06 22:19:11.817090 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:19:11.816985 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7rzhc" Jul 06 22:19:11.950752 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:19:11.950724 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-7rzhc"] Jul 06 22:19:11.952620 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:19:11.952587 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9536688_2588_4410_9d32_ca359b04815e.slice/crio-bc59da0fc98ad6bc3f71834127af3e0deb112ebc7b843f039eed0a19daceffee WatchSource:0}: Error finding container bc59da0fc98ad6bc3f71834127af3e0deb112ebc7b843f039eed0a19daceffee: Status 404 returned error can't find the container with id bc59da0fc98ad6bc3f71834127af3e0deb112ebc7b843f039eed0a19daceffee Jul 06 22:19:12.070925 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:19:12.070841 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7rzhc" event={"ID":"b9536688-2588-4410-9d32-ca359b04815e","Type":"ContainerStarted","Data":"bc59da0fc98ad6bc3f71834127af3e0deb112ebc7b843f039eed0a19daceffee"} Jul 06 22:19:13.076743 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:19:13.076705 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7rzhc" event={"ID":"b9536688-2588-4410-9d32-ca359b04815e","Type":"ContainerStarted","Data":"566834b8bdf677f90204cc45b2e32443e6edf00d64cb73f30f9f611dc6f5fc05"} Jul 06 22:19:13.096741 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:19:13.096694 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7rzhc" podStartSLOduration=1.903085629 podStartE2EDuration="2.096680175s" totalImagesPullingTime="193.594546ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:19:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:19:13.095481406 +0000 UTC m=+632.654276728" watchObservedRunningTime="2026-07-06 22:19:13.096680175 +0000 UTC m=+632.655475495" Jul 06 22:19:14.619134 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:19:14.619099 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5ghnm" Jul 06 22:19:14.619531 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:19:14.619161 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5ghnm" Jul 06 22:19:14.631742 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:19:14.631715 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5ghnm" Jul 06 22:19:15.096530 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:19:15.096500 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5ghnm" Jul 06 22:19:21.817182 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:19:21.817135 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7rzhc" Jul 06 22:19:21.817621 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:19:21.817222 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7rzhc" Jul 06 22:19:21.829878 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:19:21.829850 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7rzhc" Jul 06 22:19:22.131148 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:19:22.131056 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-7rzhc" Jul 06 22:19:22.193457 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:19:22.193410 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:19:44.750017 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:19:44.749979 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:19:53.583639 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:19:53.583606 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-7pmq7"] Jul 06 22:19:53.589308 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:19:53.589282 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7pmq7" Jul 06 22:19:53.591809 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:19:53.591785 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 06 22:19:53.598431 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:19:53.598407 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-7pmq7"] Jul 06 22:19:53.615973 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:19:53.615938 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:19:53.663188 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:19:53.663159 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/763061e4-dbed-450e-9dfa-65f51a94e41f-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-7pmq7\" (UID: \"763061e4-dbed-450e-9dfa-65f51a94e41f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7pmq7" Jul 06 22:19:53.663188 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:19:53.663198 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/763061e4-dbed-450e-9dfa-65f51a94e41f-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-7pmq7\" (UID: \"763061e4-dbed-450e-9dfa-65f51a94e41f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7pmq7" Jul 06 22:19:53.663398 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:19:53.663278 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/763061e4-dbed-450e-9dfa-65f51a94e41f-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-7pmq7\" (UID: \"763061e4-dbed-450e-9dfa-65f51a94e41f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7pmq7" Jul 06 22:19:53.663398 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:19:53.663341 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/763061e4-dbed-450e-9dfa-65f51a94e41f-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-7pmq7\" (UID: \"763061e4-dbed-450e-9dfa-65f51a94e41f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7pmq7" Jul 06 22:19:53.663398 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:19:53.663377 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/763061e4-dbed-450e-9dfa-65f51a94e41f-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-7pmq7\" (UID: \"763061e4-dbed-450e-9dfa-65f51a94e41f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7pmq7" Jul 06 22:19:53.663509 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:19:53.663415 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cbcn7\" (UniqueName: \"kubernetes.io/projected/763061e4-dbed-450e-9dfa-65f51a94e41f-kube-api-access-cbcn7\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-7pmq7\" (UID: \"763061e4-dbed-450e-9dfa-65f51a94e41f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7pmq7" Jul 06 22:19:53.764371 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:19:53.764339 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/763061e4-dbed-450e-9dfa-65f51a94e41f-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-7pmq7\" (UID: \"763061e4-dbed-450e-9dfa-65f51a94e41f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7pmq7" Jul 06 22:19:53.764535 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:19:53.764379 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cbcn7\" (UniqueName: \"kubernetes.io/projected/763061e4-dbed-450e-9dfa-65f51a94e41f-kube-api-access-cbcn7\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-7pmq7\" (UID: \"763061e4-dbed-450e-9dfa-65f51a94e41f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7pmq7" Jul 06 22:19:53.764535 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:19:53.764416 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/763061e4-dbed-450e-9dfa-65f51a94e41f-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-7pmq7\" (UID: \"763061e4-dbed-450e-9dfa-65f51a94e41f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7pmq7" Jul 06 22:19:53.764535 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:19:53.764439 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/763061e4-dbed-450e-9dfa-65f51a94e41f-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-7pmq7\" (UID: \"763061e4-dbed-450e-9dfa-65f51a94e41f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7pmq7" Jul 06 22:19:53.764535 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:19:53.764485 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/763061e4-dbed-450e-9dfa-65f51a94e41f-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-7pmq7\" (UID: \"763061e4-dbed-450e-9dfa-65f51a94e41f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7pmq7" Jul 06 22:19:53.764535 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:19:53.764526 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/763061e4-dbed-450e-9dfa-65f51a94e41f-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-7pmq7\" (UID: \"763061e4-dbed-450e-9dfa-65f51a94e41f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7pmq7" Jul 06 22:19:53.764819 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:19:53.764795 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/763061e4-dbed-450e-9dfa-65f51a94e41f-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-7pmq7\" (UID: \"763061e4-dbed-450e-9dfa-65f51a94e41f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7pmq7" Jul 06 22:19:53.764893 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:19:53.764877 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/763061e4-dbed-450e-9dfa-65f51a94e41f-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-7pmq7\" (UID: \"763061e4-dbed-450e-9dfa-65f51a94e41f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7pmq7" Jul 06 22:19:53.765024 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:19:53.764990 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/763061e4-dbed-450e-9dfa-65f51a94e41f-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-7pmq7\" (UID: \"763061e4-dbed-450e-9dfa-65f51a94e41f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7pmq7" Jul 06 22:19:53.766805 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:19:53.766781 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/763061e4-dbed-450e-9dfa-65f51a94e41f-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-7pmq7\" (UID: \"763061e4-dbed-450e-9dfa-65f51a94e41f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7pmq7" Jul 06 22:19:53.766925 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:19:53.766910 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/763061e4-dbed-450e-9dfa-65f51a94e41f-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-7pmq7\" (UID: \"763061e4-dbed-450e-9dfa-65f51a94e41f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7pmq7" Jul 06 22:19:53.773147 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:19:53.773127 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cbcn7\" (UniqueName: \"kubernetes.io/projected/763061e4-dbed-450e-9dfa-65f51a94e41f-kube-api-access-cbcn7\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-7pmq7\" (UID: \"763061e4-dbed-450e-9dfa-65f51a94e41f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7pmq7" Jul 06 22:19:53.901278 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:19:53.901196 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7pmq7" Jul 06 22:19:54.036302 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:19:54.036270 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-7pmq7"] Jul 06 22:19:54.039414 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:19:54.039319 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod763061e4_dbed_450e_9dfa_65f51a94e41f.slice/crio-f7461fd33c742f035efd7969de6ca5dae6f4e5ce7d3feb98d2056648cf6544f5 WatchSource:0}: Error finding container f7461fd33c742f035efd7969de6ca5dae6f4e5ce7d3feb98d2056648cf6544f5: Status 404 returned error can't find the container with id f7461fd33c742f035efd7969de6ca5dae6f4e5ce7d3feb98d2056648cf6544f5 Jul 06 22:19:54.040470 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:19:54.040440 2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 22:19:54.274788 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:19:54.274746 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7pmq7" event={"ID":"763061e4-dbed-450e-9dfa-65f51a94e41f","Type":"ContainerStarted","Data":"53f025a5438b3d48bf50db26f6bda2cdc94999ada5d973f20fb1a5a098a6e149"} Jul 06 22:19:54.274905 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:19:54.274801 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7pmq7" event={"ID":"763061e4-dbed-450e-9dfa-65f51a94e41f","Type":"ContainerStarted","Data":"f7461fd33c742f035efd7969de6ca5dae6f4e5ce7d3feb98d2056648cf6544f5"} Jul 06 22:19:54.296037 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:19:54.295992 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7pmq7" podStartSLOduration=1.126356017 podStartE2EDuration="1.29597809s" totalImagesPullingTime="169.622073ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:19:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:19:54.294158542 +0000 UTC m=+673.852953863" watchObservedRunningTime="2026-07-06 22:19:54.29597809 +0000 UTC m=+673.854773411" Jul 06 22:20:00.871463 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:20:00.871427 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:20:00.933614 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:20:00.933579 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:20:03.902157 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:20:03.902115 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7pmq7" Jul 06 22:20:03.902576 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:20:03.902179 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7pmq7" Jul 06 22:20:03.915164 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:20:03.915134 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7pmq7" Jul 06 22:20:04.339012 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:20:04.338978 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-7pmq7" Jul 06 22:20:19.317995 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:20:19.317957 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:20:56.867000 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:20:56.866964 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:21:07.618665 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:21:07.618619 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:21:07.696827 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:21:07.696790 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:21:14.982136 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:21:14.982100 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:21:15.052417 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:21:15.052387 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:21:17.566352 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:21:17.566319 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:21:17.663642 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:21:17.663611 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:21:18.829157 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:21:18.829117 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:21:18.995758 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:21:18.995723 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:21:19.053336 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:21:19.053292 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59fc789756-4zh9d"] Jul 06 22:21:19.053542 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:21:19.053514 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-59fc789756-4zh9d" podUID="0d712e25-5eb7-446d-8199-242e4ca84067" containerName="authorino" containerID="cri-o://2cc4c395295839cb969a9798c84e0cc872f7dc46d0936bafb752f52ff442c734" gracePeriod=30 Jul 06 22:21:19.318948 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:21:19.318924 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59fc789756-4zh9d" Jul 06 22:21:19.434631 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:21:19.434600 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6j496\" (UniqueName: \"kubernetes.io/projected/0d712e25-5eb7-446d-8199-242e4ca84067-kube-api-access-6j496\") pod \"0d712e25-5eb7-446d-8199-242e4ca84067\" (UID: \"0d712e25-5eb7-446d-8199-242e4ca84067\") " Jul 06 22:21:19.434829 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:21:19.434693 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0d712e25-5eb7-446d-8199-242e4ca84067-tls-cert\") pod \"0d712e25-5eb7-446d-8199-242e4ca84067\" (UID: \"0d712e25-5eb7-446d-8199-242e4ca84067\") " Jul 06 22:21:19.436720 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:21:19.436694 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0d712e25-5eb7-446d-8199-242e4ca84067-kube-api-access-6j496" (OuterVolumeSpecName: "kube-api-access-6j496") pod "0d712e25-5eb7-446d-8199-242e4ca84067" (UID: "0d712e25-5eb7-446d-8199-242e4ca84067"). InnerVolumeSpecName "kube-api-access-6j496". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 22:21:19.447221 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:21:19.447189 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0d712e25-5eb7-446d-8199-242e4ca84067-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "0d712e25-5eb7-446d-8199-242e4ca84067" (UID: "0d712e25-5eb7-446d-8199-242e4ca84067"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 22:21:19.536198 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:21:19.536166 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6j496\" (UniqueName: \"kubernetes.io/projected/0d712e25-5eb7-446d-8199-242e4ca84067-kube-api-access-6j496\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:21:19.536198 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:21:19.536193 2581 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/0d712e25-5eb7-446d-8199-242e4ca84067-tls-cert\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:21:19.711651 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:21:19.711563 2581 generic.go:356] "Generic (PLEG): container finished" podID="0d712e25-5eb7-446d-8199-242e4ca84067" containerID="2cc4c395295839cb969a9798c84e0cc872f7dc46d0936bafb752f52ff442c734" exitCode=0 Jul 06 22:21:19.711651 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:21:19.711603 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59fc789756-4zh9d" Jul 06 22:21:19.711833 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:21:19.711652 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59fc789756-4zh9d" event={"ID":"0d712e25-5eb7-446d-8199-242e4ca84067","Type":"ContainerDied","Data":"2cc4c395295839cb969a9798c84e0cc872f7dc46d0936bafb752f52ff442c734"} Jul 06 22:21:19.711833 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:21:19.711688 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59fc789756-4zh9d" event={"ID":"0d712e25-5eb7-446d-8199-242e4ca84067","Type":"ContainerDied","Data":"090d81e20c0f7b22228daef81ef4c8dbf8110fc32ec85f2c182a4af4f0964743"} Jul 06 22:21:19.711833 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:21:19.711710 2581 scope.go:117] "RemoveContainer" containerID="2cc4c395295839cb969a9798c84e0cc872f7dc46d0936bafb752f52ff442c734" Jul 06 22:21:19.720646 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:21:19.720630 2581 scope.go:117] "RemoveContainer" containerID="2cc4c395295839cb969a9798c84e0cc872f7dc46d0936bafb752f52ff442c734" Jul 06 22:21:19.720876 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:21:19.720858 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2cc4c395295839cb969a9798c84e0cc872f7dc46d0936bafb752f52ff442c734\": container with ID starting with 2cc4c395295839cb969a9798c84e0cc872f7dc46d0936bafb752f52ff442c734 not found: ID does not exist" containerID="2cc4c395295839cb969a9798c84e0cc872f7dc46d0936bafb752f52ff442c734" Jul 06 22:21:19.720926 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:21:19.720884 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2cc4c395295839cb969a9798c84e0cc872f7dc46d0936bafb752f52ff442c734"} err="failed to get container status \"2cc4c395295839cb969a9798c84e0cc872f7dc46d0936bafb752f52ff442c734\": rpc error: code = NotFound desc = could not find container \"2cc4c395295839cb969a9798c84e0cc872f7dc46d0936bafb752f52ff442c734\": container with ID starting with 2cc4c395295839cb969a9798c84e0cc872f7dc46d0936bafb752f52ff442c734 not found: ID does not exist" Jul 06 22:21:19.734632 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:21:19.734607 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59fc789756-4zh9d"] Jul 06 22:21:19.748248 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:21:19.748222 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-59fc789756-4zh9d"] Jul 06 22:21:20.978129 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:21:20.978098 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0d712e25-5eb7-446d-8199-242e4ca84067" path="/var/lib/kubelet/pods/0d712e25-5eb7-446d-8199-242e4ca84067/volumes" Jul 06 22:21:40.005578 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:21:40.005535 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:22:28.986916 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:22:28.986879 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:22:29.100267 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:22:29.100232 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5fb54f555-x797t"] Jul 06 22:22:29.100443 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:22:29.100420 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-5fb54f555-x797t" podUID="82ae67d0-8e90-48d5-bad6-6193f3991b61" containerName="manager" containerID="cri-o://82b5e3940dbb902e5f08b5a044fba92fb0119d74a5f65f4b5e6fbf5dd6a930a2" gracePeriod=30 Jul 06 22:22:29.353446 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:22:29.353421 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5fb54f555-x797t" Jul 06 22:22:29.370708 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:22:29.370683 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hwvf2\" (UniqueName: \"kubernetes.io/projected/82ae67d0-8e90-48d5-bad6-6193f3991b61-kube-api-access-hwvf2\") pod \"82ae67d0-8e90-48d5-bad6-6193f3991b61\" (UID: \"82ae67d0-8e90-48d5-bad6-6193f3991b61\") " Jul 06 22:22:29.370861 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:22:29.370731 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/82ae67d0-8e90-48d5-bad6-6193f3991b61-webhook-cert\") pod \"82ae67d0-8e90-48d5-bad6-6193f3991b61\" (UID: \"82ae67d0-8e90-48d5-bad6-6193f3991b61\") " Jul 06 22:22:29.372689 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:22:29.372645 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/82ae67d0-8e90-48d5-bad6-6193f3991b61-kube-api-access-hwvf2" (OuterVolumeSpecName: "kube-api-access-hwvf2") pod "82ae67d0-8e90-48d5-bad6-6193f3991b61" (UID: "82ae67d0-8e90-48d5-bad6-6193f3991b61"). InnerVolumeSpecName "kube-api-access-hwvf2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 22:22:29.372811 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:22:29.372729 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/82ae67d0-8e90-48d5-bad6-6193f3991b61-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "82ae67d0-8e90-48d5-bad6-6193f3991b61" (UID: "82ae67d0-8e90-48d5-bad6-6193f3991b61"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 22:22:29.471906 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:22:29.471876 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hwvf2\" (UniqueName: \"kubernetes.io/projected/82ae67d0-8e90-48d5-bad6-6193f3991b61-kube-api-access-hwvf2\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:22:29.471906 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:22:29.471908 2581 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/82ae67d0-8e90-48d5-bad6-6193f3991b61-webhook-cert\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:22:30.069115 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:22:30.069055 2581 generic.go:356] "Generic (PLEG): container finished" podID="82ae67d0-8e90-48d5-bad6-6193f3991b61" containerID="82b5e3940dbb902e5f08b5a044fba92fb0119d74a5f65f4b5e6fbf5dd6a930a2" exitCode=0 Jul 06 22:22:30.069588 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:22:30.069130 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5fb54f555-x797t" Jul 06 22:22:30.069588 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:22:30.069170 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5fb54f555-x797t" event={"ID":"82ae67d0-8e90-48d5-bad6-6193f3991b61","Type":"ContainerDied","Data":"82b5e3940dbb902e5f08b5a044fba92fb0119d74a5f65f4b5e6fbf5dd6a930a2"} Jul 06 22:22:30.069588 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:22:30.069239 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5fb54f555-x797t" event={"ID":"82ae67d0-8e90-48d5-bad6-6193f3991b61","Type":"ContainerDied","Data":"2fba67abf4903ce3a1233a545aae1e06eb293a1187de529f30416e89679d84cd"} Jul 06 22:22:30.069588 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:22:30.069271 2581 scope.go:117] "RemoveContainer" containerID="82b5e3940dbb902e5f08b5a044fba92fb0119d74a5f65f4b5e6fbf5dd6a930a2" Jul 06 22:22:30.085318 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:22:30.085294 2581 scope.go:117] "RemoveContainer" containerID="82b5e3940dbb902e5f08b5a044fba92fb0119d74a5f65f4b5e6fbf5dd6a930a2" Jul 06 22:22:30.085586 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:22:30.085566 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"82b5e3940dbb902e5f08b5a044fba92fb0119d74a5f65f4b5e6fbf5dd6a930a2\": container with ID starting with 82b5e3940dbb902e5f08b5a044fba92fb0119d74a5f65f4b5e6fbf5dd6a930a2 not found: ID does not exist" containerID="82b5e3940dbb902e5f08b5a044fba92fb0119d74a5f65f4b5e6fbf5dd6a930a2" Jul 06 22:22:30.085642 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:22:30.085594 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"82b5e3940dbb902e5f08b5a044fba92fb0119d74a5f65f4b5e6fbf5dd6a930a2"} err="failed to get container status \"82b5e3940dbb902e5f08b5a044fba92fb0119d74a5f65f4b5e6fbf5dd6a930a2\": rpc error: code = NotFound desc = could not find container \"82b5e3940dbb902e5f08b5a044fba92fb0119d74a5f65f4b5e6fbf5dd6a930a2\": container with ID starting with 82b5e3940dbb902e5f08b5a044fba92fb0119d74a5f65f4b5e6fbf5dd6a930a2 not found: ID does not exist" Jul 06 22:22:30.102001 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:22:30.101978 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5fb54f555-x797t"] Jul 06 22:22:30.105819 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:22:30.105797 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-5fb54f555-x797t"] Jul 06 22:22:30.795350 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:22:30.795308 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:22:30.907948 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:22:30.907920 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5fb54f555-mbndp"] Jul 06 22:22:30.908456 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:22:30.908438 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0d712e25-5eb7-446d-8199-242e4ca84067" containerName="authorino" Jul 06 22:22:30.908456 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:22:30.908456 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d712e25-5eb7-446d-8199-242e4ca84067" containerName="authorino" Jul 06 22:22:30.908592 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:22:30.908487 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="82ae67d0-8e90-48d5-bad6-6193f3991b61" containerName="manager" Jul 06 22:22:30.908592 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:22:30.908493 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="82ae67d0-8e90-48d5-bad6-6193f3991b61" containerName="manager" Jul 06 22:22:30.908592 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:22:30.908555 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="82ae67d0-8e90-48d5-bad6-6193f3991b61" containerName="manager" Jul 06 22:22:30.908592 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:22:30.908569 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="0d712e25-5eb7-446d-8199-242e4ca84067" containerName="authorino" Jul 06 22:22:30.912821 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:22:30.912806 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5fb54f555-mbndp" Jul 06 22:22:30.915766 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:22:30.915751 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 06 22:22:30.916709 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:22:30.916689 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-z45sk\"" Jul 06 22:22:30.916709 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:22:30.916704 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 06 22:22:30.927632 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:22:30.927607 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5fb54f555-mbndp"] Jul 06 22:22:30.977171 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:22:30.977140 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="82ae67d0-8e90-48d5-bad6-6193f3991b61" path="/var/lib/kubelet/pods/82ae67d0-8e90-48d5-bad6-6193f3991b61/volumes" Jul 06 22:22:30.986570 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:22:30.986545 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/aca1a96f-d595-4aec-a683-47ee9930a384-webhook-cert\") pod \"maas-controller-5fb54f555-mbndp\" (UID: \"aca1a96f-d595-4aec-a683-47ee9930a384\") " pod="opendatahub/maas-controller-5fb54f555-mbndp" Jul 06 22:22:30.986672 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:22:30.986578 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cz52f\" (UniqueName: \"kubernetes.io/projected/aca1a96f-d595-4aec-a683-47ee9930a384-kube-api-access-cz52f\") pod \"maas-controller-5fb54f555-mbndp\" (UID: \"aca1a96f-d595-4aec-a683-47ee9930a384\") " pod="opendatahub/maas-controller-5fb54f555-mbndp" Jul 06 22:22:31.088085 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:22:31.087992 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/aca1a96f-d595-4aec-a683-47ee9930a384-webhook-cert\") pod \"maas-controller-5fb54f555-mbndp\" (UID: \"aca1a96f-d595-4aec-a683-47ee9930a384\") " pod="opendatahub/maas-controller-5fb54f555-mbndp" Jul 06 22:22:31.088085 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:22:31.088028 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cz52f\" (UniqueName: \"kubernetes.io/projected/aca1a96f-d595-4aec-a683-47ee9930a384-kube-api-access-cz52f\") pod \"maas-controller-5fb54f555-mbndp\" (UID: \"aca1a96f-d595-4aec-a683-47ee9930a384\") " pod="opendatahub/maas-controller-5fb54f555-mbndp" Jul 06 22:22:31.090429 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:22:31.090395 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/aca1a96f-d595-4aec-a683-47ee9930a384-webhook-cert\") pod \"maas-controller-5fb54f555-mbndp\" (UID: \"aca1a96f-d595-4aec-a683-47ee9930a384\") " pod="opendatahub/maas-controller-5fb54f555-mbndp" Jul 06 22:22:31.097331 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:22:31.097301 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cz52f\" (UniqueName: \"kubernetes.io/projected/aca1a96f-d595-4aec-a683-47ee9930a384-kube-api-access-cz52f\") pod \"maas-controller-5fb54f555-mbndp\" (UID: \"aca1a96f-d595-4aec-a683-47ee9930a384\") " pod="opendatahub/maas-controller-5fb54f555-mbndp" Jul 06 22:22:31.223191 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:22:31.223159 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5fb54f555-mbndp" Jul 06 22:22:31.364573 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:22:31.364526 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5fb54f555-mbndp"] Jul 06 22:22:31.367240 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:22:31.367208 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaca1a96f_d595_4aec_a683_47ee9930a384.slice/crio-40a200c761349c9d10224dcecbfae8c14cfc448334c6b979d8dec8859764012e WatchSource:0}: Error finding container 40a200c761349c9d10224dcecbfae8c14cfc448334c6b979d8dec8859764012e: Status 404 returned error can't find the container with id 40a200c761349c9d10224dcecbfae8c14cfc448334c6b979d8dec8859764012e Jul 06 22:22:32.082461 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:22:32.082435 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5fb54f555-mbndp" event={"ID":"aca1a96f-d595-4aec-a683-47ee9930a384","Type":"ContainerStarted","Data":"40a200c761349c9d10224dcecbfae8c14cfc448334c6b979d8dec8859764012e"} Jul 06 22:22:33.089148 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:22:33.089112 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5fb54f555-mbndp" event={"ID":"aca1a96f-d595-4aec-a683-47ee9930a384","Type":"ContainerStarted","Data":"ac0d4fe337cfa2313af8daade7a6c02e76219bf9739750541f681915186e99d2"} Jul 06 22:22:33.089613 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:22:33.089291 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5fb54f555-mbndp" Jul 06 22:22:33.108978 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:22:33.108933 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5fb54f555-mbndp" podStartSLOduration=2.453151574 podStartE2EDuration="3.10891885s" totalImagesPullingTime="655.767276ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:22:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:22:33.107818353 +0000 UTC m=+832.666613674" watchObservedRunningTime="2026-07-06 22:22:33.10891885 +0000 UTC m=+832.667714170" Jul 06 22:22:44.100578 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:22:44.100546 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5fb54f555-mbndp" Jul 06 22:23:12.578675 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:23:12.578627 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:23:12.665655 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:23:12.665611 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:23:12.771274 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:23:12.771234 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:23:18.740483 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:23:18.740442 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:23:18.848428 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:23:18.848391 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:23:18.942042 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:23:18.942001 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:23:19.089102 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:23:19.088985 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:23:25.119317 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:23:25.119263 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:23:25.220274 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:23:25.220233 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:23:40.930736 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:23:40.930661 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-rcn5q_86b07166-2f34-443b-9c61-86b7d950ad7b/console-operator/1.log" Jul 06 22:23:40.931796 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:23:40.931774 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-rcn5q_86b07166-2f34-443b-9c61-86b7d950ad7b/console-operator/1.log" Jul 06 22:23:41.243281 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:23:41.243186 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:24:00.705896 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:24:00.705858 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:26:11.130734 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:26:11.130698 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:27:14.209593 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:27:14.209559 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-njfz2"] Jul 06 22:27:14.210414 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:27:14.209759 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-njfz2" podUID="7993f081-3f17-4868-bfc3-285e366f68fb" containerName="manager" containerID="cri-o://a0b2ee72665a221b0e813a841f8848eeb780bcd2c1cc9dda49ef41ba6dbb0c51" gracePeriod=10 Jul 06 22:27:16.772917 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:27:16.772893 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-njfz2" Jul 06 22:27:16.895839 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:27:16.895750 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7993f081-3f17-4868-bfc3-285e366f68fb-extensions-socket-volume\") pod \"7993f081-3f17-4868-bfc3-285e366f68fb\" (UID: \"7993f081-3f17-4868-bfc3-285e366f68fb\") " Jul 06 22:27:16.895839 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:27:16.895833 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xvwfj\" (UniqueName: \"kubernetes.io/projected/7993f081-3f17-4868-bfc3-285e366f68fb-kube-api-access-xvwfj\") pod \"7993f081-3f17-4868-bfc3-285e366f68fb\" (UID: \"7993f081-3f17-4868-bfc3-285e366f68fb\") " Jul 06 22:27:16.896139 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:27:16.896116 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7993f081-3f17-4868-bfc3-285e366f68fb-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "7993f081-3f17-4868-bfc3-285e366f68fb" (UID: "7993f081-3f17-4868-bfc3-285e366f68fb"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:27:16.897901 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:27:16.897878 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7993f081-3f17-4868-bfc3-285e366f68fb-kube-api-access-xvwfj" (OuterVolumeSpecName: "kube-api-access-xvwfj") pod "7993f081-3f17-4868-bfc3-285e366f68fb" (UID: "7993f081-3f17-4868-bfc3-285e366f68fb"). InnerVolumeSpecName "kube-api-access-xvwfj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 22:27:16.996976 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:27:16.996951 2581 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7993f081-3f17-4868-bfc3-285e366f68fb-extensions-socket-volume\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:27:16.996976 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:27:16.996976 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xvwfj\" (UniqueName: \"kubernetes.io/projected/7993f081-3f17-4868-bfc3-285e366f68fb-kube-api-access-xvwfj\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:27:17.486932 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:27:17.486901 2581 generic.go:356] "Generic (PLEG): container finished" podID="7993f081-3f17-4868-bfc3-285e366f68fb" containerID="a0b2ee72665a221b0e813a841f8848eeb780bcd2c1cc9dda49ef41ba6dbb0c51" exitCode=0 Jul 06 22:27:17.487142 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:27:17.486943 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-njfz2" Jul 06 22:27:17.487142 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:27:17.486987 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-njfz2" event={"ID":"7993f081-3f17-4868-bfc3-285e366f68fb","Type":"ContainerDied","Data":"a0b2ee72665a221b0e813a841f8848eeb780bcd2c1cc9dda49ef41ba6dbb0c51"} Jul 06 22:27:17.487142 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:27:17.487042 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-njfz2" event={"ID":"7993f081-3f17-4868-bfc3-285e366f68fb","Type":"ContainerDied","Data":"e7e1d7565f3fe00aa6c41c3f8e2200115702ad22a213c66403d1454a6a39fd93"} Jul 06 22:27:17.487142 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:27:17.487079 2581 scope.go:117] "RemoveContainer" containerID="a0b2ee72665a221b0e813a841f8848eeb780bcd2c1cc9dda49ef41ba6dbb0c51" Jul 06 22:27:17.496749 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:27:17.496729 2581 scope.go:117] "RemoveContainer" containerID="a0b2ee72665a221b0e813a841f8848eeb780bcd2c1cc9dda49ef41ba6dbb0c51" Jul 06 22:27:17.496991 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:27:17.496971 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a0b2ee72665a221b0e813a841f8848eeb780bcd2c1cc9dda49ef41ba6dbb0c51\": container with ID starting with a0b2ee72665a221b0e813a841f8848eeb780bcd2c1cc9dda49ef41ba6dbb0c51 not found: ID does not exist" containerID="a0b2ee72665a221b0e813a841f8848eeb780bcd2c1cc9dda49ef41ba6dbb0c51" Jul 06 22:27:17.497123 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:27:17.496998 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a0b2ee72665a221b0e813a841f8848eeb780bcd2c1cc9dda49ef41ba6dbb0c51"} err="failed to get container status \"a0b2ee72665a221b0e813a841f8848eeb780bcd2c1cc9dda49ef41ba6dbb0c51\": rpc error: code = NotFound desc = could not find container \"a0b2ee72665a221b0e813a841f8848eeb780bcd2c1cc9dda49ef41ba6dbb0c51\": container with ID starting with a0b2ee72665a221b0e813a841f8848eeb780bcd2c1cc9dda49ef41ba6dbb0c51 not found: ID does not exist" Jul 06 22:27:17.510193 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:27:17.510158 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-njfz2"] Jul 06 22:27:17.515228 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:27:17.515203 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-njfz2"] Jul 06 22:27:18.976674 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:27:18.976640 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7993f081-3f17-4868-bfc3-285e366f68fb" path="/var/lib/kubelet/pods/7993f081-3f17-4868-bfc3-285e366f68fb/volumes" Jul 06 22:28:15.370898 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:28:15.370817 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-f468p"] Jul 06 22:28:15.371400 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:28:15.371223 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7993f081-3f17-4868-bfc3-285e366f68fb" containerName="manager" Jul 06 22:28:15.371400 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:28:15.371237 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="7993f081-3f17-4868-bfc3-285e366f68fb" containerName="manager" Jul 06 22:28:15.371400 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:28:15.371303 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="7993f081-3f17-4868-bfc3-285e366f68fb" containerName="manager" Jul 06 22:28:15.374389 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:28:15.374370 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-f468p" Jul 06 22:28:15.378320 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:28:15.378300 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-cdlgd\"" Jul 06 22:28:15.405420 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:28:15.405394 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-f468p"] Jul 06 22:28:15.519631 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:28:15.519598 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b25f696a-3383-4add-a8ff-07711818c31d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-f468p\" (UID: \"b25f696a-3383-4add-a8ff-07711818c31d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-f468p" Jul 06 22:28:15.519805 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:28:15.519653 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l8bcf\" (UniqueName: \"kubernetes.io/projected/b25f696a-3383-4add-a8ff-07711818c31d-kube-api-access-l8bcf\") pod \"kuadrant-operator-controller-manager-86878c769f-f468p\" (UID: \"b25f696a-3383-4add-a8ff-07711818c31d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-f468p" Jul 06 22:28:15.620838 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:28:15.620801 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l8bcf\" (UniqueName: \"kubernetes.io/projected/b25f696a-3383-4add-a8ff-07711818c31d-kube-api-access-l8bcf\") pod \"kuadrant-operator-controller-manager-86878c769f-f468p\" (UID: \"b25f696a-3383-4add-a8ff-07711818c31d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-f468p" Jul 06 22:28:15.621036 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:28:15.620890 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b25f696a-3383-4add-a8ff-07711818c31d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-f468p\" (UID: \"b25f696a-3383-4add-a8ff-07711818c31d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-f468p" Jul 06 22:28:15.621339 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:28:15.621318 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b25f696a-3383-4add-a8ff-07711818c31d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-f468p\" (UID: \"b25f696a-3383-4add-a8ff-07711818c31d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-f468p" Jul 06 22:28:15.631669 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:28:15.631647 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l8bcf\" (UniqueName: \"kubernetes.io/projected/b25f696a-3383-4add-a8ff-07711818c31d-kube-api-access-l8bcf\") pod \"kuadrant-operator-controller-manager-86878c769f-f468p\" (UID: \"b25f696a-3383-4add-a8ff-07711818c31d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-f468p" Jul 06 22:28:15.684123 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:28:15.684095 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-f468p" Jul 06 22:28:15.827878 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:28:15.827854 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-f468p"] Jul 06 22:28:15.829750 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:28:15.829715 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb25f696a_3383_4add_a8ff_07711818c31d.slice/crio-b12da806ce5019659d02950d64ecdad2193da18c8c42e6dc511e180faeec3b90 WatchSource:0}: Error finding container b12da806ce5019659d02950d64ecdad2193da18c8c42e6dc511e180faeec3b90: Status 404 returned error can't find the container with id b12da806ce5019659d02950d64ecdad2193da18c8c42e6dc511e180faeec3b90 Jul 06 22:28:16.779322 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:28:16.779285 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-f468p" event={"ID":"b25f696a-3383-4add-a8ff-07711818c31d","Type":"ContainerStarted","Data":"98a01b2741635d95fc1e8f34f11a657e2d49a9cb4c873ac3828a1be37b361489"} Jul 06 22:28:16.779737 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:28:16.779347 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-f468p" event={"ID":"b25f696a-3383-4add-a8ff-07711818c31d","Type":"ContainerStarted","Data":"b12da806ce5019659d02950d64ecdad2193da18c8c42e6dc511e180faeec3b90"} Jul 06 22:28:16.779737 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:28:16.779375 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-f468p" Jul 06 22:28:16.808307 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:28:16.808259 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-f468p" podStartSLOduration=1.808243796 podStartE2EDuration="1.808243796s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:28:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:28:16.806779212 +0000 UTC m=+1176.365574533" watchObservedRunningTime="2026-07-06 22:28:16.808243796 +0000 UTC m=+1176.367039118" Jul 06 22:28:27.786590 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:28:27.786562 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-f468p" Jul 06 22:28:40.960695 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:28:40.960662 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-rcn5q_86b07166-2f34-443b-9c61-86b7d950ad7b/console-operator/1.log" Jul 06 22:28:40.963417 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:28:40.963393 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-rcn5q_86b07166-2f34-443b-9c61-86b7d950ad7b/console-operator/1.log" Jul 06 22:29:08.087599 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:29:08.087559 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:29:08.174806 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:29:08.174777 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:29:31.834547 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:29:31.834509 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:30:00.153254 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:30:00.153218 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-29722950-79m5s"] Jul 06 22:30:00.156776 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:30:00.156758 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29722950-79m5s" Jul 06 22:30:00.160059 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:30:00.160041 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-kw98h\"" Jul 06 22:30:00.173267 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:30:00.173235 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29722950-79m5s"] Jul 06 22:30:00.265465 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:30:00.265431 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-srw5x\" (UniqueName: \"kubernetes.io/projected/1a705704-330f-498f-b7a3-696be4ace5a6-kube-api-access-srw5x\") pod \"maas-api-key-cleanup-29722950-79m5s\" (UID: \"1a705704-330f-498f-b7a3-696be4ace5a6\") " pod="opendatahub/maas-api-key-cleanup-29722950-79m5s" Jul 06 22:30:00.366919 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:30:00.366886 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-srw5x\" (UniqueName: \"kubernetes.io/projected/1a705704-330f-498f-b7a3-696be4ace5a6-kube-api-access-srw5x\") pod \"maas-api-key-cleanup-29722950-79m5s\" (UID: \"1a705704-330f-498f-b7a3-696be4ace5a6\") " pod="opendatahub/maas-api-key-cleanup-29722950-79m5s" Jul 06 22:30:00.381490 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:30:00.381461 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-srw5x\" (UniqueName: \"kubernetes.io/projected/1a705704-330f-498f-b7a3-696be4ace5a6-kube-api-access-srw5x\") pod \"maas-api-key-cleanup-29722950-79m5s\" (UID: \"1a705704-330f-498f-b7a3-696be4ace5a6\") " pod="opendatahub/maas-api-key-cleanup-29722950-79m5s" Jul 06 22:30:00.467108 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:30:00.467075 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29722950-79m5s" Jul 06 22:30:00.603328 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:30:00.603187 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29722950-79m5s"] Jul 06 22:30:00.605583 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:30:00.605552 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a705704_330f_498f_b7a3_696be4ace5a6.slice/crio-1a4447263174f23806398e24772f60529b0b177529135a8d82ca21a52377c551 WatchSource:0}: Error finding container 1a4447263174f23806398e24772f60529b0b177529135a8d82ca21a52377c551: Status 404 returned error can't find the container with id 1a4447263174f23806398e24772f60529b0b177529135a8d82ca21a52377c551 Jul 06 22:30:00.607499 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:30:00.607475 2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 22:30:01.291682 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:30:01.291639 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29722950-79m5s" event={"ID":"1a705704-330f-498f-b7a3-696be4ace5a6","Type":"ContainerStarted","Data":"1a4447263174f23806398e24772f60529b0b177529135a8d82ca21a52377c551"} Jul 06 22:30:03.303380 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:30:03.303284 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29722950-79m5s" event={"ID":"1a705704-330f-498f-b7a3-696be4ace5a6","Type":"ContainerStarted","Data":"20a2ba901458e53f9531a3cb9662f1809b9f2feb162140fb3b75e0893679a5b3"} Jul 06 22:30:03.322117 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:30:03.322035 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-key-cleanup-29722950-79m5s" podStartSLOduration=1.136197902 podStartE2EDuration="3.322019418s" totalImagesPullingTime="2.185821516s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:30:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:30:03.320783086 +0000 UTC m=+1282.879578407" watchObservedRunningTime="2026-07-06 22:30:03.322019418 +0000 UTC m=+1282.880814742" Jul 06 22:30:04.309983 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:30:04.309947 2581 generic.go:356] "Generic (PLEG): container finished" podID="1a705704-330f-498f-b7a3-696be4ace5a6" containerID="20a2ba901458e53f9531a3cb9662f1809b9f2feb162140fb3b75e0893679a5b3" exitCode=0 Jul 06 22:30:04.310411 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:30:04.310033 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29722950-79m5s" event={"ID":"1a705704-330f-498f-b7a3-696be4ace5a6","Type":"ContainerDied","Data":"20a2ba901458e53f9531a3cb9662f1809b9f2feb162140fb3b75e0893679a5b3"} Jul 06 22:30:05.457452 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:30:05.457423 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29722950-79m5s" Jul 06 22:30:05.615989 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:30:05.615900 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-srw5x\" (UniqueName: \"kubernetes.io/projected/1a705704-330f-498f-b7a3-696be4ace5a6-kube-api-access-srw5x\") pod \"1a705704-330f-498f-b7a3-696be4ace5a6\" (UID: \"1a705704-330f-498f-b7a3-696be4ace5a6\") " Jul 06 22:30:05.618149 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:30:05.618121 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1a705704-330f-498f-b7a3-696be4ace5a6-kube-api-access-srw5x" (OuterVolumeSpecName: "kube-api-access-srw5x") pod "1a705704-330f-498f-b7a3-696be4ace5a6" (UID: "1a705704-330f-498f-b7a3-696be4ace5a6"). InnerVolumeSpecName "kube-api-access-srw5x". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 22:30:05.717118 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:30:05.717044 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-srw5x\" (UniqueName: \"kubernetes.io/projected/1a705704-330f-498f-b7a3-696be4ace5a6-kube-api-access-srw5x\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:30:06.322542 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:30:06.322510 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29722950-79m5s" Jul 06 22:30:06.322721 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:30:06.322513 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29722950-79m5s" event={"ID":"1a705704-330f-498f-b7a3-696be4ace5a6","Type":"ContainerDied","Data":"1a4447263174f23806398e24772f60529b0b177529135a8d82ca21a52377c551"} Jul 06 22:30:06.322721 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:30:06.322638 2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1a4447263174f23806398e24772f60529b0b177529135a8d82ca21a52377c551" Jul 06 22:30:09.208866 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:30:09.208826 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:33:40.990148 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:33:40.990121 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-rcn5q_86b07166-2f34-443b-9c61-86b7d950ad7b/console-operator/1.log" Jul 06 22:33:40.995153 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:33:40.995130 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-rcn5q_86b07166-2f34-443b-9c61-86b7d950ad7b/console-operator/1.log" Jul 06 22:33:59.796147 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:33:59.796106 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:34:06.600967 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:06.600878 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:34:14.871695 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:14.871661 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:34:20.074345 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:20.074309 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:34:22.948907 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:22.948871 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:34:23.056813 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:23.056780 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:34:23.457258 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:23.457227 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:34:29.206136 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:29.206095 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:34:29.306493 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:29.306443 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:34:29.395869 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:29.395839 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-697d6823-openshift-default-6b6f55d7c-88jxd"] Jul 06 22:34:29.396304 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:29.396288 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1a705704-330f-498f-b7a3-696be4ace5a6" containerName="cleanup" Jul 06 22:34:29.396390 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:29.396307 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="1a705704-330f-498f-b7a3-696be4ace5a6" containerName="cleanup" Jul 06 22:34:29.396444 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:29.396422 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="1a705704-330f-498f-b7a3-696be4ace5a6" containerName="cleanup" Jul 06 22:34:29.399093 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:29.399043 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-697d6823-openshift-default-6b6f55d7c-88jxd" Jul 06 22:34:29.401596 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:29.401577 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-697d6823-openshift-default-dockercfg-gq7r6\"" Jul 06 22:34:29.421671 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:29.421641 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-697d6823-openshift-default-6b6f55d7c-88jxd"] Jul 06 22:34:29.430028 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:29.429986 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:34:29.449531 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:29.449502 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4ff115a1-1b61-46a9-96d2-38fc5504e42a-istio-envoy\") pod \"e2e-ait-697d6823-openshift-default-6b6f55d7c-88jxd\" (UID: \"4ff115a1-1b61-46a9-96d2-38fc5504e42a\") " pod="openshift-ingress/e2e-ait-697d6823-openshift-default-6b6f55d7c-88jxd" Jul 06 22:34:29.449661 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:29.449541 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4ff115a1-1b61-46a9-96d2-38fc5504e42a-credential-socket\") pod \"e2e-ait-697d6823-openshift-default-6b6f55d7c-88jxd\" (UID: \"4ff115a1-1b61-46a9-96d2-38fc5504e42a\") " pod="openshift-ingress/e2e-ait-697d6823-openshift-default-6b6f55d7c-88jxd" Jul 06 22:34:29.449661 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:29.449560 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sms2h\" (UniqueName: \"kubernetes.io/projected/4ff115a1-1b61-46a9-96d2-38fc5504e42a-kube-api-access-sms2h\") pod \"e2e-ait-697d6823-openshift-default-6b6f55d7c-88jxd\" (UID: \"4ff115a1-1b61-46a9-96d2-38fc5504e42a\") " pod="openshift-ingress/e2e-ait-697d6823-openshift-default-6b6f55d7c-88jxd" Jul 06 22:34:29.449661 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:29.449624 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4ff115a1-1b61-46a9-96d2-38fc5504e42a-workload-socket\") pod \"e2e-ait-697d6823-openshift-default-6b6f55d7c-88jxd\" (UID: \"4ff115a1-1b61-46a9-96d2-38fc5504e42a\") " pod="openshift-ingress/e2e-ait-697d6823-openshift-default-6b6f55d7c-88jxd" Jul 06 22:34:29.449661 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:29.449639 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4ff115a1-1b61-46a9-96d2-38fc5504e42a-istio-data\") pod \"e2e-ait-697d6823-openshift-default-6b6f55d7c-88jxd\" (UID: \"4ff115a1-1b61-46a9-96d2-38fc5504e42a\") " pod="openshift-ingress/e2e-ait-697d6823-openshift-default-6b6f55d7c-88jxd" Jul 06 22:34:29.449661 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:29.449652 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4ff115a1-1b61-46a9-96d2-38fc5504e42a-istio-token\") pod \"e2e-ait-697d6823-openshift-default-6b6f55d7c-88jxd\" (UID: \"4ff115a1-1b61-46a9-96d2-38fc5504e42a\") " pod="openshift-ingress/e2e-ait-697d6823-openshift-default-6b6f55d7c-88jxd" Jul 06 22:34:29.449822 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:29.449732 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4ff115a1-1b61-46a9-96d2-38fc5504e42a-workload-certs\") pod \"e2e-ait-697d6823-openshift-default-6b6f55d7c-88jxd\" (UID: \"4ff115a1-1b61-46a9-96d2-38fc5504e42a\") " pod="openshift-ingress/e2e-ait-697d6823-openshift-default-6b6f55d7c-88jxd" Jul 06 22:34:29.449822 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:29.449758 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4ff115a1-1b61-46a9-96d2-38fc5504e42a-istio-podinfo\") pod \"e2e-ait-697d6823-openshift-default-6b6f55d7c-88jxd\" (UID: \"4ff115a1-1b61-46a9-96d2-38fc5504e42a\") " pod="openshift-ingress/e2e-ait-697d6823-openshift-default-6b6f55d7c-88jxd" Jul 06 22:34:29.449822 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:29.449776 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4ff115a1-1b61-46a9-96d2-38fc5504e42a-istiod-ca-cert\") pod \"e2e-ait-697d6823-openshift-default-6b6f55d7c-88jxd\" (UID: \"4ff115a1-1b61-46a9-96d2-38fc5504e42a\") " pod="openshift-ingress/e2e-ait-697d6823-openshift-default-6b6f55d7c-88jxd" Jul 06 22:34:29.490849 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:29.490729 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:34:29.551008 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:29.550966 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4ff115a1-1b61-46a9-96d2-38fc5504e42a-workload-certs\") pod \"e2e-ait-697d6823-openshift-default-6b6f55d7c-88jxd\" (UID: \"4ff115a1-1b61-46a9-96d2-38fc5504e42a\") " pod="openshift-ingress/e2e-ait-697d6823-openshift-default-6b6f55d7c-88jxd" Jul 06 22:34:29.551008 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:29.551012 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4ff115a1-1b61-46a9-96d2-38fc5504e42a-istio-podinfo\") pod \"e2e-ait-697d6823-openshift-default-6b6f55d7c-88jxd\" (UID: \"4ff115a1-1b61-46a9-96d2-38fc5504e42a\") " pod="openshift-ingress/e2e-ait-697d6823-openshift-default-6b6f55d7c-88jxd" Jul 06 22:34:29.551313 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:29.551033 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4ff115a1-1b61-46a9-96d2-38fc5504e42a-istiod-ca-cert\") pod \"e2e-ait-697d6823-openshift-default-6b6f55d7c-88jxd\" (UID: \"4ff115a1-1b61-46a9-96d2-38fc5504e42a\") " pod="openshift-ingress/e2e-ait-697d6823-openshift-default-6b6f55d7c-88jxd" Jul 06 22:34:29.551313 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:29.551126 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4ff115a1-1b61-46a9-96d2-38fc5504e42a-istio-envoy\") pod \"e2e-ait-697d6823-openshift-default-6b6f55d7c-88jxd\" (UID: \"4ff115a1-1b61-46a9-96d2-38fc5504e42a\") " pod="openshift-ingress/e2e-ait-697d6823-openshift-default-6b6f55d7c-88jxd" Jul 06 22:34:29.551313 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:29.551154 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4ff115a1-1b61-46a9-96d2-38fc5504e42a-credential-socket\") pod \"e2e-ait-697d6823-openshift-default-6b6f55d7c-88jxd\" (UID: \"4ff115a1-1b61-46a9-96d2-38fc5504e42a\") " pod="openshift-ingress/e2e-ait-697d6823-openshift-default-6b6f55d7c-88jxd" Jul 06 22:34:29.551313 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:29.551170 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sms2h\" (UniqueName: \"kubernetes.io/projected/4ff115a1-1b61-46a9-96d2-38fc5504e42a-kube-api-access-sms2h\") pod \"e2e-ait-697d6823-openshift-default-6b6f55d7c-88jxd\" (UID: \"4ff115a1-1b61-46a9-96d2-38fc5504e42a\") " pod="openshift-ingress/e2e-ait-697d6823-openshift-default-6b6f55d7c-88jxd" Jul 06 22:34:29.551313 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:29.551232 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4ff115a1-1b61-46a9-96d2-38fc5504e42a-workload-socket\") pod \"e2e-ait-697d6823-openshift-default-6b6f55d7c-88jxd\" (UID: \"4ff115a1-1b61-46a9-96d2-38fc5504e42a\") " pod="openshift-ingress/e2e-ait-697d6823-openshift-default-6b6f55d7c-88jxd" Jul 06 22:34:29.551313 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:29.551258 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4ff115a1-1b61-46a9-96d2-38fc5504e42a-istio-data\") pod \"e2e-ait-697d6823-openshift-default-6b6f55d7c-88jxd\" (UID: \"4ff115a1-1b61-46a9-96d2-38fc5504e42a\") " pod="openshift-ingress/e2e-ait-697d6823-openshift-default-6b6f55d7c-88jxd" Jul 06 22:34:29.551313 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:29.551282 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4ff115a1-1b61-46a9-96d2-38fc5504e42a-istio-token\") pod \"e2e-ait-697d6823-openshift-default-6b6f55d7c-88jxd\" (UID: \"4ff115a1-1b61-46a9-96d2-38fc5504e42a\") " pod="openshift-ingress/e2e-ait-697d6823-openshift-default-6b6f55d7c-88jxd" Jul 06 22:34:29.551708 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:29.551486 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4ff115a1-1b61-46a9-96d2-38fc5504e42a-workload-certs\") pod \"e2e-ait-697d6823-openshift-default-6b6f55d7c-88jxd\" (UID: \"4ff115a1-1b61-46a9-96d2-38fc5504e42a\") " pod="openshift-ingress/e2e-ait-697d6823-openshift-default-6b6f55d7c-88jxd" Jul 06 22:34:29.551708 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:29.551518 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4ff115a1-1b61-46a9-96d2-38fc5504e42a-credential-socket\") pod \"e2e-ait-697d6823-openshift-default-6b6f55d7c-88jxd\" (UID: \"4ff115a1-1b61-46a9-96d2-38fc5504e42a\") " pod="openshift-ingress/e2e-ait-697d6823-openshift-default-6b6f55d7c-88jxd" Jul 06 22:34:29.551708 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:29.551690 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4ff115a1-1b61-46a9-96d2-38fc5504e42a-workload-socket\") pod \"e2e-ait-697d6823-openshift-default-6b6f55d7c-88jxd\" (UID: \"4ff115a1-1b61-46a9-96d2-38fc5504e42a\") " pod="openshift-ingress/e2e-ait-697d6823-openshift-default-6b6f55d7c-88jxd" Jul 06 22:34:29.551857 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:29.551792 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4ff115a1-1b61-46a9-96d2-38fc5504e42a-istio-data\") pod \"e2e-ait-697d6823-openshift-default-6b6f55d7c-88jxd\" (UID: \"4ff115a1-1b61-46a9-96d2-38fc5504e42a\") " pod="openshift-ingress/e2e-ait-697d6823-openshift-default-6b6f55d7c-88jxd" Jul 06 22:34:29.552130 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:29.552115 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4ff115a1-1b61-46a9-96d2-38fc5504e42a-istiod-ca-cert\") pod \"e2e-ait-697d6823-openshift-default-6b6f55d7c-88jxd\" (UID: \"4ff115a1-1b61-46a9-96d2-38fc5504e42a\") " pod="openshift-ingress/e2e-ait-697d6823-openshift-default-6b6f55d7c-88jxd" Jul 06 22:34:29.553598 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:29.553576 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4ff115a1-1b61-46a9-96d2-38fc5504e42a-istio-podinfo\") pod \"e2e-ait-697d6823-openshift-default-6b6f55d7c-88jxd\" (UID: \"4ff115a1-1b61-46a9-96d2-38fc5504e42a\") " pod="openshift-ingress/e2e-ait-697d6823-openshift-default-6b6f55d7c-88jxd" Jul 06 22:34:29.553743 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:29.553727 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4ff115a1-1b61-46a9-96d2-38fc5504e42a-istio-envoy\") pod \"e2e-ait-697d6823-openshift-default-6b6f55d7c-88jxd\" (UID: \"4ff115a1-1b61-46a9-96d2-38fc5504e42a\") " pod="openshift-ingress/e2e-ait-697d6823-openshift-default-6b6f55d7c-88jxd" Jul 06 22:34:29.559162 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:29.559135 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4ff115a1-1b61-46a9-96d2-38fc5504e42a-istio-token\") pod \"e2e-ait-697d6823-openshift-default-6b6f55d7c-88jxd\" (UID: \"4ff115a1-1b61-46a9-96d2-38fc5504e42a\") " pod="openshift-ingress/e2e-ait-697d6823-openshift-default-6b6f55d7c-88jxd" Jul 06 22:34:29.559293 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:29.559247 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sms2h\" (UniqueName: \"kubernetes.io/projected/4ff115a1-1b61-46a9-96d2-38fc5504e42a-kube-api-access-sms2h\") pod \"e2e-ait-697d6823-openshift-default-6b6f55d7c-88jxd\" (UID: \"4ff115a1-1b61-46a9-96d2-38fc5504e42a\") " pod="openshift-ingress/e2e-ait-697d6823-openshift-default-6b6f55d7c-88jxd" Jul 06 22:34:29.709802 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:29.709757 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-697d6823-openshift-default-6b6f55d7c-88jxd" Jul 06 22:34:29.847042 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:29.847018 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-697d6823-openshift-default-6b6f55d7c-88jxd"] Jul 06 22:34:29.849362 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:34:29.849337 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ff115a1_1b61_46a9_96d2_38fc5504e42a.slice/crio-7b0544b58f14781dadc939a6fa3e76e5ab667e41be16c59b55ea56ca0accf8c3 WatchSource:0}: Error finding container 7b0544b58f14781dadc939a6fa3e76e5ab667e41be16c59b55ea56ca0accf8c3: Status 404 returned error can't find the container with id 7b0544b58f14781dadc939a6fa3e76e5ab667e41be16c59b55ea56ca0accf8c3 Jul 06 22:34:29.851595 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:29.851562 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 22:34:29.851677 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:29.851622 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 22:34:29.851677 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:29.851656 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 22:34:30.606060 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:30.605973 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:34:30.675157 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:30.675116 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-697d6823-openshift-default-6b6f55d7c-88jxd" event={"ID":"4ff115a1-1b61-46a9-96d2-38fc5504e42a","Type":"ContainerStarted","Data":"32c8d55f99c2cc92e30ff91f579c28c98e13dbbe83621cbc4326dbb134a95932"} Jul 06 22:34:30.675322 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:30.675173 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-697d6823-openshift-default-6b6f55d7c-88jxd" event={"ID":"4ff115a1-1b61-46a9-96d2-38fc5504e42a","Type":"ContainerStarted","Data":"7b0544b58f14781dadc939a6fa3e76e5ab667e41be16c59b55ea56ca0accf8c3"} Jul 06 22:34:30.695514 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:30.695486 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:34:30.709953 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:30.709926 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-697d6823-openshift-default-6b6f55d7c-88jxd" Jul 06 22:34:30.714581 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:30.714557 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-697d6823-openshift-default-6b6f55d7c-88jxd" Jul 06 22:34:30.718927 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:30.718864 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-697d6823-openshift-default-6b6f55d7c-88jxd" podStartSLOduration=1.718845574 podStartE2EDuration="1.718845574s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:34:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:34:30.716263545 +0000 UTC m=+1550.275058866" watchObservedRunningTime="2026-07-06 22:34:30.718845574 +0000 UTC m=+1550.277640899" Jul 06 22:34:30.760829 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:30.760806 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:34:30.987563 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:30.987529 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:34:31.256392 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:31.256287 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:34:31.357275 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:31.357220 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:34:31.679929 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:31.679890 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-697d6823-openshift-default-6b6f55d7c-88jxd" Jul 06 22:34:31.680777 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:31.680754 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-697d6823-openshift-default-6b6f55d7c-88jxd" Jul 06 22:34:32.681435 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:32.681402 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:34:35.666611 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:35.666572 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:34:35.829730 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:35.829696 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-697d6823-openshift-default-6b6f55d7c-88jxd"] Jul 06 22:34:35.829900 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:35.829876 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-697d6823-openshift-default-6b6f55d7c-88jxd" podUID="4ff115a1-1b61-46a9-96d2-38fc5504e42a" containerName="istio-proxy" containerID="cri-o://32c8d55f99c2cc92e30ff91f579c28c98e13dbbe83621cbc4326dbb134a95932" gracePeriod=30 Jul 06 22:34:35.860858 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:35.860712 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:34:35.964792 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:35.964705 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:34:36.070104 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:36.070031 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:34:39.202083 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:39.202005 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:34:40.991461 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:40.991437 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-697d6823-openshift-default-6b6f55d7c-88jxd" Jul 06 22:34:41.058238 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:41.058207 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4ff115a1-1b61-46a9-96d2-38fc5504e42a-istio-token\") pod \"4ff115a1-1b61-46a9-96d2-38fc5504e42a\" (UID: \"4ff115a1-1b61-46a9-96d2-38fc5504e42a\") " Jul 06 22:34:41.058238 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:41.058244 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4ff115a1-1b61-46a9-96d2-38fc5504e42a-istio-podinfo\") pod \"4ff115a1-1b61-46a9-96d2-38fc5504e42a\" (UID: \"4ff115a1-1b61-46a9-96d2-38fc5504e42a\") " Jul 06 22:34:41.058425 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:41.058267 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4ff115a1-1b61-46a9-96d2-38fc5504e42a-credential-socket\") pod \"4ff115a1-1b61-46a9-96d2-38fc5504e42a\" (UID: \"4ff115a1-1b61-46a9-96d2-38fc5504e42a\") " Jul 06 22:34:41.058425 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:41.058329 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4ff115a1-1b61-46a9-96d2-38fc5504e42a-workload-certs\") pod \"4ff115a1-1b61-46a9-96d2-38fc5504e42a\" (UID: \"4ff115a1-1b61-46a9-96d2-38fc5504e42a\") " Jul 06 22:34:41.058425 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:41.058381 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4ff115a1-1b61-46a9-96d2-38fc5504e42a-istio-data\") pod \"4ff115a1-1b61-46a9-96d2-38fc5504e42a\" (UID: \"4ff115a1-1b61-46a9-96d2-38fc5504e42a\") " Jul 06 22:34:41.058425 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:41.058402 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sms2h\" (UniqueName: \"kubernetes.io/projected/4ff115a1-1b61-46a9-96d2-38fc5504e42a-kube-api-access-sms2h\") pod \"4ff115a1-1b61-46a9-96d2-38fc5504e42a\" (UID: \"4ff115a1-1b61-46a9-96d2-38fc5504e42a\") " Jul 06 22:34:41.058425 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:41.058420 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4ff115a1-1b61-46a9-96d2-38fc5504e42a-workload-socket\") pod \"4ff115a1-1b61-46a9-96d2-38fc5504e42a\" (UID: \"4ff115a1-1b61-46a9-96d2-38fc5504e42a\") " Jul 06 22:34:41.058674 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:41.058444 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4ff115a1-1b61-46a9-96d2-38fc5504e42a-istio-envoy\") pod \"4ff115a1-1b61-46a9-96d2-38fc5504e42a\" (UID: \"4ff115a1-1b61-46a9-96d2-38fc5504e42a\") " Jul 06 22:34:41.058674 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:41.058464 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4ff115a1-1b61-46a9-96d2-38fc5504e42a-istiod-ca-cert\") pod \"4ff115a1-1b61-46a9-96d2-38fc5504e42a\" (UID: \"4ff115a1-1b61-46a9-96d2-38fc5504e42a\") " Jul 06 22:34:41.058674 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:41.058629 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4ff115a1-1b61-46a9-96d2-38fc5504e42a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "4ff115a1-1b61-46a9-96d2-38fc5504e42a" (UID: "4ff115a1-1b61-46a9-96d2-38fc5504e42a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:34:41.058835 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:41.058727 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4ff115a1-1b61-46a9-96d2-38fc5504e42a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "4ff115a1-1b61-46a9-96d2-38fc5504e42a" (UID: "4ff115a1-1b61-46a9-96d2-38fc5504e42a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:34:41.058933 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:41.058913 2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4ff115a1-1b61-46a9-96d2-38fc5504e42a-workload-socket\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:34:41.058998 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:41.058937 2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4ff115a1-1b61-46a9-96d2-38fc5504e42a-credential-socket\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:34:41.058998 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:41.058972 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4ff115a1-1b61-46a9-96d2-38fc5504e42a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "4ff115a1-1b61-46a9-96d2-38fc5504e42a" (UID: "4ff115a1-1b61-46a9-96d2-38fc5504e42a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:34:41.059266 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:41.059243 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4ff115a1-1b61-46a9-96d2-38fc5504e42a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "4ff115a1-1b61-46a9-96d2-38fc5504e42a" (UID: "4ff115a1-1b61-46a9-96d2-38fc5504e42a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 22:34:41.059266 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:41.059253 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4ff115a1-1b61-46a9-96d2-38fc5504e42a-istio-data" (OuterVolumeSpecName: "istio-data") pod "4ff115a1-1b61-46a9-96d2-38fc5504e42a" (UID: "4ff115a1-1b61-46a9-96d2-38fc5504e42a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:34:41.060537 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:41.060508 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4ff115a1-1b61-46a9-96d2-38fc5504e42a-istio-token" (OuterVolumeSpecName: "istio-token") pod "4ff115a1-1b61-46a9-96d2-38fc5504e42a" (UID: "4ff115a1-1b61-46a9-96d2-38fc5504e42a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 22:34:41.060537 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:41.060508 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4ff115a1-1b61-46a9-96d2-38fc5504e42a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "4ff115a1-1b61-46a9-96d2-38fc5504e42a" (UID: "4ff115a1-1b61-46a9-96d2-38fc5504e42a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 22:34:41.060835 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:41.060813 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4ff115a1-1b61-46a9-96d2-38fc5504e42a-kube-api-access-sms2h" (OuterVolumeSpecName: "kube-api-access-sms2h") pod "4ff115a1-1b61-46a9-96d2-38fc5504e42a" (UID: "4ff115a1-1b61-46a9-96d2-38fc5504e42a"). InnerVolumeSpecName "kube-api-access-sms2h". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 22:34:41.060940 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:41.060919 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4ff115a1-1b61-46a9-96d2-38fc5504e42a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "4ff115a1-1b61-46a9-96d2-38fc5504e42a" (UID: "4ff115a1-1b61-46a9-96d2-38fc5504e42a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:34:41.160035 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:41.160008 2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4ff115a1-1b61-46a9-96d2-38fc5504e42a-istio-data\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:34:41.160035 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:41.160032 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sms2h\" (UniqueName: \"kubernetes.io/projected/4ff115a1-1b61-46a9-96d2-38fc5504e42a-kube-api-access-sms2h\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:34:41.160035 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:41.160041 2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4ff115a1-1b61-46a9-96d2-38fc5504e42a-istio-envoy\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:34:41.160483 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:41.160050 2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4ff115a1-1b61-46a9-96d2-38fc5504e42a-istiod-ca-cert\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:34:41.160483 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:41.160058 2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4ff115a1-1b61-46a9-96d2-38fc5504e42a-istio-token\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:34:41.160483 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:41.160088 2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4ff115a1-1b61-46a9-96d2-38fc5504e42a-istio-podinfo\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:34:41.160483 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:41.160095 2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4ff115a1-1b61-46a9-96d2-38fc5504e42a-workload-certs\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:34:41.670207 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:41.670172 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:34:41.743242 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:41.743201 2581 generic.go:356] "Generic (PLEG): container finished" podID="4ff115a1-1b61-46a9-96d2-38fc5504e42a" containerID="32c8d55f99c2cc92e30ff91f579c28c98e13dbbe83621cbc4326dbb134a95932" exitCode=0 Jul 06 22:34:41.743242 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:41.743237 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-697d6823-openshift-default-6b6f55d7c-88jxd" Jul 06 22:34:41.743497 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:41.743279 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-697d6823-openshift-default-6b6f55d7c-88jxd" event={"ID":"4ff115a1-1b61-46a9-96d2-38fc5504e42a","Type":"ContainerDied","Data":"32c8d55f99c2cc92e30ff91f579c28c98e13dbbe83621cbc4326dbb134a95932"} Jul 06 22:34:41.743497 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:41.743333 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-697d6823-openshift-default-6b6f55d7c-88jxd" event={"ID":"4ff115a1-1b61-46a9-96d2-38fc5504e42a","Type":"ContainerDied","Data":"7b0544b58f14781dadc939a6fa3e76e5ab667e41be16c59b55ea56ca0accf8c3"} Jul 06 22:34:41.743497 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:41.743354 2581 scope.go:117] "RemoveContainer" containerID="32c8d55f99c2cc92e30ff91f579c28c98e13dbbe83621cbc4326dbb134a95932" Jul 06 22:34:41.752969 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:41.752952 2581 scope.go:117] "RemoveContainer" containerID="32c8d55f99c2cc92e30ff91f579c28c98e13dbbe83621cbc4326dbb134a95932" Jul 06 22:34:41.753271 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:34:41.753251 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"32c8d55f99c2cc92e30ff91f579c28c98e13dbbe83621cbc4326dbb134a95932\": container with ID starting with 32c8d55f99c2cc92e30ff91f579c28c98e13dbbe83621cbc4326dbb134a95932 not found: ID does not exist" containerID="32c8d55f99c2cc92e30ff91f579c28c98e13dbbe83621cbc4326dbb134a95932" Jul 06 22:34:41.753334 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:41.753280 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"32c8d55f99c2cc92e30ff91f579c28c98e13dbbe83621cbc4326dbb134a95932"} err="failed to get container status \"32c8d55f99c2cc92e30ff91f579c28c98e13dbbe83621cbc4326dbb134a95932\": rpc error: code = NotFound desc = could not find container \"32c8d55f99c2cc92e30ff91f579c28c98e13dbbe83621cbc4326dbb134a95932\": container with ID starting with 32c8d55f99c2cc92e30ff91f579c28c98e13dbbe83621cbc4326dbb134a95932 not found: ID does not exist" Jul 06 22:34:41.765400 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:41.765378 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-697d6823-openshift-default-6b6f55d7c-88jxd"] Jul 06 22:34:41.768200 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:41.768179 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:34:41.769413 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:41.769396 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-697d6823-openshift-default-6b6f55d7c-88jxd"] Jul 06 22:34:41.852631 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:41.852599 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-754cf73a-openshift-default-6b59879d8c-frkmc"] Jul 06 22:34:41.853008 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:41.852995 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4ff115a1-1b61-46a9-96d2-38fc5504e42a" containerName="istio-proxy" Jul 06 22:34:41.853056 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:41.853010 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="4ff115a1-1b61-46a9-96d2-38fc5504e42a" containerName="istio-proxy" Jul 06 22:34:41.853116 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:41.853085 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="4ff115a1-1b61-46a9-96d2-38fc5504e42a" containerName="istio-proxy" Jul 06 22:34:41.857702 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:41.857680 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-754cf73a-openshift-default-6b59879d8c-frkmc" Jul 06 22:34:41.860286 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:41.860262 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-754cf73a-openshift-default-dockercfg-rjqb8\"" Jul 06 22:34:41.870539 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:41.870512 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-754cf73a-openshift-default-6b59879d8c-frkmc"] Jul 06 22:34:41.893288 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:41.893256 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:34:41.966921 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:41.966890 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c840a66b-bbbb-4b56-98ae-f8ab6da6ecc8-workload-socket\") pod \"e2e-ait-754cf73a-openshift-default-6b59879d8c-frkmc\" (UID: \"c840a66b-bbbb-4b56-98ae-f8ab6da6ecc8\") " pod="openshift-ingress/e2e-ait-754cf73a-openshift-default-6b59879d8c-frkmc" Jul 06 22:34:41.967123 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:41.966926 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c840a66b-bbbb-4b56-98ae-f8ab6da6ecc8-istio-data\") pod \"e2e-ait-754cf73a-openshift-default-6b59879d8c-frkmc\" (UID: \"c840a66b-bbbb-4b56-98ae-f8ab6da6ecc8\") " pod="openshift-ingress/e2e-ait-754cf73a-openshift-default-6b59879d8c-frkmc" Jul 06 22:34:41.967123 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:41.966960 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c840a66b-bbbb-4b56-98ae-f8ab6da6ecc8-istio-podinfo\") pod \"e2e-ait-754cf73a-openshift-default-6b59879d8c-frkmc\" (UID: \"c840a66b-bbbb-4b56-98ae-f8ab6da6ecc8\") " pod="openshift-ingress/e2e-ait-754cf73a-openshift-default-6b59879d8c-frkmc" Jul 06 22:34:41.967123 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:41.966980 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c840a66b-bbbb-4b56-98ae-f8ab6da6ecc8-istiod-ca-cert\") pod \"e2e-ait-754cf73a-openshift-default-6b59879d8c-frkmc\" (UID: \"c840a66b-bbbb-4b56-98ae-f8ab6da6ecc8\") " pod="openshift-ingress/e2e-ait-754cf73a-openshift-default-6b59879d8c-frkmc" Jul 06 22:34:41.967123 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:41.967039 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c840a66b-bbbb-4b56-98ae-f8ab6da6ecc8-workload-certs\") pod \"e2e-ait-754cf73a-openshift-default-6b59879d8c-frkmc\" (UID: \"c840a66b-bbbb-4b56-98ae-f8ab6da6ecc8\") " pod="openshift-ingress/e2e-ait-754cf73a-openshift-default-6b59879d8c-frkmc" Jul 06 22:34:41.967123 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:41.967118 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p49w8\" (UniqueName: \"kubernetes.io/projected/c840a66b-bbbb-4b56-98ae-f8ab6da6ecc8-kube-api-access-p49w8\") pod \"e2e-ait-754cf73a-openshift-default-6b59879d8c-frkmc\" (UID: \"c840a66b-bbbb-4b56-98ae-f8ab6da6ecc8\") " pod="openshift-ingress/e2e-ait-754cf73a-openshift-default-6b59879d8c-frkmc" Jul 06 22:34:41.967386 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:41.967145 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c840a66b-bbbb-4b56-98ae-f8ab6da6ecc8-istio-token\") pod \"e2e-ait-754cf73a-openshift-default-6b59879d8c-frkmc\" (UID: \"c840a66b-bbbb-4b56-98ae-f8ab6da6ecc8\") " pod="openshift-ingress/e2e-ait-754cf73a-openshift-default-6b59879d8c-frkmc" Jul 06 22:34:41.967386 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:41.967261 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c840a66b-bbbb-4b56-98ae-f8ab6da6ecc8-credential-socket\") pod \"e2e-ait-754cf73a-openshift-default-6b59879d8c-frkmc\" (UID: \"c840a66b-bbbb-4b56-98ae-f8ab6da6ecc8\") " pod="openshift-ingress/e2e-ait-754cf73a-openshift-default-6b59879d8c-frkmc" Jul 06 22:34:41.967386 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:41.967297 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c840a66b-bbbb-4b56-98ae-f8ab6da6ecc8-istio-envoy\") pod \"e2e-ait-754cf73a-openshift-default-6b59879d8c-frkmc\" (UID: \"c840a66b-bbbb-4b56-98ae-f8ab6da6ecc8\") " pod="openshift-ingress/e2e-ait-754cf73a-openshift-default-6b59879d8c-frkmc" Jul 06 22:34:41.993344 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:41.993316 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:34:42.068654 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:42.068618 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p49w8\" (UniqueName: \"kubernetes.io/projected/c840a66b-bbbb-4b56-98ae-f8ab6da6ecc8-kube-api-access-p49w8\") pod \"e2e-ait-754cf73a-openshift-default-6b59879d8c-frkmc\" (UID: \"c840a66b-bbbb-4b56-98ae-f8ab6da6ecc8\") " pod="openshift-ingress/e2e-ait-754cf73a-openshift-default-6b59879d8c-frkmc" Jul 06 22:34:42.068654 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:42.068658 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c840a66b-bbbb-4b56-98ae-f8ab6da6ecc8-istio-token\") pod \"e2e-ait-754cf73a-openshift-default-6b59879d8c-frkmc\" (UID: \"c840a66b-bbbb-4b56-98ae-f8ab6da6ecc8\") " pod="openshift-ingress/e2e-ait-754cf73a-openshift-default-6b59879d8c-frkmc" Jul 06 22:34:42.068896 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:42.068725 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c840a66b-bbbb-4b56-98ae-f8ab6da6ecc8-credential-socket\") pod \"e2e-ait-754cf73a-openshift-default-6b59879d8c-frkmc\" (UID: \"c840a66b-bbbb-4b56-98ae-f8ab6da6ecc8\") " pod="openshift-ingress/e2e-ait-754cf73a-openshift-default-6b59879d8c-frkmc" Jul 06 22:34:42.068896 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:42.068753 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c840a66b-bbbb-4b56-98ae-f8ab6da6ecc8-istio-envoy\") pod \"e2e-ait-754cf73a-openshift-default-6b59879d8c-frkmc\" (UID: \"c840a66b-bbbb-4b56-98ae-f8ab6da6ecc8\") " pod="openshift-ingress/e2e-ait-754cf73a-openshift-default-6b59879d8c-frkmc" Jul 06 22:34:42.068896 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:42.068789 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c840a66b-bbbb-4b56-98ae-f8ab6da6ecc8-workload-socket\") pod \"e2e-ait-754cf73a-openshift-default-6b59879d8c-frkmc\" (UID: \"c840a66b-bbbb-4b56-98ae-f8ab6da6ecc8\") " pod="openshift-ingress/e2e-ait-754cf73a-openshift-default-6b59879d8c-frkmc" Jul 06 22:34:42.068896 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:42.068811 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c840a66b-bbbb-4b56-98ae-f8ab6da6ecc8-istio-data\") pod \"e2e-ait-754cf73a-openshift-default-6b59879d8c-frkmc\" (UID: \"c840a66b-bbbb-4b56-98ae-f8ab6da6ecc8\") " pod="openshift-ingress/e2e-ait-754cf73a-openshift-default-6b59879d8c-frkmc" Jul 06 22:34:42.068896 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:42.068846 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c840a66b-bbbb-4b56-98ae-f8ab6da6ecc8-istio-podinfo\") pod \"e2e-ait-754cf73a-openshift-default-6b59879d8c-frkmc\" (UID: \"c840a66b-bbbb-4b56-98ae-f8ab6da6ecc8\") " pod="openshift-ingress/e2e-ait-754cf73a-openshift-default-6b59879d8c-frkmc" Jul 06 22:34:42.068896 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:42.068872 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c840a66b-bbbb-4b56-98ae-f8ab6da6ecc8-istiod-ca-cert\") pod \"e2e-ait-754cf73a-openshift-default-6b59879d8c-frkmc\" (UID: \"c840a66b-bbbb-4b56-98ae-f8ab6da6ecc8\") " pod="openshift-ingress/e2e-ait-754cf73a-openshift-default-6b59879d8c-frkmc" Jul 06 22:34:42.069241 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:42.068898 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c840a66b-bbbb-4b56-98ae-f8ab6da6ecc8-workload-certs\") pod \"e2e-ait-754cf73a-openshift-default-6b59879d8c-frkmc\" (UID: \"c840a66b-bbbb-4b56-98ae-f8ab6da6ecc8\") " pod="openshift-ingress/e2e-ait-754cf73a-openshift-default-6b59879d8c-frkmc" Jul 06 22:34:42.069300 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:42.069234 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c840a66b-bbbb-4b56-98ae-f8ab6da6ecc8-istio-data\") pod \"e2e-ait-754cf73a-openshift-default-6b59879d8c-frkmc\" (UID: \"c840a66b-bbbb-4b56-98ae-f8ab6da6ecc8\") " pod="openshift-ingress/e2e-ait-754cf73a-openshift-default-6b59879d8c-frkmc" Jul 06 22:34:42.069300 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:42.069269 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c840a66b-bbbb-4b56-98ae-f8ab6da6ecc8-workload-socket\") pod \"e2e-ait-754cf73a-openshift-default-6b59879d8c-frkmc\" (UID: \"c840a66b-bbbb-4b56-98ae-f8ab6da6ecc8\") " pod="openshift-ingress/e2e-ait-754cf73a-openshift-default-6b59879d8c-frkmc" Jul 06 22:34:42.069401 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:42.069368 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c840a66b-bbbb-4b56-98ae-f8ab6da6ecc8-workload-certs\") pod \"e2e-ait-754cf73a-openshift-default-6b59879d8c-frkmc\" (UID: \"c840a66b-bbbb-4b56-98ae-f8ab6da6ecc8\") " pod="openshift-ingress/e2e-ait-754cf73a-openshift-default-6b59879d8c-frkmc" Jul 06 22:34:42.069491 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:42.069472 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c840a66b-bbbb-4b56-98ae-f8ab6da6ecc8-credential-socket\") pod \"e2e-ait-754cf73a-openshift-default-6b59879d8c-frkmc\" (UID: \"c840a66b-bbbb-4b56-98ae-f8ab6da6ecc8\") " pod="openshift-ingress/e2e-ait-754cf73a-openshift-default-6b59879d8c-frkmc" Jul 06 22:34:42.069646 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:42.069629 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c840a66b-bbbb-4b56-98ae-f8ab6da6ecc8-istiod-ca-cert\") pod \"e2e-ait-754cf73a-openshift-default-6b59879d8c-frkmc\" (UID: \"c840a66b-bbbb-4b56-98ae-f8ab6da6ecc8\") " pod="openshift-ingress/e2e-ait-754cf73a-openshift-default-6b59879d8c-frkmc" Jul 06 22:34:42.071191 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:42.071168 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c840a66b-bbbb-4b56-98ae-f8ab6da6ecc8-istio-envoy\") pod \"e2e-ait-754cf73a-openshift-default-6b59879d8c-frkmc\" (UID: \"c840a66b-bbbb-4b56-98ae-f8ab6da6ecc8\") " pod="openshift-ingress/e2e-ait-754cf73a-openshift-default-6b59879d8c-frkmc" Jul 06 22:34:42.071399 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:42.071382 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c840a66b-bbbb-4b56-98ae-f8ab6da6ecc8-istio-podinfo\") pod \"e2e-ait-754cf73a-openshift-default-6b59879d8c-frkmc\" (UID: \"c840a66b-bbbb-4b56-98ae-f8ab6da6ecc8\") " pod="openshift-ingress/e2e-ait-754cf73a-openshift-default-6b59879d8c-frkmc" Jul 06 22:34:42.076496 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:42.076477 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c840a66b-bbbb-4b56-98ae-f8ab6da6ecc8-istio-token\") pod \"e2e-ait-754cf73a-openshift-default-6b59879d8c-frkmc\" (UID: \"c840a66b-bbbb-4b56-98ae-f8ab6da6ecc8\") " pod="openshift-ingress/e2e-ait-754cf73a-openshift-default-6b59879d8c-frkmc" Jul 06 22:34:42.076677 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:42.076661 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p49w8\" (UniqueName: \"kubernetes.io/projected/c840a66b-bbbb-4b56-98ae-f8ab6da6ecc8-kube-api-access-p49w8\") pod \"e2e-ait-754cf73a-openshift-default-6b59879d8c-frkmc\" (UID: \"c840a66b-bbbb-4b56-98ae-f8ab6da6ecc8\") " pod="openshift-ingress/e2e-ait-754cf73a-openshift-default-6b59879d8c-frkmc" Jul 06 22:34:42.170242 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:42.170210 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-754cf73a-openshift-default-6b59879d8c-frkmc" Jul 06 22:34:42.304745 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:42.304717 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-754cf73a-openshift-default-6b59879d8c-frkmc"] Jul 06 22:34:42.306372 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:34:42.306340 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc840a66b_bbbb_4b56_98ae_f8ab6da6ecc8.slice/crio-b2c1553f9ef0705f12e0f7021b40358013483abf828e799f34e591c2c0679c6d WatchSource:0}: Error finding container b2c1553f9ef0705f12e0f7021b40358013483abf828e799f34e591c2c0679c6d: Status 404 returned error can't find the container with id b2c1553f9ef0705f12e0f7021b40358013483abf828e799f34e591c2c0679c6d Jul 06 22:34:42.308531 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:42.308501 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 22:34:42.308610 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:42.308566 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 22:34:42.308610 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:42.308605 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 22:34:42.749779 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:42.749743 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-754cf73a-openshift-default-6b59879d8c-frkmc" event={"ID":"c840a66b-bbbb-4b56-98ae-f8ab6da6ecc8","Type":"ContainerStarted","Data":"f864e837e959fd109f3db115d9b647a14de311284356e1fe073a1f24e81a2196"} Jul 06 22:34:42.749981 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:42.749803 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-754cf73a-openshift-default-6b59879d8c-frkmc" event={"ID":"c840a66b-bbbb-4b56-98ae-f8ab6da6ecc8","Type":"ContainerStarted","Data":"b2c1553f9ef0705f12e0f7021b40358013483abf828e799f34e591c2c0679c6d"} Jul 06 22:34:42.770279 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:42.770229 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-754cf73a-openshift-default-6b59879d8c-frkmc" podStartSLOduration=1.770212627 podStartE2EDuration="1.770212627s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:34:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:34:42.767832938 +0000 UTC m=+1562.326628270" watchObservedRunningTime="2026-07-06 22:34:42.770212627 +0000 UTC m=+1562.329007948" Jul 06 22:34:42.977315 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:42.977280 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4ff115a1-1b61-46a9-96d2-38fc5504e42a" path="/var/lib/kubelet/pods/4ff115a1-1b61-46a9-96d2-38fc5504e42a/volumes" Jul 06 22:34:42.992874 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:42.992827 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:34:43.086154 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:43.086053 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:34:43.171265 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:43.171228 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-754cf73a-openshift-default-6b59879d8c-frkmc" Jul 06 22:34:43.175385 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:43.175360 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-754cf73a-openshift-default-6b59879d8c-frkmc" Jul 06 22:34:43.219165 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:43.219131 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:34:43.641877 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:43.641841 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:34:43.746554 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:43.746521 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:34:43.756437 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:43.756413 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-754cf73a-openshift-default-6b59879d8c-frkmc" Jul 06 22:34:43.757253 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:43.757238 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-754cf73a-openshift-default-6b59879d8c-frkmc" Jul 06 22:34:43.842318 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:43.842267 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:34:45.279331 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:45.279302 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:34:45.384645 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:45.384608 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:34:48.182736 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:48.182701 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:34:48.518087 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:48.517962 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:34:54.245949 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:54.245909 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-754cf73a-openshift-default-6b59879d8c-frkmc"] Jul 06 22:34:54.246446 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:54.246158 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-754cf73a-openshift-default-6b59879d8c-frkmc" podUID="c840a66b-bbbb-4b56-98ae-f8ab6da6ecc8" containerName="istio-proxy" containerID="cri-o://f864e837e959fd109f3db115d9b647a14de311284356e1fe073a1f24e81a2196" gracePeriod=30 Jul 06 22:34:54.263593 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:54.263569 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:34:54.373907 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:54.373876 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:34:57.236135 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:57.236102 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:34:58.374705 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:58.374672 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:34:58.508391 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:58.508360 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:34:58.758784 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:58.758745 2581 patch_prober.go:28] interesting pod/e2e-ait-754cf73a-openshift-default-6b59879d8c-frkmc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 06 22:34:58.758960 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:58.758818 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-754cf73a-openshift-default-6b59879d8c-frkmc" podUID="c840a66b-bbbb-4b56-98ae-f8ab6da6ecc8" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 22:34:59.510101 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:59.510025 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-754cf73a-openshift-default-6b59879d8c-frkmc" Jul 06 22:34:59.635744 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:59.635713 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p49w8\" (UniqueName: \"kubernetes.io/projected/c840a66b-bbbb-4b56-98ae-f8ab6da6ecc8-kube-api-access-p49w8\") pod \"c840a66b-bbbb-4b56-98ae-f8ab6da6ecc8\" (UID: \"c840a66b-bbbb-4b56-98ae-f8ab6da6ecc8\") " Jul 06 22:34:59.635907 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:59.635809 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c840a66b-bbbb-4b56-98ae-f8ab6da6ecc8-istio-envoy\") pod \"c840a66b-bbbb-4b56-98ae-f8ab6da6ecc8\" (UID: \"c840a66b-bbbb-4b56-98ae-f8ab6da6ecc8\") " Jul 06 22:34:59.635907 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:59.635841 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c840a66b-bbbb-4b56-98ae-f8ab6da6ecc8-istio-data\") pod \"c840a66b-bbbb-4b56-98ae-f8ab6da6ecc8\" (UID: \"c840a66b-bbbb-4b56-98ae-f8ab6da6ecc8\") " Jul 06 22:34:59.635994 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:59.635916 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c840a66b-bbbb-4b56-98ae-f8ab6da6ecc8-workload-socket\") pod \"c840a66b-bbbb-4b56-98ae-f8ab6da6ecc8\" (UID: \"c840a66b-bbbb-4b56-98ae-f8ab6da6ecc8\") " Jul 06 22:34:59.635994 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:59.635942 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c840a66b-bbbb-4b56-98ae-f8ab6da6ecc8-istiod-ca-cert\") pod \"c840a66b-bbbb-4b56-98ae-f8ab6da6ecc8\" (UID: \"c840a66b-bbbb-4b56-98ae-f8ab6da6ecc8\") " Jul 06 22:34:59.635994 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:59.635980 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c840a66b-bbbb-4b56-98ae-f8ab6da6ecc8-istio-podinfo\") pod \"c840a66b-bbbb-4b56-98ae-f8ab6da6ecc8\" (UID: \"c840a66b-bbbb-4b56-98ae-f8ab6da6ecc8\") " Jul 06 22:34:59.636177 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:59.636003 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c840a66b-bbbb-4b56-98ae-f8ab6da6ecc8-workload-certs\") pod \"c840a66b-bbbb-4b56-98ae-f8ab6da6ecc8\" (UID: \"c840a66b-bbbb-4b56-98ae-f8ab6da6ecc8\") " Jul 06 22:34:59.636177 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:59.636036 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c840a66b-bbbb-4b56-98ae-f8ab6da6ecc8-istio-token\") pod \"c840a66b-bbbb-4b56-98ae-f8ab6da6ecc8\" (UID: \"c840a66b-bbbb-4b56-98ae-f8ab6da6ecc8\") " Jul 06 22:34:59.636177 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:59.636092 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c840a66b-bbbb-4b56-98ae-f8ab6da6ecc8-credential-socket\") pod \"c840a66b-bbbb-4b56-98ae-f8ab6da6ecc8\" (UID: \"c840a66b-bbbb-4b56-98ae-f8ab6da6ecc8\") " Jul 06 22:34:59.636334 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:59.636219 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c840a66b-bbbb-4b56-98ae-f8ab6da6ecc8-istio-data" (OuterVolumeSpecName: "istio-data") pod "c840a66b-bbbb-4b56-98ae-f8ab6da6ecc8" (UID: "c840a66b-bbbb-4b56-98ae-f8ab6da6ecc8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:34:59.636334 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:59.636239 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c840a66b-bbbb-4b56-98ae-f8ab6da6ecc8-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c840a66b-bbbb-4b56-98ae-f8ab6da6ecc8" (UID: "c840a66b-bbbb-4b56-98ae-f8ab6da6ecc8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:34:59.636432 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:59.636351 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c840a66b-bbbb-4b56-98ae-f8ab6da6ecc8-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c840a66b-bbbb-4b56-98ae-f8ab6da6ecc8" (UID: "c840a66b-bbbb-4b56-98ae-f8ab6da6ecc8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 22:34:59.636484 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:59.636446 2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c840a66b-bbbb-4b56-98ae-f8ab6da6ecc8-workload-certs\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:34:59.636484 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:59.636448 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c840a66b-bbbb-4b56-98ae-f8ab6da6ecc8-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c840a66b-bbbb-4b56-98ae-f8ab6da6ecc8" (UID: "c840a66b-bbbb-4b56-98ae-f8ab6da6ecc8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:34:59.636591 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:59.636483 2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c840a66b-bbbb-4b56-98ae-f8ab6da6ecc8-istio-data\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:34:59.636717 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:59.636687 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c840a66b-bbbb-4b56-98ae-f8ab6da6ecc8-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c840a66b-bbbb-4b56-98ae-f8ab6da6ecc8" (UID: "c840a66b-bbbb-4b56-98ae-f8ab6da6ecc8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:34:59.637976 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:59.637941 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c840a66b-bbbb-4b56-98ae-f8ab6da6ecc8-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c840a66b-bbbb-4b56-98ae-f8ab6da6ecc8" (UID: "c840a66b-bbbb-4b56-98ae-f8ab6da6ecc8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:34:59.638117 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:59.637980 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c840a66b-bbbb-4b56-98ae-f8ab6da6ecc8-kube-api-access-p49w8" (OuterVolumeSpecName: "kube-api-access-p49w8") pod "c840a66b-bbbb-4b56-98ae-f8ab6da6ecc8" (UID: "c840a66b-bbbb-4b56-98ae-f8ab6da6ecc8"). InnerVolumeSpecName "kube-api-access-p49w8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 22:34:59.638471 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:59.638439 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c840a66b-bbbb-4b56-98ae-f8ab6da6ecc8-istio-token" (OuterVolumeSpecName: "istio-token") pod "c840a66b-bbbb-4b56-98ae-f8ab6da6ecc8" (UID: "c840a66b-bbbb-4b56-98ae-f8ab6da6ecc8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 22:34:59.638471 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:59.638440 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c840a66b-bbbb-4b56-98ae-f8ab6da6ecc8-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c840a66b-bbbb-4b56-98ae-f8ab6da6ecc8" (UID: "c840a66b-bbbb-4b56-98ae-f8ab6da6ecc8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 22:34:59.737029 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:59.736996 2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c840a66b-bbbb-4b56-98ae-f8ab6da6ecc8-istio-envoy\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:34:59.737029 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:59.737023 2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c840a66b-bbbb-4b56-98ae-f8ab6da6ecc8-workload-socket\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:34:59.737029 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:59.737032 2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c840a66b-bbbb-4b56-98ae-f8ab6da6ecc8-istiod-ca-cert\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:34:59.737029 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:59.737041 2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c840a66b-bbbb-4b56-98ae-f8ab6da6ecc8-istio-podinfo\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:34:59.737288 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:59.737049 2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c840a66b-bbbb-4b56-98ae-f8ab6da6ecc8-istio-token\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:34:59.737288 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:59.737057 2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c840a66b-bbbb-4b56-98ae-f8ab6da6ecc8-credential-socket\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:34:59.737288 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:59.737088 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p49w8\" (UniqueName: \"kubernetes.io/projected/c840a66b-bbbb-4b56-98ae-f8ab6da6ecc8-kube-api-access-p49w8\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:34:59.841958 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:59.841863 2581 generic.go:356] "Generic (PLEG): container finished" podID="c840a66b-bbbb-4b56-98ae-f8ab6da6ecc8" containerID="f864e837e959fd109f3db115d9b647a14de311284356e1fe073a1f24e81a2196" exitCode=0 Jul 06 22:34:59.841958 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:59.841906 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-754cf73a-openshift-default-6b59879d8c-frkmc" Jul 06 22:34:59.841958 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:59.841943 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-754cf73a-openshift-default-6b59879d8c-frkmc" event={"ID":"c840a66b-bbbb-4b56-98ae-f8ab6da6ecc8","Type":"ContainerDied","Data":"f864e837e959fd109f3db115d9b647a14de311284356e1fe073a1f24e81a2196"} Jul 06 22:34:59.842275 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:59.842022 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-754cf73a-openshift-default-6b59879d8c-frkmc" event={"ID":"c840a66b-bbbb-4b56-98ae-f8ab6da6ecc8","Type":"ContainerDied","Data":"b2c1553f9ef0705f12e0f7021b40358013483abf828e799f34e591c2c0679c6d"} Jul 06 22:34:59.842275 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:59.842053 2581 scope.go:117] "RemoveContainer" containerID="f864e837e959fd109f3db115d9b647a14de311284356e1fe073a1f24e81a2196" Jul 06 22:34:59.851946 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:59.851927 2581 scope.go:117] "RemoveContainer" containerID="f864e837e959fd109f3db115d9b647a14de311284356e1fe073a1f24e81a2196" Jul 06 22:34:59.852228 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:34:59.852207 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f864e837e959fd109f3db115d9b647a14de311284356e1fe073a1f24e81a2196\": container with ID starting with f864e837e959fd109f3db115d9b647a14de311284356e1fe073a1f24e81a2196 not found: ID does not exist" containerID="f864e837e959fd109f3db115d9b647a14de311284356e1fe073a1f24e81a2196" Jul 06 22:34:59.852303 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:59.852236 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f864e837e959fd109f3db115d9b647a14de311284356e1fe073a1f24e81a2196"} err="failed to get container status \"f864e837e959fd109f3db115d9b647a14de311284356e1fe073a1f24e81a2196\": rpc error: code = NotFound desc = could not find container \"f864e837e959fd109f3db115d9b647a14de311284356e1fe073a1f24e81a2196\": container with ID starting with f864e837e959fd109f3db115d9b647a14de311284356e1fe073a1f24e81a2196 not found: ID does not exist" Jul 06 22:34:59.865108 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:59.865081 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-754cf73a-openshift-default-6b59879d8c-frkmc"] Jul 06 22:34:59.870811 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:34:59.870791 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-754cf73a-openshift-default-6b59879d8c-frkmc"] Jul 06 22:35:00.685771 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:35:00.685733 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:35:00.762544 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:35:00.762510 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:35:00.934008 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:35:00.933978 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-d4b08138-openshift-default-7f68b9f947-dxmrk"] Jul 06 22:35:00.934566 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:35:00.934536 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c840a66b-bbbb-4b56-98ae-f8ab6da6ecc8" containerName="istio-proxy" Jul 06 22:35:00.934566 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:35:00.934560 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="c840a66b-bbbb-4b56-98ae-f8ab6da6ecc8" containerName="istio-proxy" Jul 06 22:35:00.934771 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:35:00.934675 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="c840a66b-bbbb-4b56-98ae-f8ab6da6ecc8" containerName="istio-proxy" Jul 06 22:35:00.939516 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:35:00.939455 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-d4b08138-openshift-default-7f68b9f947-dxmrk" Jul 06 22:35:00.942566 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:35:00.942547 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-d4b08138-openshift-default-dockercfg-4vtsl\"" Jul 06 22:35:00.977733 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:35:00.977703 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c840a66b-bbbb-4b56-98ae-f8ab6da6ecc8" path="/var/lib/kubelet/pods/c840a66b-bbbb-4b56-98ae-f8ab6da6ecc8/volumes" Jul 06 22:35:00.977981 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:35:00.977959 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-d4b08138-openshift-default-7f68b9f947-dxmrk"] Jul 06 22:35:00.986357 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:35:00.986336 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:35:01.049591 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:35:01.049559 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/037ebfa8-d25c-4832-b2b3-7e0c0d35c081-workload-socket\") pod \"e2e-derive-d4b08138-openshift-default-7f68b9f947-dxmrk\" (UID: \"037ebfa8-d25c-4832-b2b3-7e0c0d35c081\") " pod="openshift-ingress/e2e-derive-d4b08138-openshift-default-7f68b9f947-dxmrk" Jul 06 22:35:01.049775 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:35:01.049604 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/037ebfa8-d25c-4832-b2b3-7e0c0d35c081-istio-data\") pod \"e2e-derive-d4b08138-openshift-default-7f68b9f947-dxmrk\" (UID: \"037ebfa8-d25c-4832-b2b3-7e0c0d35c081\") " pod="openshift-ingress/e2e-derive-d4b08138-openshift-default-7f68b9f947-dxmrk" Jul 06 22:35:01.049775 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:35:01.049628 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/037ebfa8-d25c-4832-b2b3-7e0c0d35c081-istio-envoy\") pod \"e2e-derive-d4b08138-openshift-default-7f68b9f947-dxmrk\" (UID: \"037ebfa8-d25c-4832-b2b3-7e0c0d35c081\") " pod="openshift-ingress/e2e-derive-d4b08138-openshift-default-7f68b9f947-dxmrk" Jul 06 22:35:01.049775 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:35:01.049645 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vvk6t\" (UniqueName: \"kubernetes.io/projected/037ebfa8-d25c-4832-b2b3-7e0c0d35c081-kube-api-access-vvk6t\") pod \"e2e-derive-d4b08138-openshift-default-7f68b9f947-dxmrk\" (UID: \"037ebfa8-d25c-4832-b2b3-7e0c0d35c081\") " pod="openshift-ingress/e2e-derive-d4b08138-openshift-default-7f68b9f947-dxmrk" Jul 06 22:35:01.049775 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:35:01.049707 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/037ebfa8-d25c-4832-b2b3-7e0c0d35c081-credential-socket\") pod \"e2e-derive-d4b08138-openshift-default-7f68b9f947-dxmrk\" (UID: \"037ebfa8-d25c-4832-b2b3-7e0c0d35c081\") " pod="openshift-ingress/e2e-derive-d4b08138-openshift-default-7f68b9f947-dxmrk" Jul 06 22:35:01.049775 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:35:01.049747 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/037ebfa8-d25c-4832-b2b3-7e0c0d35c081-istio-token\") pod \"e2e-derive-d4b08138-openshift-default-7f68b9f947-dxmrk\" (UID: \"037ebfa8-d25c-4832-b2b3-7e0c0d35c081\") " pod="openshift-ingress/e2e-derive-d4b08138-openshift-default-7f68b9f947-dxmrk" Jul 06 22:35:01.049992 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:35:01.049816 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/037ebfa8-d25c-4832-b2b3-7e0c0d35c081-istio-podinfo\") pod \"e2e-derive-d4b08138-openshift-default-7f68b9f947-dxmrk\" (UID: \"037ebfa8-d25c-4832-b2b3-7e0c0d35c081\") " pod="openshift-ingress/e2e-derive-d4b08138-openshift-default-7f68b9f947-dxmrk" Jul 06 22:35:01.049992 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:35:01.049840 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/037ebfa8-d25c-4832-b2b3-7e0c0d35c081-istiod-ca-cert\") pod \"e2e-derive-d4b08138-openshift-default-7f68b9f947-dxmrk\" (UID: \"037ebfa8-d25c-4832-b2b3-7e0c0d35c081\") " pod="openshift-ingress/e2e-derive-d4b08138-openshift-default-7f68b9f947-dxmrk" Jul 06 22:35:01.049992 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:35:01.049917 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/037ebfa8-d25c-4832-b2b3-7e0c0d35c081-workload-certs\") pod \"e2e-derive-d4b08138-openshift-default-7f68b9f947-dxmrk\" (UID: \"037ebfa8-d25c-4832-b2b3-7e0c0d35c081\") " pod="openshift-ingress/e2e-derive-d4b08138-openshift-default-7f68b9f947-dxmrk" Jul 06 22:35:01.147323 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:35:01.147286 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:35:01.151395 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:35:01.151375 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/037ebfa8-d25c-4832-b2b3-7e0c0d35c081-workload-socket\") pod \"e2e-derive-d4b08138-openshift-default-7f68b9f947-dxmrk\" (UID: \"037ebfa8-d25c-4832-b2b3-7e0c0d35c081\") " pod="openshift-ingress/e2e-derive-d4b08138-openshift-default-7f68b9f947-dxmrk" Jul 06 22:35:01.151507 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:35:01.151420 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/037ebfa8-d25c-4832-b2b3-7e0c0d35c081-istio-data\") pod \"e2e-derive-d4b08138-openshift-default-7f68b9f947-dxmrk\" (UID: \"037ebfa8-d25c-4832-b2b3-7e0c0d35c081\") " pod="openshift-ingress/e2e-derive-d4b08138-openshift-default-7f68b9f947-dxmrk" Jul 06 22:35:01.151507 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:35:01.151439 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/037ebfa8-d25c-4832-b2b3-7e0c0d35c081-istio-envoy\") pod \"e2e-derive-d4b08138-openshift-default-7f68b9f947-dxmrk\" (UID: \"037ebfa8-d25c-4832-b2b3-7e0c0d35c081\") " pod="openshift-ingress/e2e-derive-d4b08138-openshift-default-7f68b9f947-dxmrk" Jul 06 22:35:01.151507 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:35:01.151460 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vvk6t\" (UniqueName: \"kubernetes.io/projected/037ebfa8-d25c-4832-b2b3-7e0c0d35c081-kube-api-access-vvk6t\") pod \"e2e-derive-d4b08138-openshift-default-7f68b9f947-dxmrk\" (UID: \"037ebfa8-d25c-4832-b2b3-7e0c0d35c081\") " pod="openshift-ingress/e2e-derive-d4b08138-openshift-default-7f68b9f947-dxmrk" Jul 06 22:35:01.151507 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:35:01.151492 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/037ebfa8-d25c-4832-b2b3-7e0c0d35c081-credential-socket\") pod \"e2e-derive-d4b08138-openshift-default-7f68b9f947-dxmrk\" (UID: \"037ebfa8-d25c-4832-b2b3-7e0c0d35c081\") " pod="openshift-ingress/e2e-derive-d4b08138-openshift-default-7f68b9f947-dxmrk" Jul 06 22:35:01.151714 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:35:01.151524 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/037ebfa8-d25c-4832-b2b3-7e0c0d35c081-istio-token\") pod \"e2e-derive-d4b08138-openshift-default-7f68b9f947-dxmrk\" (UID: \"037ebfa8-d25c-4832-b2b3-7e0c0d35c081\") " pod="openshift-ingress/e2e-derive-d4b08138-openshift-default-7f68b9f947-dxmrk" Jul 06 22:35:01.151714 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:35:01.151596 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/037ebfa8-d25c-4832-b2b3-7e0c0d35c081-istio-podinfo\") pod \"e2e-derive-d4b08138-openshift-default-7f68b9f947-dxmrk\" (UID: \"037ebfa8-d25c-4832-b2b3-7e0c0d35c081\") " pod="openshift-ingress/e2e-derive-d4b08138-openshift-default-7f68b9f947-dxmrk" Jul 06 22:35:01.151714 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:35:01.151617 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/037ebfa8-d25c-4832-b2b3-7e0c0d35c081-istiod-ca-cert\") pod \"e2e-derive-d4b08138-openshift-default-7f68b9f947-dxmrk\" (UID: \"037ebfa8-d25c-4832-b2b3-7e0c0d35c081\") " pod="openshift-ingress/e2e-derive-d4b08138-openshift-default-7f68b9f947-dxmrk" Jul 06 22:35:01.151714 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:35:01.151659 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/037ebfa8-d25c-4832-b2b3-7e0c0d35c081-workload-certs\") pod \"e2e-derive-d4b08138-openshift-default-7f68b9f947-dxmrk\" (UID: \"037ebfa8-d25c-4832-b2b3-7e0c0d35c081\") " pod="openshift-ingress/e2e-derive-d4b08138-openshift-default-7f68b9f947-dxmrk" Jul 06 22:35:01.151902 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:35:01.151776 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/037ebfa8-d25c-4832-b2b3-7e0c0d35c081-workload-socket\") pod \"e2e-derive-d4b08138-openshift-default-7f68b9f947-dxmrk\" (UID: \"037ebfa8-d25c-4832-b2b3-7e0c0d35c081\") " pod="openshift-ingress/e2e-derive-d4b08138-openshift-default-7f68b9f947-dxmrk" Jul 06 22:35:01.151902 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:35:01.151881 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/037ebfa8-d25c-4832-b2b3-7e0c0d35c081-istio-data\") pod \"e2e-derive-d4b08138-openshift-default-7f68b9f947-dxmrk\" (UID: \"037ebfa8-d25c-4832-b2b3-7e0c0d35c081\") " pod="openshift-ingress/e2e-derive-d4b08138-openshift-default-7f68b9f947-dxmrk" Jul 06 22:35:01.151994 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:35:01.151950 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/037ebfa8-d25c-4832-b2b3-7e0c0d35c081-credential-socket\") pod \"e2e-derive-d4b08138-openshift-default-7f68b9f947-dxmrk\" (UID: \"037ebfa8-d25c-4832-b2b3-7e0c0d35c081\") " pod="openshift-ingress/e2e-derive-d4b08138-openshift-default-7f68b9f947-dxmrk" Jul 06 22:35:01.152249 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:35:01.152234 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/037ebfa8-d25c-4832-b2b3-7e0c0d35c081-workload-certs\") pod \"e2e-derive-d4b08138-openshift-default-7f68b9f947-dxmrk\" (UID: \"037ebfa8-d25c-4832-b2b3-7e0c0d35c081\") " pod="openshift-ingress/e2e-derive-d4b08138-openshift-default-7f68b9f947-dxmrk" Jul 06 22:35:01.152537 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:35:01.152512 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/037ebfa8-d25c-4832-b2b3-7e0c0d35c081-istiod-ca-cert\") pod \"e2e-derive-d4b08138-openshift-default-7f68b9f947-dxmrk\" (UID: \"037ebfa8-d25c-4832-b2b3-7e0c0d35c081\") " pod="openshift-ingress/e2e-derive-d4b08138-openshift-default-7f68b9f947-dxmrk" Jul 06 22:35:01.153797 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:35:01.153775 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/037ebfa8-d25c-4832-b2b3-7e0c0d35c081-istio-envoy\") pod \"e2e-derive-d4b08138-openshift-default-7f68b9f947-dxmrk\" (UID: \"037ebfa8-d25c-4832-b2b3-7e0c0d35c081\") " pod="openshift-ingress/e2e-derive-d4b08138-openshift-default-7f68b9f947-dxmrk" Jul 06 22:35:01.154117 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:35:01.154098 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/037ebfa8-d25c-4832-b2b3-7e0c0d35c081-istio-podinfo\") pod \"e2e-derive-d4b08138-openshift-default-7f68b9f947-dxmrk\" (UID: \"037ebfa8-d25c-4832-b2b3-7e0c0d35c081\") " pod="openshift-ingress/e2e-derive-d4b08138-openshift-default-7f68b9f947-dxmrk" Jul 06 22:35:01.162086 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:35:01.162045 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/037ebfa8-d25c-4832-b2b3-7e0c0d35c081-istio-token\") pod \"e2e-derive-d4b08138-openshift-default-7f68b9f947-dxmrk\" (UID: \"037ebfa8-d25c-4832-b2b3-7e0c0d35c081\") " pod="openshift-ingress/e2e-derive-d4b08138-openshift-default-7f68b9f947-dxmrk" Jul 06 22:35:01.162300 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:35:01.162279 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vvk6t\" (UniqueName: \"kubernetes.io/projected/037ebfa8-d25c-4832-b2b3-7e0c0d35c081-kube-api-access-vvk6t\") pod \"e2e-derive-d4b08138-openshift-default-7f68b9f947-dxmrk\" (UID: \"037ebfa8-d25c-4832-b2b3-7e0c0d35c081\") " pod="openshift-ingress/e2e-derive-d4b08138-openshift-default-7f68b9f947-dxmrk" Jul 06 22:35:01.249905 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:35:01.249842 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-d4b08138-openshift-default-7f68b9f947-dxmrk" Jul 06 22:35:01.393540 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:35:01.393515 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-d4b08138-openshift-default-7f68b9f947-dxmrk"] Jul 06 22:35:01.395604 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:35:01.395567 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod037ebfa8_d25c_4832_b2b3_7e0c0d35c081.slice/crio-8fb6295a614621afd5b9cf8d9d2f3032eb7cfccecf5978a0006ee0474d42eafe WatchSource:0}: Error finding container 8fb6295a614621afd5b9cf8d9d2f3032eb7cfccecf5978a0006ee0474d42eafe: Status 404 returned error can't find the container with id 8fb6295a614621afd5b9cf8d9d2f3032eb7cfccecf5978a0006ee0474d42eafe Jul 06 22:35:01.397689 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:35:01.397646 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 22:35:01.397787 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:35:01.397716 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 22:35:01.397787 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:35:01.397752 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 22:35:01.856287 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:35:01.856239 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-d4b08138-openshift-default-7f68b9f947-dxmrk" event={"ID":"037ebfa8-d25c-4832-b2b3-7e0c0d35c081","Type":"ContainerStarted","Data":"d74e66ba4efc9ccc91423673ce218ae763b379c0ddfd4d613369f15170600fff"} Jul 06 22:35:01.856731 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:35:01.856305 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-d4b08138-openshift-default-7f68b9f947-dxmrk" event={"ID":"037ebfa8-d25c-4832-b2b3-7e0c0d35c081","Type":"ContainerStarted","Data":"8fb6295a614621afd5b9cf8d9d2f3032eb7cfccecf5978a0006ee0474d42eafe"} Jul 06 22:35:01.876576 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:35:01.876518 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-d4b08138-openshift-default-7f68b9f947-dxmrk" podStartSLOduration=1.8765013590000001 podStartE2EDuration="1.876501359s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:35:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:35:01.875484263 +0000 UTC m=+1581.434279583" watchObservedRunningTime="2026-07-06 22:35:01.876501359 +0000 UTC m=+1581.435296677" Jul 06 22:35:02.101588 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:35:02.101555 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:35:02.198042 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:35:02.197984 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:35:02.250910 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:35:02.250872 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-d4b08138-openshift-default-7f68b9f947-dxmrk" Jul 06 22:35:02.255984 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:35:02.255955 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-d4b08138-openshift-default-7f68b9f947-dxmrk" Jul 06 22:35:02.366168 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:35:02.366132 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:35:02.481034 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:35:02.480917 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:35:02.768635 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:35:02.768541 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:35:02.861218 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:35:02.861180 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-d4b08138-openshift-default-7f68b9f947-dxmrk" Jul 06 22:35:02.862373 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:35:02.862351 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-d4b08138-openshift-default-7f68b9f947-dxmrk" Jul 06 22:35:02.869205 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:35:02.869182 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:35:04.271989 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:35:04.271956 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:35:06.485311 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:35:06.485275 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:35:06.653274 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:35:06.653229 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-d4b08138-openshift-default-7f68b9f947-dxmrk"] Jul 06 22:35:06.653479 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:35:06.653454 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-d4b08138-openshift-default-7f68b9f947-dxmrk" podUID="037ebfa8-d25c-4832-b2b3-7e0c0d35c081" containerName="istio-proxy" containerID="cri-o://d74e66ba4efc9ccc91423673ce218ae763b379c0ddfd4d613369f15170600fff" gracePeriod=30 Jul 06 22:35:06.679143 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:35:06.679091 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:35:06.765976 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:35:06.765877 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:35:06.878348 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:35:06.878295 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:35:08.497059 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:35:08.497026 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:35:12.811320 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:35:12.811282 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:35:12.871534 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:35:12.871513 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:35:12.916784 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:35:12.916745 2581 generic.go:356] "Generic (PLEG): container finished" podID="037ebfa8-d25c-4832-b2b3-7e0c0d35c081" containerID="d74e66ba4efc9ccc91423673ce218ae763b379c0ddfd4d613369f15170600fff" exitCode=0 Jul 06 22:35:12.916932 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:35:12.916800 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-d4b08138-openshift-default-7f68b9f947-dxmrk" event={"ID":"037ebfa8-d25c-4832-b2b3-7e0c0d35c081","Type":"ContainerDied","Data":"d74e66ba4efc9ccc91423673ce218ae763b379c0ddfd4d613369f15170600fff"} Jul 06 22:35:13.014752 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:35:13.014725 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-d4b08138-openshift-default-7f68b9f947-dxmrk" Jul 06 22:35:13.055633 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:35:13.055604 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/037ebfa8-d25c-4832-b2b3-7e0c0d35c081-credential-socket\") pod \"037ebfa8-d25c-4832-b2b3-7e0c0d35c081\" (UID: \"037ebfa8-d25c-4832-b2b3-7e0c0d35c081\") " Jul 06 22:35:13.055806 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:35:13.055641 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/037ebfa8-d25c-4832-b2b3-7e0c0d35c081-workload-certs\") pod \"037ebfa8-d25c-4832-b2b3-7e0c0d35c081\" (UID: \"037ebfa8-d25c-4832-b2b3-7e0c0d35c081\") " Jul 06 22:35:13.055806 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:35:13.055711 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/037ebfa8-d25c-4832-b2b3-7e0c0d35c081-istio-data\") pod \"037ebfa8-d25c-4832-b2b3-7e0c0d35c081\" (UID: \"037ebfa8-d25c-4832-b2b3-7e0c0d35c081\") " Jul 06 22:35:13.055932 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:35:13.055801 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/037ebfa8-d25c-4832-b2b3-7e0c0d35c081-istio-envoy\") pod \"037ebfa8-d25c-4832-b2b3-7e0c0d35c081\" (UID: \"037ebfa8-d25c-4832-b2b3-7e0c0d35c081\") " Jul 06 22:35:13.055932 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:35:13.055872 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/037ebfa8-d25c-4832-b2b3-7e0c0d35c081-istio-podinfo\") pod \"037ebfa8-d25c-4832-b2b3-7e0c0d35c081\" (UID: \"037ebfa8-d25c-4832-b2b3-7e0c0d35c081\") " Jul 06 22:35:13.055932 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:35:13.055911 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/037ebfa8-d25c-4832-b2b3-7e0c0d35c081-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "037ebfa8-d25c-4832-b2b3-7e0c0d35c081" (UID: "037ebfa8-d25c-4832-b2b3-7e0c0d35c081"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:35:13.056107 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:35:13.055917 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vvk6t\" (UniqueName: \"kubernetes.io/projected/037ebfa8-d25c-4832-b2b3-7e0c0d35c081-kube-api-access-vvk6t\") pod \"037ebfa8-d25c-4832-b2b3-7e0c0d35c081\" (UID: \"037ebfa8-d25c-4832-b2b3-7e0c0d35c081\") " Jul 06 22:35:13.056107 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:35:13.056015 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/037ebfa8-d25c-4832-b2b3-7e0c0d35c081-istio-token\") pod \"037ebfa8-d25c-4832-b2b3-7e0c0d35c081\" (UID: \"037ebfa8-d25c-4832-b2b3-7e0c0d35c081\") " Jul 06 22:35:13.056107 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:35:13.056045 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/037ebfa8-d25c-4832-b2b3-7e0c0d35c081-istiod-ca-cert\") pod \"037ebfa8-d25c-4832-b2b3-7e0c0d35c081\" (UID: \"037ebfa8-d25c-4832-b2b3-7e0c0d35c081\") " Jul 06 22:35:13.056277 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:35:13.056124 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/037ebfa8-d25c-4832-b2b3-7e0c0d35c081-workload-socket\") pod \"037ebfa8-d25c-4832-b2b3-7e0c0d35c081\" (UID: \"037ebfa8-d25c-4832-b2b3-7e0c0d35c081\") " Jul 06 22:35:13.056891 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:35:13.056516 2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/037ebfa8-d25c-4832-b2b3-7e0c0d35c081-credential-socket\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:35:13.056891 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:35:13.056573 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/037ebfa8-d25c-4832-b2b3-7e0c0d35c081-istio-data" (OuterVolumeSpecName: "istio-data") pod "037ebfa8-d25c-4832-b2b3-7e0c0d35c081" (UID: "037ebfa8-d25c-4832-b2b3-7e0c0d35c081"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:35:13.056891 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:35:13.056826 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/037ebfa8-d25c-4832-b2b3-7e0c0d35c081-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "037ebfa8-d25c-4832-b2b3-7e0c0d35c081" (UID: "037ebfa8-d25c-4832-b2b3-7e0c0d35c081"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:35:13.057150 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:35:13.057115 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/037ebfa8-d25c-4832-b2b3-7e0c0d35c081-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "037ebfa8-d25c-4832-b2b3-7e0c0d35c081" (UID: "037ebfa8-d25c-4832-b2b3-7e0c0d35c081"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 22:35:13.057189 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:35:13.057174 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/037ebfa8-d25c-4832-b2b3-7e0c0d35c081-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "037ebfa8-d25c-4832-b2b3-7e0c0d35c081" (UID: "037ebfa8-d25c-4832-b2b3-7e0c0d35c081"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:35:13.058798 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:35:13.058760 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/037ebfa8-d25c-4832-b2b3-7e0c0d35c081-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "037ebfa8-d25c-4832-b2b3-7e0c0d35c081" (UID: "037ebfa8-d25c-4832-b2b3-7e0c0d35c081"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:35:13.058944 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:35:13.058917 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/037ebfa8-d25c-4832-b2b3-7e0c0d35c081-kube-api-access-vvk6t" (OuterVolumeSpecName: "kube-api-access-vvk6t") pod "037ebfa8-d25c-4832-b2b3-7e0c0d35c081" (UID: "037ebfa8-d25c-4832-b2b3-7e0c0d35c081"). InnerVolumeSpecName "kube-api-access-vvk6t". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 22:35:13.059081 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:35:13.059038 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/037ebfa8-d25c-4832-b2b3-7e0c0d35c081-istio-token" (OuterVolumeSpecName: "istio-token") pod "037ebfa8-d25c-4832-b2b3-7e0c0d35c081" (UID: "037ebfa8-d25c-4832-b2b3-7e0c0d35c081"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 22:35:13.059147 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:35:13.059044 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/037ebfa8-d25c-4832-b2b3-7e0c0d35c081-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "037ebfa8-d25c-4832-b2b3-7e0c0d35c081" (UID: "037ebfa8-d25c-4832-b2b3-7e0c0d35c081"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 22:35:13.157745 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:35:13.157707 2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/037ebfa8-d25c-4832-b2b3-7e0c0d35c081-istio-data\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:35:13.157745 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:35:13.157740 2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/037ebfa8-d25c-4832-b2b3-7e0c0d35c081-istio-envoy\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:35:13.157745 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:35:13.157752 2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/037ebfa8-d25c-4832-b2b3-7e0c0d35c081-istio-podinfo\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:35:13.157992 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:35:13.157766 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vvk6t\" (UniqueName: \"kubernetes.io/projected/037ebfa8-d25c-4832-b2b3-7e0c0d35c081-kube-api-access-vvk6t\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:35:13.157992 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:35:13.157781 2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/037ebfa8-d25c-4832-b2b3-7e0c0d35c081-istio-token\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:35:13.157992 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:35:13.157789 2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/037ebfa8-d25c-4832-b2b3-7e0c0d35c081-istiod-ca-cert\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:35:13.157992 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:35:13.157797 2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/037ebfa8-d25c-4832-b2b3-7e0c0d35c081-workload-socket\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:35:13.157992 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:35:13.157804 2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/037ebfa8-d25c-4832-b2b3-7e0c0d35c081-workload-certs\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:35:13.289042 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:35:13.289007 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:35:13.388313 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:35:13.388231 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:35:13.489330 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:35:13.489297 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:35:13.923493 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:35:13.923468 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-d4b08138-openshift-default-7f68b9f947-dxmrk" Jul 06 22:35:13.923897 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:35:13.923504 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-d4b08138-openshift-default-7f68b9f947-dxmrk" event={"ID":"037ebfa8-d25c-4832-b2b3-7e0c0d35c081","Type":"ContainerDied","Data":"8fb6295a614621afd5b9cf8d9d2f3032eb7cfccecf5978a0006ee0474d42eafe"} Jul 06 22:35:13.923897 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:35:13.923560 2581 scope.go:117] "RemoveContainer" containerID="d74e66ba4efc9ccc91423673ce218ae763b379c0ddfd4d613369f15170600fff" Jul 06 22:35:13.946028 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:35:13.946001 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-d4b08138-openshift-default-7f68b9f947-dxmrk"] Jul 06 22:35:13.949120 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:35:13.949098 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-d4b08138-openshift-default-7f68b9f947-dxmrk"] Jul 06 22:35:14.824162 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:35:14.824135 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:35:14.976351 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:35:14.976319 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="037ebfa8-d25c-4832-b2b3-7e0c0d35c081" path="/var/lib/kubelet/pods/037ebfa8-d25c-4832-b2b3-7e0c0d35c081/volumes" Jul 06 22:35:20.683358 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:35:20.683325 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:35:20.749011 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:35:20.748975 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:35:20.831057 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:35:20.831025 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:35:20.928198 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:35:20.928169 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:35:21.028113 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:35:21.028006 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:35:21.398622 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:35:21.398542 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:35:47.398041 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:35:47.398007 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:35:53.167324 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:35:53.167283 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:35:53.271115 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:35:53.271079 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:35:53.367572 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:35:53.367539 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:36:09.280798 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:36:09.280769 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:36:15.367576 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:36:15.367540 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:36:15.477369 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:36:15.477336 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:36:15.568103 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:36:15.568055 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:36:26.072547 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:36:26.072514 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:36:26.891700 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:36:26.891665 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:36:32.677990 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:36:32.677956 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:36:32.732799 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:36:32.732746 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:36:32.808269 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:36:32.808236 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:36:32.909296 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:36:32.909264 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:36:33.009110 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:36:33.008977 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:36:33.747766 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:36:33.747723 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:36:33.842217 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:36:33.842188 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:36:33.963169 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:36:33.963125 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:36:35.594619 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:36:35.594571 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:36:41.517491 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:36:41.517453 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:36:41.617549 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:36:41.617496 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:36:42.199148 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:36:42.199119 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:36:48.042845 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:36:48.042801 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:36:48.144895 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:36:48.144858 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:36:48.273377 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:36:48.273340 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:36:48.540836 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:36:48.540798 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-57b5abb4-openshift-default-58ccbd7fff-gg65j"] Jul 06 22:36:48.542387 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:36:48.542363 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="037ebfa8-d25c-4832-b2b3-7e0c0d35c081" containerName="istio-proxy" Jul 06 22:36:48.542387 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:36:48.542384 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="037ebfa8-d25c-4832-b2b3-7e0c0d35c081" containerName="istio-proxy" Jul 06 22:36:48.542556 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:36:48.542450 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="037ebfa8-d25c-4832-b2b3-7e0c0d35c081" containerName="istio-proxy" Jul 06 22:36:48.546249 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:36:48.546228 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-57b5abb4-openshift-default-58ccbd7fff-gg65j" Jul 06 22:36:48.548725 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:36:48.548703 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-57b5abb4-openshift-default-dockercfg-r2frn\"" Jul 06 22:36:48.569944 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:36:48.569919 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-57b5abb4-openshift-default-58ccbd7fff-gg65j"] Jul 06 22:36:48.571121 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:36:48.571059 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:36:48.612496 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:36:48.612460 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f310a441-d7a1-44f3-a5cf-4d32a52458c8-workload-socket\") pod \"e2e-mt-57b5abb4-openshift-default-58ccbd7fff-gg65j\" (UID: \"f310a441-d7a1-44f3-a5cf-4d32a52458c8\") " pod="openshift-ingress/e2e-mt-57b5abb4-openshift-default-58ccbd7fff-gg65j" Jul 06 22:36:48.612671 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:36:48.612526 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f310a441-d7a1-44f3-a5cf-4d32a52458c8-workload-certs\") pod \"e2e-mt-57b5abb4-openshift-default-58ccbd7fff-gg65j\" (UID: \"f310a441-d7a1-44f3-a5cf-4d32a52458c8\") " pod="openshift-ingress/e2e-mt-57b5abb4-openshift-default-58ccbd7fff-gg65j" Jul 06 22:36:48.612671 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:36:48.612570 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f310a441-d7a1-44f3-a5cf-4d32a52458c8-istio-data\") pod \"e2e-mt-57b5abb4-openshift-default-58ccbd7fff-gg65j\" (UID: \"f310a441-d7a1-44f3-a5cf-4d32a52458c8\") " pod="openshift-ingress/e2e-mt-57b5abb4-openshift-default-58ccbd7fff-gg65j" Jul 06 22:36:48.612671 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:36:48.612598 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f310a441-d7a1-44f3-a5cf-4d32a52458c8-credential-socket\") pod \"e2e-mt-57b5abb4-openshift-default-58ccbd7fff-gg65j\" (UID: \"f310a441-d7a1-44f3-a5cf-4d32a52458c8\") " pod="openshift-ingress/e2e-mt-57b5abb4-openshift-default-58ccbd7fff-gg65j" Jul 06 22:36:48.612671 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:36:48.612622 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f310a441-d7a1-44f3-a5cf-4d32a52458c8-istio-envoy\") pod \"e2e-mt-57b5abb4-openshift-default-58ccbd7fff-gg65j\" (UID: \"f310a441-d7a1-44f3-a5cf-4d32a52458c8\") " pod="openshift-ingress/e2e-mt-57b5abb4-openshift-default-58ccbd7fff-gg65j" Jul 06 22:36:48.612815 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:36:48.612672 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f310a441-d7a1-44f3-a5cf-4d32a52458c8-istio-podinfo\") pod \"e2e-mt-57b5abb4-openshift-default-58ccbd7fff-gg65j\" (UID: \"f310a441-d7a1-44f3-a5cf-4d32a52458c8\") " pod="openshift-ingress/e2e-mt-57b5abb4-openshift-default-58ccbd7fff-gg65j" Jul 06 22:36:48.612815 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:36:48.612707 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f310a441-d7a1-44f3-a5cf-4d32a52458c8-istiod-ca-cert\") pod \"e2e-mt-57b5abb4-openshift-default-58ccbd7fff-gg65j\" (UID: \"f310a441-d7a1-44f3-a5cf-4d32a52458c8\") " pod="openshift-ingress/e2e-mt-57b5abb4-openshift-default-58ccbd7fff-gg65j" Jul 06 22:36:48.612815 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:36:48.612730 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4997q\" (UniqueName: \"kubernetes.io/projected/f310a441-d7a1-44f3-a5cf-4d32a52458c8-kube-api-access-4997q\") pod \"e2e-mt-57b5abb4-openshift-default-58ccbd7fff-gg65j\" (UID: \"f310a441-d7a1-44f3-a5cf-4d32a52458c8\") " pod="openshift-ingress/e2e-mt-57b5abb4-openshift-default-58ccbd7fff-gg65j" Jul 06 22:36:48.612815 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:36:48.612794 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f310a441-d7a1-44f3-a5cf-4d32a52458c8-istio-token\") pod \"e2e-mt-57b5abb4-openshift-default-58ccbd7fff-gg65j\" (UID: \"f310a441-d7a1-44f3-a5cf-4d32a52458c8\") " pod="openshift-ingress/e2e-mt-57b5abb4-openshift-default-58ccbd7fff-gg65j" Jul 06 22:36:48.713786 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:36:48.713740 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f310a441-d7a1-44f3-a5cf-4d32a52458c8-workload-socket\") pod \"e2e-mt-57b5abb4-openshift-default-58ccbd7fff-gg65j\" (UID: \"f310a441-d7a1-44f3-a5cf-4d32a52458c8\") " pod="openshift-ingress/e2e-mt-57b5abb4-openshift-default-58ccbd7fff-gg65j" Jul 06 22:36:48.713980 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:36:48.713799 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f310a441-d7a1-44f3-a5cf-4d32a52458c8-workload-certs\") pod \"e2e-mt-57b5abb4-openshift-default-58ccbd7fff-gg65j\" (UID: \"f310a441-d7a1-44f3-a5cf-4d32a52458c8\") " pod="openshift-ingress/e2e-mt-57b5abb4-openshift-default-58ccbd7fff-gg65j" Jul 06 22:36:48.713980 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:36:48.713828 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f310a441-d7a1-44f3-a5cf-4d32a52458c8-istio-data\") pod \"e2e-mt-57b5abb4-openshift-default-58ccbd7fff-gg65j\" (UID: \"f310a441-d7a1-44f3-a5cf-4d32a52458c8\") " pod="openshift-ingress/e2e-mt-57b5abb4-openshift-default-58ccbd7fff-gg65j" Jul 06 22:36:48.713980 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:36:48.713850 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f310a441-d7a1-44f3-a5cf-4d32a52458c8-credential-socket\") pod \"e2e-mt-57b5abb4-openshift-default-58ccbd7fff-gg65j\" (UID: \"f310a441-d7a1-44f3-a5cf-4d32a52458c8\") " pod="openshift-ingress/e2e-mt-57b5abb4-openshift-default-58ccbd7fff-gg65j" Jul 06 22:36:48.713980 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:36:48.713865 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f310a441-d7a1-44f3-a5cf-4d32a52458c8-istio-envoy\") pod \"e2e-mt-57b5abb4-openshift-default-58ccbd7fff-gg65j\" (UID: \"f310a441-d7a1-44f3-a5cf-4d32a52458c8\") " pod="openshift-ingress/e2e-mt-57b5abb4-openshift-default-58ccbd7fff-gg65j" Jul 06 22:36:48.713980 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:36:48.713891 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f310a441-d7a1-44f3-a5cf-4d32a52458c8-istio-podinfo\") pod \"e2e-mt-57b5abb4-openshift-default-58ccbd7fff-gg65j\" (UID: \"f310a441-d7a1-44f3-a5cf-4d32a52458c8\") " pod="openshift-ingress/e2e-mt-57b5abb4-openshift-default-58ccbd7fff-gg65j" Jul 06 22:36:48.713980 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:36:48.713929 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f310a441-d7a1-44f3-a5cf-4d32a52458c8-istiod-ca-cert\") pod \"e2e-mt-57b5abb4-openshift-default-58ccbd7fff-gg65j\" (UID: \"f310a441-d7a1-44f3-a5cf-4d32a52458c8\") " pod="openshift-ingress/e2e-mt-57b5abb4-openshift-default-58ccbd7fff-gg65j" Jul 06 22:36:48.713980 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:36:48.713955 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4997q\" (UniqueName: \"kubernetes.io/projected/f310a441-d7a1-44f3-a5cf-4d32a52458c8-kube-api-access-4997q\") pod \"e2e-mt-57b5abb4-openshift-default-58ccbd7fff-gg65j\" (UID: \"f310a441-d7a1-44f3-a5cf-4d32a52458c8\") " pod="openshift-ingress/e2e-mt-57b5abb4-openshift-default-58ccbd7fff-gg65j" Jul 06 22:36:48.714393 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:36:48.713996 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f310a441-d7a1-44f3-a5cf-4d32a52458c8-istio-token\") pod \"e2e-mt-57b5abb4-openshift-default-58ccbd7fff-gg65j\" (UID: \"f310a441-d7a1-44f3-a5cf-4d32a52458c8\") " pod="openshift-ingress/e2e-mt-57b5abb4-openshift-default-58ccbd7fff-gg65j" Jul 06 22:36:48.714393 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:36:48.714251 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f310a441-d7a1-44f3-a5cf-4d32a52458c8-workload-socket\") pod \"e2e-mt-57b5abb4-openshift-default-58ccbd7fff-gg65j\" (UID: \"f310a441-d7a1-44f3-a5cf-4d32a52458c8\") " pod="openshift-ingress/e2e-mt-57b5abb4-openshift-default-58ccbd7fff-gg65j" Jul 06 22:36:48.714393 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:36:48.714342 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f310a441-d7a1-44f3-a5cf-4d32a52458c8-credential-socket\") pod \"e2e-mt-57b5abb4-openshift-default-58ccbd7fff-gg65j\" (UID: \"f310a441-d7a1-44f3-a5cf-4d32a52458c8\") " pod="openshift-ingress/e2e-mt-57b5abb4-openshift-default-58ccbd7fff-gg65j" Jul 06 22:36:48.714552 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:36:48.714395 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f310a441-d7a1-44f3-a5cf-4d32a52458c8-istio-data\") pod \"e2e-mt-57b5abb4-openshift-default-58ccbd7fff-gg65j\" (UID: \"f310a441-d7a1-44f3-a5cf-4d32a52458c8\") " pod="openshift-ingress/e2e-mt-57b5abb4-openshift-default-58ccbd7fff-gg65j" Jul 06 22:36:48.714655 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:36:48.714634 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f310a441-d7a1-44f3-a5cf-4d32a52458c8-workload-certs\") pod \"e2e-mt-57b5abb4-openshift-default-58ccbd7fff-gg65j\" (UID: \"f310a441-d7a1-44f3-a5cf-4d32a52458c8\") " pod="openshift-ingress/e2e-mt-57b5abb4-openshift-default-58ccbd7fff-gg65j" Jul 06 22:36:48.714918 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:36:48.714898 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f310a441-d7a1-44f3-a5cf-4d32a52458c8-istiod-ca-cert\") pod \"e2e-mt-57b5abb4-openshift-default-58ccbd7fff-gg65j\" (UID: \"f310a441-d7a1-44f3-a5cf-4d32a52458c8\") " pod="openshift-ingress/e2e-mt-57b5abb4-openshift-default-58ccbd7fff-gg65j" Jul 06 22:36:48.716290 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:36:48.716266 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f310a441-d7a1-44f3-a5cf-4d32a52458c8-istio-envoy\") pod \"e2e-mt-57b5abb4-openshift-default-58ccbd7fff-gg65j\" (UID: \"f310a441-d7a1-44f3-a5cf-4d32a52458c8\") " pod="openshift-ingress/e2e-mt-57b5abb4-openshift-default-58ccbd7fff-gg65j" Jul 06 22:36:48.716813 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:36:48.716795 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f310a441-d7a1-44f3-a5cf-4d32a52458c8-istio-podinfo\") pod \"e2e-mt-57b5abb4-openshift-default-58ccbd7fff-gg65j\" (UID: \"f310a441-d7a1-44f3-a5cf-4d32a52458c8\") " pod="openshift-ingress/e2e-mt-57b5abb4-openshift-default-58ccbd7fff-gg65j" Jul 06 22:36:48.723438 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:36:48.723405 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4997q\" (UniqueName: \"kubernetes.io/projected/f310a441-d7a1-44f3-a5cf-4d32a52458c8-kube-api-access-4997q\") pod \"e2e-mt-57b5abb4-openshift-default-58ccbd7fff-gg65j\" (UID: \"f310a441-d7a1-44f3-a5cf-4d32a52458c8\") " pod="openshift-ingress/e2e-mt-57b5abb4-openshift-default-58ccbd7fff-gg65j" Jul 06 22:36:48.724290 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:36:48.724270 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f310a441-d7a1-44f3-a5cf-4d32a52458c8-istio-token\") pod \"e2e-mt-57b5abb4-openshift-default-58ccbd7fff-gg65j\" (UID: \"f310a441-d7a1-44f3-a5cf-4d32a52458c8\") " pod="openshift-ingress/e2e-mt-57b5abb4-openshift-default-58ccbd7fff-gg65j" Jul 06 22:36:48.790568 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:36:48.790535 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:36:48.856937 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:36:48.856849 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-57b5abb4-openshift-default-58ccbd7fff-gg65j" Jul 06 22:36:48.998611 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:36:48.998534 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-57b5abb4-openshift-default-58ccbd7fff-gg65j"] Jul 06 22:36:49.000865 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:36:49.000838 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf310a441_d7a1_44f3_a5cf_4d32a52458c8.slice/crio-c4d2e5ed177eb71ce0cf1a5c13ca23385b99d3752d8b3479eba8bd14bfaa9b85 WatchSource:0}: Error finding container c4d2e5ed177eb71ce0cf1a5c13ca23385b99d3752d8b3479eba8bd14bfaa9b85: Status 404 returned error can't find the container with id c4d2e5ed177eb71ce0cf1a5c13ca23385b99d3752d8b3479eba8bd14bfaa9b85 Jul 06 22:36:49.002977 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:36:49.002945 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 22:36:49.003061 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:36:49.003014 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 22:36:49.003061 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:36:49.003043 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 22:36:49.398691 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:36:49.398654 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-57b5abb4-openshift-default-58ccbd7fff-gg65j" event={"ID":"f310a441-d7a1-44f3-a5cf-4d32a52458c8","Type":"ContainerStarted","Data":"3cdd34b5076c960e622262a636b1b439defb743c1b1881611313ff4270faccdf"} Jul 06 22:36:49.399109 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:36:49.398707 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-57b5abb4-openshift-default-58ccbd7fff-gg65j" event={"ID":"f310a441-d7a1-44f3-a5cf-4d32a52458c8","Type":"ContainerStarted","Data":"c4d2e5ed177eb71ce0cf1a5c13ca23385b99d3752d8b3479eba8bd14bfaa9b85"} Jul 06 22:36:49.418663 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:36:49.418603 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-57b5abb4-openshift-default-58ccbd7fff-gg65j" podStartSLOduration=1.418586672 podStartE2EDuration="1.418586672s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:36:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:36:49.417405747 +0000 UTC m=+1688.976201068" watchObservedRunningTime="2026-07-06 22:36:49.418586672 +0000 UTC m=+1688.977382005" Jul 06 22:36:49.857611 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:36:49.857576 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-57b5abb4-openshift-default-58ccbd7fff-gg65j" Jul 06 22:36:50.862519 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:36:50.862489 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-57b5abb4-openshift-default-58ccbd7fff-gg65j" Jul 06 22:36:51.351754 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:36:51.351715 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:36:51.412086 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:36:51.412047 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-57b5abb4-openshift-default-58ccbd7fff-gg65j" Jul 06 22:36:51.413121 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:36:51.413101 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-57b5abb4-openshift-default-58ccbd7fff-gg65j" Jul 06 22:36:51.437925 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:36:51.437841 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:36:51.642165 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:36:51.642047 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:36:52.077078 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:36:52.077025 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:36:52.178307 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:36:52.178276 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:36:52.241626 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:36:52.241586 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:36:55.861333 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:36:55.861301 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:36:56.535378 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:36:56.535346 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:36:56.633152 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:36:56.633114 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:36:56.986730 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:36:56.986697 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:37:02.826707 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:02.826674 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:37:02.891427 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:02.891388 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:37:03.288712 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:03.288679 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:37:03.738543 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:03.738510 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:37:03.839331 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:03.839295 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:37:04.487961 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:04.487933 2581 generic.go:356] "Generic (PLEG): container finished" podID="f310a441-d7a1-44f3-a5cf-4d32a52458c8" containerID="3cdd34b5076c960e622262a636b1b439defb743c1b1881611313ff4270faccdf" exitCode=0 Jul 06 22:37:04.488130 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:04.487979 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-57b5abb4-openshift-default-58ccbd7fff-gg65j" event={"ID":"f310a441-d7a1-44f3-a5cf-4d32a52458c8","Type":"ContainerDied","Data":"3cdd34b5076c960e622262a636b1b439defb743c1b1881611313ff4270faccdf"} Jul 06 22:37:04.488382 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:04.488369 2581 scope.go:117] "RemoveContainer" containerID="3cdd34b5076c960e622262a636b1b439defb743c1b1881611313ff4270faccdf" Jul 06 22:37:04.489371 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:04.489338 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 22:37:04.489479 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:04.489453 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 22:37:04.489595 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:04.489530 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 22:37:05.495395 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:05.495359 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-57b5abb4-openshift-default-58ccbd7fff-gg65j" event={"ID":"f310a441-d7a1-44f3-a5cf-4d32a52458c8","Type":"ContainerStarted","Data":"05b2768dc1d93546beb8f9903759f54b7400de299166748906124fa79558ff82"} Jul 06 22:37:05.857816 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:05.857712 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-57b5abb4-openshift-default-58ccbd7fff-gg65j" Jul 06 22:37:05.859019 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:05.858981 2581 patch_prober.go:28] interesting pod/e2e-mt-57b5abb4-openshift-default-58ccbd7fff-gg65j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body= Jul 06 22:37:05.859164 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:05.859046 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-57b5abb4-openshift-default-58ccbd7fff-gg65j" podUID="f310a441-d7a1-44f3-a5cf-4d32a52458c8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" Jul 06 22:37:06.412177 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:06.412138 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-57b5abb4-openshift-default-58ccbd7fff-gg65j" Jul 06 22:37:06.858161 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:06.858125 2581 patch_prober.go:28] interesting pod/e2e-mt-57b5abb4-openshift-default-58ccbd7fff-gg65j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body= Jul 06 22:37:06.858756 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:06.858711 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-57b5abb4-openshift-default-58ccbd7fff-gg65j" podUID="f310a441-d7a1-44f3-a5cf-4d32a52458c8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" Jul 06 22:37:07.858189 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:07.857759 2581 patch_prober.go:28] interesting pod/e2e-mt-57b5abb4-openshift-default-58ccbd7fff-gg65j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body= Jul 06 22:37:07.858189 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:07.857819 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-57b5abb4-openshift-default-58ccbd7fff-gg65j" podUID="f310a441-d7a1-44f3-a5cf-4d32a52458c8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" Jul 06 22:37:08.857608 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:08.857183 2581 patch_prober.go:28] interesting pod/e2e-mt-57b5abb4-openshift-default-58ccbd7fff-gg65j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body= Jul 06 22:37:08.857608 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:08.857247 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-57b5abb4-openshift-default-58ccbd7fff-gg65j" podUID="f310a441-d7a1-44f3-a5cf-4d32a52458c8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" Jul 06 22:37:09.582298 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:09.582266 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:37:09.682000 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:09.681949 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:37:09.838976 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:09.838886 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-57b5abb4-openshift-default-58ccbd7fff-gg65j"] Jul 06 22:37:09.839168 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:09.839085 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-57b5abb4-openshift-default-58ccbd7fff-gg65j" podUID="f310a441-d7a1-44f3-a5cf-4d32a52458c8" containerName="istio-proxy" containerID="cri-o://05b2768dc1d93546beb8f9903759f54b7400de299166748906124fa79558ff82" gracePeriod=30 Jul 06 22:37:09.856426 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:09.856392 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:37:10.085941 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:10.085907 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:37:10.226054 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:10.226021 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:37:10.318113 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:10.318053 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:37:10.440747 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:10.440720 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:37:15.108713 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:15.108653 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-57b5abb4-openshift-default-58ccbd7fff-gg65j" Jul 06 22:37:15.271186 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:15.271151 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f310a441-d7a1-44f3-a5cf-4d32a52458c8-istio-envoy\") pod \"f310a441-d7a1-44f3-a5cf-4d32a52458c8\" (UID: \"f310a441-d7a1-44f3-a5cf-4d32a52458c8\") " Jul 06 22:37:15.271186 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:15.271191 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f310a441-d7a1-44f3-a5cf-4d32a52458c8-istio-data\") pod \"f310a441-d7a1-44f3-a5cf-4d32a52458c8\" (UID: \"f310a441-d7a1-44f3-a5cf-4d32a52458c8\") " Jul 06 22:37:15.271453 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:15.271206 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f310a441-d7a1-44f3-a5cf-4d32a52458c8-credential-socket\") pod \"f310a441-d7a1-44f3-a5cf-4d32a52458c8\" (UID: \"f310a441-d7a1-44f3-a5cf-4d32a52458c8\") " Jul 06 22:37:15.271453 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:15.271234 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4997q\" (UniqueName: \"kubernetes.io/projected/f310a441-d7a1-44f3-a5cf-4d32a52458c8-kube-api-access-4997q\") pod \"f310a441-d7a1-44f3-a5cf-4d32a52458c8\" (UID: \"f310a441-d7a1-44f3-a5cf-4d32a52458c8\") " Jul 06 22:37:15.271453 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:15.271266 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f310a441-d7a1-44f3-a5cf-4d32a52458c8-istiod-ca-cert\") pod \"f310a441-d7a1-44f3-a5cf-4d32a52458c8\" (UID: \"f310a441-d7a1-44f3-a5cf-4d32a52458c8\") " Jul 06 22:37:15.271453 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:15.271380 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f310a441-d7a1-44f3-a5cf-4d32a52458c8-istio-token\") pod \"f310a441-d7a1-44f3-a5cf-4d32a52458c8\" (UID: \"f310a441-d7a1-44f3-a5cf-4d32a52458c8\") " Jul 06 22:37:15.271453 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:15.271414 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f310a441-d7a1-44f3-a5cf-4d32a52458c8-workload-certs\") pod \"f310a441-d7a1-44f3-a5cf-4d32a52458c8\" (UID: \"f310a441-d7a1-44f3-a5cf-4d32a52458c8\") " Jul 06 22:37:15.271453 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:15.271440 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f310a441-d7a1-44f3-a5cf-4d32a52458c8-istio-podinfo\") pod \"f310a441-d7a1-44f3-a5cf-4d32a52458c8\" (UID: \"f310a441-d7a1-44f3-a5cf-4d32a52458c8\") " Jul 06 22:37:15.271746 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:15.271470 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f310a441-d7a1-44f3-a5cf-4d32a52458c8-workload-socket\") pod \"f310a441-d7a1-44f3-a5cf-4d32a52458c8\" (UID: \"f310a441-d7a1-44f3-a5cf-4d32a52458c8\") " Jul 06 22:37:15.271746 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:15.271540 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f310a441-d7a1-44f3-a5cf-4d32a52458c8-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f310a441-d7a1-44f3-a5cf-4d32a52458c8" (UID: "f310a441-d7a1-44f3-a5cf-4d32a52458c8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:37:15.271849 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:15.271763 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f310a441-d7a1-44f3-a5cf-4d32a52458c8-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f310a441-d7a1-44f3-a5cf-4d32a52458c8" (UID: "f310a441-d7a1-44f3-a5cf-4d32a52458c8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:37:15.271849 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:15.271841 2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f310a441-d7a1-44f3-a5cf-4d32a52458c8-workload-socket\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:37:15.271958 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:15.271862 2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f310a441-d7a1-44f3-a5cf-4d32a52458c8-credential-socket\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:37:15.272015 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:15.271967 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f310a441-d7a1-44f3-a5cf-4d32a52458c8-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f310a441-d7a1-44f3-a5cf-4d32a52458c8" (UID: "f310a441-d7a1-44f3-a5cf-4d32a52458c8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 22:37:15.272015 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:15.271967 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f310a441-d7a1-44f3-a5cf-4d32a52458c8-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f310a441-d7a1-44f3-a5cf-4d32a52458c8" (UID: "f310a441-d7a1-44f3-a5cf-4d32a52458c8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:37:15.272138 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:15.272047 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f310a441-d7a1-44f3-a5cf-4d32a52458c8-istio-data" (OuterVolumeSpecName: "istio-data") pod "f310a441-d7a1-44f3-a5cf-4d32a52458c8" (UID: "f310a441-d7a1-44f3-a5cf-4d32a52458c8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:37:15.273542 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:15.273508 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f310a441-d7a1-44f3-a5cf-4d32a52458c8-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f310a441-d7a1-44f3-a5cf-4d32a52458c8" (UID: "f310a441-d7a1-44f3-a5cf-4d32a52458c8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:37:15.273659 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:15.273564 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f310a441-d7a1-44f3-a5cf-4d32a52458c8-istio-token" (OuterVolumeSpecName: "istio-token") pod "f310a441-d7a1-44f3-a5cf-4d32a52458c8" (UID: "f310a441-d7a1-44f3-a5cf-4d32a52458c8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 22:37:15.273659 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:15.273612 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f310a441-d7a1-44f3-a5cf-4d32a52458c8-kube-api-access-4997q" (OuterVolumeSpecName: "kube-api-access-4997q") pod "f310a441-d7a1-44f3-a5cf-4d32a52458c8" (UID: "f310a441-d7a1-44f3-a5cf-4d32a52458c8"). InnerVolumeSpecName "kube-api-access-4997q". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 22:37:15.273659 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:15.273644 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f310a441-d7a1-44f3-a5cf-4d32a52458c8-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f310a441-d7a1-44f3-a5cf-4d32a52458c8" (UID: "f310a441-d7a1-44f3-a5cf-4d32a52458c8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 22:37:15.373175 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:15.373095 2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f310a441-d7a1-44f3-a5cf-4d32a52458c8-istio-token\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:37:15.373175 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:15.373122 2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f310a441-d7a1-44f3-a5cf-4d32a52458c8-workload-certs\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:37:15.373175 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:15.373132 2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f310a441-d7a1-44f3-a5cf-4d32a52458c8-istio-podinfo\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:37:15.373175 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:15.373141 2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f310a441-d7a1-44f3-a5cf-4d32a52458c8-istio-envoy\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:37:15.373175 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:15.373149 2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f310a441-d7a1-44f3-a5cf-4d32a52458c8-istio-data\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:37:15.373175 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:15.373158 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4997q\" (UniqueName: \"kubernetes.io/projected/f310a441-d7a1-44f3-a5cf-4d32a52458c8-kube-api-access-4997q\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:37:15.373175 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:15.373166 2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f310a441-d7a1-44f3-a5cf-4d32a52458c8-istiod-ca-cert\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:37:15.544481 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:15.544445 2581 generic.go:356] "Generic (PLEG): container finished" podID="f310a441-d7a1-44f3-a5cf-4d32a52458c8" containerID="05b2768dc1d93546beb8f9903759f54b7400de299166748906124fa79558ff82" exitCode=0 Jul 06 22:37:15.544669 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:15.544489 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-57b5abb4-openshift-default-58ccbd7fff-gg65j" Jul 06 22:37:15.544669 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:15.544539 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-57b5abb4-openshift-default-58ccbd7fff-gg65j" event={"ID":"f310a441-d7a1-44f3-a5cf-4d32a52458c8","Type":"ContainerDied","Data":"05b2768dc1d93546beb8f9903759f54b7400de299166748906124fa79558ff82"} Jul 06 22:37:15.544669 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:15.544589 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-57b5abb4-openshift-default-58ccbd7fff-gg65j" event={"ID":"f310a441-d7a1-44f3-a5cf-4d32a52458c8","Type":"ContainerDied","Data":"c4d2e5ed177eb71ce0cf1a5c13ca23385b99d3752d8b3479eba8bd14bfaa9b85"} Jul 06 22:37:15.544669 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:15.544612 2581 scope.go:117] "RemoveContainer" containerID="05b2768dc1d93546beb8f9903759f54b7400de299166748906124fa79558ff82" Jul 06 22:37:15.554469 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:15.554452 2581 scope.go:117] "RemoveContainer" containerID="3cdd34b5076c960e622262a636b1b439defb743c1b1881611313ff4270faccdf" Jul 06 22:37:15.562138 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:15.562121 2581 scope.go:117] "RemoveContainer" containerID="05b2768dc1d93546beb8f9903759f54b7400de299166748906124fa79558ff82" Jul 06 22:37:15.562391 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:37:15.562366 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"05b2768dc1d93546beb8f9903759f54b7400de299166748906124fa79558ff82\": container with ID starting with 05b2768dc1d93546beb8f9903759f54b7400de299166748906124fa79558ff82 not found: ID does not exist" containerID="05b2768dc1d93546beb8f9903759f54b7400de299166748906124fa79558ff82" Jul 06 22:37:15.562439 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:15.562403 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"05b2768dc1d93546beb8f9903759f54b7400de299166748906124fa79558ff82"} err="failed to get container status \"05b2768dc1d93546beb8f9903759f54b7400de299166748906124fa79558ff82\": rpc error: code = NotFound desc = could not find container \"05b2768dc1d93546beb8f9903759f54b7400de299166748906124fa79558ff82\": container with ID starting with 05b2768dc1d93546beb8f9903759f54b7400de299166748906124fa79558ff82 not found: ID does not exist" Jul 06 22:37:15.562439 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:15.562425 2581 scope.go:117] "RemoveContainer" containerID="3cdd34b5076c960e622262a636b1b439defb743c1b1881611313ff4270faccdf" Jul 06 22:37:15.562651 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:37:15.562631 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3cdd34b5076c960e622262a636b1b439defb743c1b1881611313ff4270faccdf\": container with ID starting with 3cdd34b5076c960e622262a636b1b439defb743c1b1881611313ff4270faccdf not found: ID does not exist" containerID="3cdd34b5076c960e622262a636b1b439defb743c1b1881611313ff4270faccdf" Jul 06 22:37:15.562693 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:15.562658 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3cdd34b5076c960e622262a636b1b439defb743c1b1881611313ff4270faccdf"} err="failed to get container status \"3cdd34b5076c960e622262a636b1b439defb743c1b1881611313ff4270faccdf\": rpc error: code = NotFound desc = could not find container \"3cdd34b5076c960e622262a636b1b439defb743c1b1881611313ff4270faccdf\": container with ID starting with 3cdd34b5076c960e622262a636b1b439defb743c1b1881611313ff4270faccdf not found: ID does not exist" Jul 06 22:37:15.567976 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:15.567955 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-57b5abb4-openshift-default-58ccbd7fff-gg65j"] Jul 06 22:37:15.571322 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:15.571303 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-57b5abb4-openshift-default-58ccbd7fff-gg65j"] Jul 06 22:37:16.314060 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:16.314032 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:37:16.482125 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:16.482092 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:37:16.605843 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:16.605758 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:37:16.713322 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:16.713294 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:37:16.976522 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:16.976484 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f310a441-d7a1-44f3-a5cf-4d32a52458c8" path="/var/lib/kubelet/pods/f310a441-d7a1-44f3-a5cf-4d32a52458c8/volumes" Jul 06 22:37:22.506826 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:22.506791 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:37:22.606888 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:22.606845 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:37:29.440378 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:29.440346 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:37:29.745720 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:29.745636 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-b6d6e159-openshift-default-754cc48dd7-qdnpq"] Jul 06 22:37:29.746256 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:29.746233 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f310a441-d7a1-44f3-a5cf-4d32a52458c8" containerName="istio-proxy" Jul 06 22:37:29.746415 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:29.746285 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="f310a441-d7a1-44f3-a5cf-4d32a52458c8" containerName="istio-proxy" Jul 06 22:37:29.746415 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:29.746339 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f310a441-d7a1-44f3-a5cf-4d32a52458c8" containerName="istio-proxy" Jul 06 22:37:29.746415 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:29.746350 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="f310a441-d7a1-44f3-a5cf-4d32a52458c8" containerName="istio-proxy" Jul 06 22:37:29.746595 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:29.746493 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="f310a441-d7a1-44f3-a5cf-4d32a52458c8" containerName="istio-proxy" Jul 06 22:37:29.746595 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:29.746517 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="f310a441-d7a1-44f3-a5cf-4d32a52458c8" containerName="istio-proxy" Jul 06 22:37:29.750725 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:29.750706 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-b6d6e159-openshift-default-754cc48dd7-qdnpq" Jul 06 22:37:29.753910 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:29.753888 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-b6d6e159-openshift-default-dockercfg-56fkt\"" Jul 06 22:37:29.761906 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:29.761884 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-b6d6e159-openshift-default-754cc48dd7-qdnpq"] Jul 06 22:37:29.776530 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:29.776478 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:37:29.796959 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:29.796929 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-swb2c\" (UniqueName: \"kubernetes.io/projected/eecc11c5-ab05-4449-85ae-cd684f2774af-kube-api-access-swb2c\") pod \"e2e-mt-b6d6e159-openshift-default-754cc48dd7-qdnpq\" (UID: \"eecc11c5-ab05-4449-85ae-cd684f2774af\") " pod="openshift-ingress/e2e-mt-b6d6e159-openshift-default-754cc48dd7-qdnpq" Jul 06 22:37:29.797122 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:29.796992 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eecc11c5-ab05-4449-85ae-cd684f2774af-istiod-ca-cert\") pod \"e2e-mt-b6d6e159-openshift-default-754cc48dd7-qdnpq\" (UID: \"eecc11c5-ab05-4449-85ae-cd684f2774af\") " pod="openshift-ingress/e2e-mt-b6d6e159-openshift-default-754cc48dd7-qdnpq" Jul 06 22:37:29.797122 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:29.797087 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eecc11c5-ab05-4449-85ae-cd684f2774af-workload-certs\") pod \"e2e-mt-b6d6e159-openshift-default-754cc48dd7-qdnpq\" (UID: \"eecc11c5-ab05-4449-85ae-cd684f2774af\") " pod="openshift-ingress/e2e-mt-b6d6e159-openshift-default-754cc48dd7-qdnpq" Jul 06 22:37:29.797251 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:29.797132 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eecc11c5-ab05-4449-85ae-cd684f2774af-workload-socket\") pod \"e2e-mt-b6d6e159-openshift-default-754cc48dd7-qdnpq\" (UID: \"eecc11c5-ab05-4449-85ae-cd684f2774af\") " pod="openshift-ingress/e2e-mt-b6d6e159-openshift-default-754cc48dd7-qdnpq" Jul 06 22:37:29.797251 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:29.797158 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eecc11c5-ab05-4449-85ae-cd684f2774af-istio-data\") pod \"e2e-mt-b6d6e159-openshift-default-754cc48dd7-qdnpq\" (UID: \"eecc11c5-ab05-4449-85ae-cd684f2774af\") " pod="openshift-ingress/e2e-mt-b6d6e159-openshift-default-754cc48dd7-qdnpq" Jul 06 22:37:29.797251 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:29.797202 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eecc11c5-ab05-4449-85ae-cd684f2774af-credential-socket\") pod \"e2e-mt-b6d6e159-openshift-default-754cc48dd7-qdnpq\" (UID: \"eecc11c5-ab05-4449-85ae-cd684f2774af\") " pod="openshift-ingress/e2e-mt-b6d6e159-openshift-default-754cc48dd7-qdnpq" Jul 06 22:37:29.797251 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:29.797233 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eecc11c5-ab05-4449-85ae-cd684f2774af-istio-podinfo\") pod \"e2e-mt-b6d6e159-openshift-default-754cc48dd7-qdnpq\" (UID: \"eecc11c5-ab05-4449-85ae-cd684f2774af\") " pod="openshift-ingress/e2e-mt-b6d6e159-openshift-default-754cc48dd7-qdnpq" Jul 06 22:37:29.797457 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:29.797266 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eecc11c5-ab05-4449-85ae-cd684f2774af-istio-envoy\") pod \"e2e-mt-b6d6e159-openshift-default-754cc48dd7-qdnpq\" (UID: \"eecc11c5-ab05-4449-85ae-cd684f2774af\") " pod="openshift-ingress/e2e-mt-b6d6e159-openshift-default-754cc48dd7-qdnpq" Jul 06 22:37:29.797457 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:29.797371 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eecc11c5-ab05-4449-85ae-cd684f2774af-istio-token\") pod \"e2e-mt-b6d6e159-openshift-default-754cc48dd7-qdnpq\" (UID: \"eecc11c5-ab05-4449-85ae-cd684f2774af\") " pod="openshift-ingress/e2e-mt-b6d6e159-openshift-default-754cc48dd7-qdnpq" Jul 06 22:37:29.898189 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:29.898152 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eecc11c5-ab05-4449-85ae-cd684f2774af-istio-token\") pod \"e2e-mt-b6d6e159-openshift-default-754cc48dd7-qdnpq\" (UID: \"eecc11c5-ab05-4449-85ae-cd684f2774af\") " pod="openshift-ingress/e2e-mt-b6d6e159-openshift-default-754cc48dd7-qdnpq" Jul 06 22:37:29.898396 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:29.898208 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-swb2c\" (UniqueName: \"kubernetes.io/projected/eecc11c5-ab05-4449-85ae-cd684f2774af-kube-api-access-swb2c\") pod \"e2e-mt-b6d6e159-openshift-default-754cc48dd7-qdnpq\" (UID: \"eecc11c5-ab05-4449-85ae-cd684f2774af\") " pod="openshift-ingress/e2e-mt-b6d6e159-openshift-default-754cc48dd7-qdnpq" Jul 06 22:37:29.898396 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:29.898247 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eecc11c5-ab05-4449-85ae-cd684f2774af-istiod-ca-cert\") pod \"e2e-mt-b6d6e159-openshift-default-754cc48dd7-qdnpq\" (UID: \"eecc11c5-ab05-4449-85ae-cd684f2774af\") " pod="openshift-ingress/e2e-mt-b6d6e159-openshift-default-754cc48dd7-qdnpq" Jul 06 22:37:29.898396 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:29.898274 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eecc11c5-ab05-4449-85ae-cd684f2774af-workload-certs\") pod \"e2e-mt-b6d6e159-openshift-default-754cc48dd7-qdnpq\" (UID: \"eecc11c5-ab05-4449-85ae-cd684f2774af\") " pod="openshift-ingress/e2e-mt-b6d6e159-openshift-default-754cc48dd7-qdnpq" Jul 06 22:37:29.898396 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:29.898296 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eecc11c5-ab05-4449-85ae-cd684f2774af-workload-socket\") pod \"e2e-mt-b6d6e159-openshift-default-754cc48dd7-qdnpq\" (UID: \"eecc11c5-ab05-4449-85ae-cd684f2774af\") " pod="openshift-ingress/e2e-mt-b6d6e159-openshift-default-754cc48dd7-qdnpq" Jul 06 22:37:29.898965 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:29.898461 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eecc11c5-ab05-4449-85ae-cd684f2774af-istio-data\") pod \"e2e-mt-b6d6e159-openshift-default-754cc48dd7-qdnpq\" (UID: \"eecc11c5-ab05-4449-85ae-cd684f2774af\") " pod="openshift-ingress/e2e-mt-b6d6e159-openshift-default-754cc48dd7-qdnpq" Jul 06 22:37:29.898965 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:29.898520 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eecc11c5-ab05-4449-85ae-cd684f2774af-credential-socket\") pod \"e2e-mt-b6d6e159-openshift-default-754cc48dd7-qdnpq\" (UID: \"eecc11c5-ab05-4449-85ae-cd684f2774af\") " pod="openshift-ingress/e2e-mt-b6d6e159-openshift-default-754cc48dd7-qdnpq" Jul 06 22:37:29.898965 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:29.898559 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eecc11c5-ab05-4449-85ae-cd684f2774af-istio-podinfo\") pod \"e2e-mt-b6d6e159-openshift-default-754cc48dd7-qdnpq\" (UID: \"eecc11c5-ab05-4449-85ae-cd684f2774af\") " pod="openshift-ingress/e2e-mt-b6d6e159-openshift-default-754cc48dd7-qdnpq" Jul 06 22:37:29.898965 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:29.898589 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eecc11c5-ab05-4449-85ae-cd684f2774af-istio-envoy\") pod \"e2e-mt-b6d6e159-openshift-default-754cc48dd7-qdnpq\" (UID: \"eecc11c5-ab05-4449-85ae-cd684f2774af\") " pod="openshift-ingress/e2e-mt-b6d6e159-openshift-default-754cc48dd7-qdnpq" Jul 06 22:37:29.898965 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:29.898750 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eecc11c5-ab05-4449-85ae-cd684f2774af-workload-socket\") pod \"e2e-mt-b6d6e159-openshift-default-754cc48dd7-qdnpq\" (UID: \"eecc11c5-ab05-4449-85ae-cd684f2774af\") " pod="openshift-ingress/e2e-mt-b6d6e159-openshift-default-754cc48dd7-qdnpq" Jul 06 22:37:29.898965 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:29.898787 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eecc11c5-ab05-4449-85ae-cd684f2774af-workload-certs\") pod \"e2e-mt-b6d6e159-openshift-default-754cc48dd7-qdnpq\" (UID: \"eecc11c5-ab05-4449-85ae-cd684f2774af\") " pod="openshift-ingress/e2e-mt-b6d6e159-openshift-default-754cc48dd7-qdnpq" Jul 06 22:37:29.898965 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:29.898841 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eecc11c5-ab05-4449-85ae-cd684f2774af-istio-data\") pod \"e2e-mt-b6d6e159-openshift-default-754cc48dd7-qdnpq\" (UID: \"eecc11c5-ab05-4449-85ae-cd684f2774af\") " pod="openshift-ingress/e2e-mt-b6d6e159-openshift-default-754cc48dd7-qdnpq" Jul 06 22:37:29.899357 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:29.899011 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eecc11c5-ab05-4449-85ae-cd684f2774af-credential-socket\") pod \"e2e-mt-b6d6e159-openshift-default-754cc48dd7-qdnpq\" (UID: \"eecc11c5-ab05-4449-85ae-cd684f2774af\") " pod="openshift-ingress/e2e-mt-b6d6e159-openshift-default-754cc48dd7-qdnpq" Jul 06 22:37:29.899357 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:29.899059 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eecc11c5-ab05-4449-85ae-cd684f2774af-istiod-ca-cert\") pod \"e2e-mt-b6d6e159-openshift-default-754cc48dd7-qdnpq\" (UID: \"eecc11c5-ab05-4449-85ae-cd684f2774af\") " pod="openshift-ingress/e2e-mt-b6d6e159-openshift-default-754cc48dd7-qdnpq" Jul 06 22:37:29.900762 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:29.900743 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eecc11c5-ab05-4449-85ae-cd684f2774af-istio-envoy\") pod \"e2e-mt-b6d6e159-openshift-default-754cc48dd7-qdnpq\" (UID: \"eecc11c5-ab05-4449-85ae-cd684f2774af\") " pod="openshift-ingress/e2e-mt-b6d6e159-openshift-default-754cc48dd7-qdnpq" Jul 06 22:37:29.901039 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:29.901022 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eecc11c5-ab05-4449-85ae-cd684f2774af-istio-podinfo\") pod \"e2e-mt-b6d6e159-openshift-default-754cc48dd7-qdnpq\" (UID: \"eecc11c5-ab05-4449-85ae-cd684f2774af\") " pod="openshift-ingress/e2e-mt-b6d6e159-openshift-default-754cc48dd7-qdnpq" Jul 06 22:37:29.905898 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:29.905874 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-swb2c\" (UniqueName: \"kubernetes.io/projected/eecc11c5-ab05-4449-85ae-cd684f2774af-kube-api-access-swb2c\") pod \"e2e-mt-b6d6e159-openshift-default-754cc48dd7-qdnpq\" (UID: \"eecc11c5-ab05-4449-85ae-cd684f2774af\") " pod="openshift-ingress/e2e-mt-b6d6e159-openshift-default-754cc48dd7-qdnpq" Jul 06 22:37:29.906013 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:29.905926 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eecc11c5-ab05-4449-85ae-cd684f2774af-istio-token\") pod \"e2e-mt-b6d6e159-openshift-default-754cc48dd7-qdnpq\" (UID: \"eecc11c5-ab05-4449-85ae-cd684f2774af\") " pod="openshift-ingress/e2e-mt-b6d6e159-openshift-default-754cc48dd7-qdnpq" Jul 06 22:37:30.063341 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:30.063251 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-b6d6e159-openshift-default-754cc48dd7-qdnpq" Jul 06 22:37:30.121055 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:30.121000 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:37:30.207770 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:30.207744 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-b6d6e159-openshift-default-754cc48dd7-qdnpq"] Jul 06 22:37:30.210148 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:37:30.210126 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeecc11c5_ab05_4449_85ae_cd684f2774af.slice/crio-db584b4b25d23946486b9f8c695adbddc7720735d3f2e251c121f56870d7b0e3 WatchSource:0}: Error finding container db584b4b25d23946486b9f8c695adbddc7720735d3f2e251c121f56870d7b0e3: Status 404 returned error can't find the container with id db584b4b25d23946486b9f8c695adbddc7720735d3f2e251c121f56870d7b0e3 Jul 06 22:37:30.212277 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:30.212241 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 22:37:30.212368 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:30.212310 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 22:37:30.212368 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:30.212340 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 22:37:30.622644 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:30.622607 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-b6d6e159-openshift-default-754cc48dd7-qdnpq" event={"ID":"eecc11c5-ab05-4449-85ae-cd684f2774af","Type":"ContainerStarted","Data":"2c106871c0c2c575e87a7c2f336938281c83b52fdf253a5cd3d636036a8e5f22"} Jul 06 22:37:30.623037 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:30.622673 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-b6d6e159-openshift-default-754cc48dd7-qdnpq" event={"ID":"eecc11c5-ab05-4449-85ae-cd684f2774af","Type":"ContainerStarted","Data":"db584b4b25d23946486b9f8c695adbddc7720735d3f2e251c121f56870d7b0e3"} Jul 06 22:37:30.643366 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:30.643310 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-b6d6e159-openshift-default-754cc48dd7-qdnpq" podStartSLOduration=1.643289759 podStartE2EDuration="1.643289759s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:37:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:37:30.640472804 +0000 UTC m=+1730.199268125" watchObservedRunningTime="2026-07-06 22:37:30.643289759 +0000 UTC m=+1730.202085081" Jul 06 22:37:31.064199 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:31.064164 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-b6d6e159-openshift-default-754cc48dd7-qdnpq" Jul 06 22:37:31.068971 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:31.068941 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-b6d6e159-openshift-default-754cc48dd7-qdnpq" Jul 06 22:37:31.628184 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:31.628156 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-b6d6e159-openshift-default-754cc48dd7-qdnpq" Jul 06 22:37:31.628986 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:31.628967 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-b6d6e159-openshift-default-754cc48dd7-qdnpq" Jul 06 22:37:32.561572 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:32.561536 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:37:32.647327 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:32.647299 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:37:32.720430 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:32.720394 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:37:32.823120 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:32.823000 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:37:33.378022 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:33.377991 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:37:33.466965 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:33.466936 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:37:35.094215 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:35.094183 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:37:35.155297 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:35.155263 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:37:37.071048 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:37.071016 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:37:46.661513 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:46.659882 2581 patch_prober.go:28] interesting pod/e2e-mt-b6d6e159-openshift-default-754cc48dd7-qdnpq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": read tcp 10.132.0.2:60004->10.132.0.60:15021: read: connection reset by peer" start-of-body= Jul 06 22:37:46.661513 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:46.659979 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-b6d6e159-openshift-default-754cc48dd7-qdnpq" podUID="eecc11c5-ab05-4449-85ae-cd684f2774af" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": read tcp 10.132.0.2:60004->10.132.0.60:15021: read: connection reset by peer" Jul 06 22:37:46.708757 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:46.708729 2581 generic.go:356] "Generic (PLEG): container finished" podID="eecc11c5-ab05-4449-85ae-cd684f2774af" containerID="2c106871c0c2c575e87a7c2f336938281c83b52fdf253a5cd3d636036a8e5f22" exitCode=0 Jul 06 22:37:46.708881 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:46.708804 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-b6d6e159-openshift-default-754cc48dd7-qdnpq" event={"ID":"eecc11c5-ab05-4449-85ae-cd684f2774af","Type":"ContainerDied","Data":"2c106871c0c2c575e87a7c2f336938281c83b52fdf253a5cd3d636036a8e5f22"} Jul 06 22:37:46.709230 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:46.709217 2581 scope.go:117] "RemoveContainer" containerID="2c106871c0c2c575e87a7c2f336938281c83b52fdf253a5cd3d636036a8e5f22" Jul 06 22:37:46.710239 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:46.710210 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 22:37:46.710316 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:46.710297 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 22:37:46.710357 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:46.710338 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 22:37:47.716585 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:47.716554 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-b6d6e159-openshift-default-754cc48dd7-qdnpq" event={"ID":"eecc11c5-ab05-4449-85ae-cd684f2774af","Type":"ContainerStarted","Data":"2ea92dbe009366dc56d1f894ad3797af9dee8d7eb2ea0e6261de7d183f7efdb2"} Jul 06 22:37:48.064262 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:48.064159 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-b6d6e159-openshift-default-754cc48dd7-qdnpq" Jul 06 22:37:48.064443 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:48.064321 2581 patch_prober.go:28] interesting pod/e2e-mt-b6d6e159-openshift-default-754cc48dd7-qdnpq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body= Jul 06 22:37:48.064443 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:48.064376 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-b6d6e159-openshift-default-754cc48dd7-qdnpq" podUID="eecc11c5-ab05-4449-85ae-cd684f2774af" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" Jul 06 22:37:49.064221 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:49.064183 2581 patch_prober.go:28] interesting pod/e2e-mt-b6d6e159-openshift-default-754cc48dd7-qdnpq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body= Jul 06 22:37:49.064632 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:49.064238 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-b6d6e159-openshift-default-754cc48dd7-qdnpq" podUID="eecc11c5-ab05-4449-85ae-cd684f2774af" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" Jul 06 22:37:50.064284 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:50.064248 2581 patch_prober.go:28] interesting pod/e2e-mt-b6d6e159-openshift-default-754cc48dd7-qdnpq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body= Jul 06 22:37:50.064696 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:50.064313 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-b6d6e159-openshift-default-754cc48dd7-qdnpq" podUID="eecc11c5-ab05-4449-85ae-cd684f2774af" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" Jul 06 22:37:51.063770 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:51.063733 2581 patch_prober.go:28] interesting pod/e2e-mt-b6d6e159-openshift-default-754cc48dd7-qdnpq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body= Jul 06 22:37:51.063959 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:51.063788 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-b6d6e159-openshift-default-754cc48dd7-qdnpq" podUID="eecc11c5-ab05-4449-85ae-cd684f2774af" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" Jul 06 22:37:52.067885 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:52.067849 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-b6d6e159-openshift-default-754cc48dd7-qdnpq" Jul 06 22:37:52.068394 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:52.068252 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-b6d6e159-openshift-default-754cc48dd7-qdnpq" Jul 06 22:37:52.068904 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:52.068886 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-b6d6e159-openshift-default-754cc48dd7-qdnpq" Jul 06 22:37:55.643787 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:55.643754 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:37:55.745341 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:37:55.745310 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:38:00.683238 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:38:00.683205 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:38:30.009406 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:38:30.009356 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:38:30.439011 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:38:30.438977 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:38:30.540398 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:38:30.540353 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:38:30.700486 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:38:30.700393 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:38:36.165947 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:38:36.165861 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:38:41.025395 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:38:41.025364 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-rcn5q_86b07166-2f34-443b-9c61-86b7d950ad7b/console-operator/1.log" Jul 06 22:38:41.031729 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:38:41.031704 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-rcn5q_86b07166-2f34-443b-9c61-86b7d950ad7b/console-operator/1.log" Jul 06 22:38:41.155891 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:38:41.155855 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:39:17.844885 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:39:17.844815 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:39:18.886382 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:39:18.886350 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:39:23.920615 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:39:23.920587 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:39:24.005044 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:39:24.005013 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:39:24.200025 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:39:24.199988 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-b6d6e159-openshift-default-754cc48dd7-qdnpq"] Jul 06 22:39:24.200262 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:39:24.200235 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-b6d6e159-openshift-default-754cc48dd7-qdnpq" podUID="eecc11c5-ab05-4449-85ae-cd684f2774af" containerName="istio-proxy" containerID="cri-o://2ea92dbe009366dc56d1f894ad3797af9dee8d7eb2ea0e6261de7d183f7efdb2" gracePeriod=30 Jul 06 22:39:24.223641 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:39:24.223613 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:39:24.451006 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:39:24.450923 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:39:25.471508 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:39:25.471464 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:39:25.577261 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:39:25.577218 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:39:25.669706 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:39:25.669666 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:39:26.387190 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:39:26.387150 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:39:26.481192 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:39:26.481159 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:39:26.580083 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:39:26.580044 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:39:28.002587 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:39:28.002561 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:39:29.456539 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:39:29.456513 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-b6d6e159-openshift-default-754cc48dd7-qdnpq" Jul 06 22:39:29.531975 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:39:29.531898 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eecc11c5-ab05-4449-85ae-cd684f2774af-workload-socket\") pod \"eecc11c5-ab05-4449-85ae-cd684f2774af\" (UID: \"eecc11c5-ab05-4449-85ae-cd684f2774af\") " Jul 06 22:39:29.531975 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:39:29.531927 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eecc11c5-ab05-4449-85ae-cd684f2774af-credential-socket\") pod \"eecc11c5-ab05-4449-85ae-cd684f2774af\" (UID: \"eecc11c5-ab05-4449-85ae-cd684f2774af\") " Jul 06 22:39:29.531975 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:39:29.531949 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eecc11c5-ab05-4449-85ae-cd684f2774af-istio-envoy\") pod \"eecc11c5-ab05-4449-85ae-cd684f2774af\" (UID: \"eecc11c5-ab05-4449-85ae-cd684f2774af\") " Jul 06 22:39:29.532285 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:39:29.531989 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eecc11c5-ab05-4449-85ae-cd684f2774af-istio-podinfo\") pod \"eecc11c5-ab05-4449-85ae-cd684f2774af\" (UID: \"eecc11c5-ab05-4449-85ae-cd684f2774af\") " Jul 06 22:39:29.532285 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:39:29.532109 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-swb2c\" (UniqueName: \"kubernetes.io/projected/eecc11c5-ab05-4449-85ae-cd684f2774af-kube-api-access-swb2c\") pod \"eecc11c5-ab05-4449-85ae-cd684f2774af\" (UID: \"eecc11c5-ab05-4449-85ae-cd684f2774af\") " Jul 06 22:39:29.532285 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:39:29.532137 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eecc11c5-ab05-4449-85ae-cd684f2774af-istio-data\") pod \"eecc11c5-ab05-4449-85ae-cd684f2774af\" (UID: \"eecc11c5-ab05-4449-85ae-cd684f2774af\") " Jul 06 22:39:29.532285 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:39:29.532208 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eecc11c5-ab05-4449-85ae-cd684f2774af-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "eecc11c5-ab05-4449-85ae-cd684f2774af" (UID: "eecc11c5-ab05-4449-85ae-cd684f2774af"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:39:29.532285 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:39:29.532216 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eecc11c5-ab05-4449-85ae-cd684f2774af-istiod-ca-cert\") pod \"eecc11c5-ab05-4449-85ae-cd684f2774af\" (UID: \"eecc11c5-ab05-4449-85ae-cd684f2774af\") " Jul 06 22:39:29.532552 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:39:29.532281 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eecc11c5-ab05-4449-85ae-cd684f2774af-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "eecc11c5-ab05-4449-85ae-cd684f2774af" (UID: "eecc11c5-ab05-4449-85ae-cd684f2774af"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:39:29.532552 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:39:29.532311 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eecc11c5-ab05-4449-85ae-cd684f2774af-workload-certs\") pod \"eecc11c5-ab05-4449-85ae-cd684f2774af\" (UID: \"eecc11c5-ab05-4449-85ae-cd684f2774af\") " Jul 06 22:39:29.532552 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:39:29.532342 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eecc11c5-ab05-4449-85ae-cd684f2774af-istio-token\") pod \"eecc11c5-ab05-4449-85ae-cd684f2774af\" (UID: \"eecc11c5-ab05-4449-85ae-cd684f2774af\") " Jul 06 22:39:29.532819 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:39:29.532586 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eecc11c5-ab05-4449-85ae-cd684f2774af-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "eecc11c5-ab05-4449-85ae-cd684f2774af" (UID: "eecc11c5-ab05-4449-85ae-cd684f2774af"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:39:29.532819 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:39:29.532643 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eecc11c5-ab05-4449-85ae-cd684f2774af-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "eecc11c5-ab05-4449-85ae-cd684f2774af" (UID: "eecc11c5-ab05-4449-85ae-cd684f2774af"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 22:39:29.532819 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:39:29.532685 2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eecc11c5-ab05-4449-85ae-cd684f2774af-workload-certs\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:39:29.532819 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:39:29.532702 2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eecc11c5-ab05-4449-85ae-cd684f2774af-workload-socket\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:39:29.532819 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:39:29.532715 2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eecc11c5-ab05-4449-85ae-cd684f2774af-credential-socket\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:39:29.533140 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:39:29.533112 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eecc11c5-ab05-4449-85ae-cd684f2774af-istio-data" (OuterVolumeSpecName: "istio-data") pod "eecc11c5-ab05-4449-85ae-cd684f2774af" (UID: "eecc11c5-ab05-4449-85ae-cd684f2774af"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:39:29.534506 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:39:29.534476 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eecc11c5-ab05-4449-85ae-cd684f2774af-istio-token" (OuterVolumeSpecName: "istio-token") pod "eecc11c5-ab05-4449-85ae-cd684f2774af" (UID: "eecc11c5-ab05-4449-85ae-cd684f2774af"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 22:39:29.534676 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:39:29.534651 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/eecc11c5-ab05-4449-85ae-cd684f2774af-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "eecc11c5-ab05-4449-85ae-cd684f2774af" (UID: "eecc11c5-ab05-4449-85ae-cd684f2774af"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 22:39:29.534676 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:39:29.534667 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eecc11c5-ab05-4449-85ae-cd684f2774af-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "eecc11c5-ab05-4449-85ae-cd684f2774af" (UID: "eecc11c5-ab05-4449-85ae-cd684f2774af"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:39:29.534805 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:39:29.534744 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eecc11c5-ab05-4449-85ae-cd684f2774af-kube-api-access-swb2c" (OuterVolumeSpecName: "kube-api-access-swb2c") pod "eecc11c5-ab05-4449-85ae-cd684f2774af" (UID: "eecc11c5-ab05-4449-85ae-cd684f2774af"). InnerVolumeSpecName "kube-api-access-swb2c". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 22:39:29.633332 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:39:29.633298 2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eecc11c5-ab05-4449-85ae-cd684f2774af-istio-token\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:39:29.633332 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:39:29.633326 2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eecc11c5-ab05-4449-85ae-cd684f2774af-istio-envoy\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:39:29.633332 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:39:29.633336 2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eecc11c5-ab05-4449-85ae-cd684f2774af-istio-podinfo\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:39:29.633568 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:39:29.633346 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-swb2c\" (UniqueName: \"kubernetes.io/projected/eecc11c5-ab05-4449-85ae-cd684f2774af-kube-api-access-swb2c\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:39:29.633568 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:39:29.633356 2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eecc11c5-ab05-4449-85ae-cd684f2774af-istio-data\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:39:29.633568 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:39:29.633364 2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eecc11c5-ab05-4449-85ae-cd684f2774af-istiod-ca-cert\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:39:30.238702 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:39:30.238666 2581 generic.go:356] "Generic (PLEG): container finished" podID="eecc11c5-ab05-4449-85ae-cd684f2774af" containerID="2ea92dbe009366dc56d1f894ad3797af9dee8d7eb2ea0e6261de7d183f7efdb2" exitCode=0 Jul 06 22:39:30.238863 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:39:30.238716 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-b6d6e159-openshift-default-754cc48dd7-qdnpq" Jul 06 22:39:30.238863 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:39:30.238743 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-b6d6e159-openshift-default-754cc48dd7-qdnpq" event={"ID":"eecc11c5-ab05-4449-85ae-cd684f2774af","Type":"ContainerDied","Data":"2ea92dbe009366dc56d1f894ad3797af9dee8d7eb2ea0e6261de7d183f7efdb2"} Jul 06 22:39:30.238863 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:39:30.238797 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-b6d6e159-openshift-default-754cc48dd7-qdnpq" event={"ID":"eecc11c5-ab05-4449-85ae-cd684f2774af","Type":"ContainerDied","Data":"db584b4b25d23946486b9f8c695adbddc7720735d3f2e251c121f56870d7b0e3"} Jul 06 22:39:30.238863 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:39:30.238821 2581 scope.go:117] "RemoveContainer" containerID="2ea92dbe009366dc56d1f894ad3797af9dee8d7eb2ea0e6261de7d183f7efdb2" Jul 06 22:39:30.248630 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:39:30.248614 2581 scope.go:117] "RemoveContainer" containerID="2c106871c0c2c575e87a7c2f336938281c83b52fdf253a5cd3d636036a8e5f22" Jul 06 22:39:30.256504 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:39:30.256485 2581 scope.go:117] "RemoveContainer" containerID="2ea92dbe009366dc56d1f894ad3797af9dee8d7eb2ea0e6261de7d183f7efdb2" Jul 06 22:39:30.256716 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:39:30.256700 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2ea92dbe009366dc56d1f894ad3797af9dee8d7eb2ea0e6261de7d183f7efdb2\": container with ID starting with 2ea92dbe009366dc56d1f894ad3797af9dee8d7eb2ea0e6261de7d183f7efdb2 not found: ID does not exist" containerID="2ea92dbe009366dc56d1f894ad3797af9dee8d7eb2ea0e6261de7d183f7efdb2" Jul 06 22:39:30.256766 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:39:30.256723 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2ea92dbe009366dc56d1f894ad3797af9dee8d7eb2ea0e6261de7d183f7efdb2"} err="failed to get container status \"2ea92dbe009366dc56d1f894ad3797af9dee8d7eb2ea0e6261de7d183f7efdb2\": rpc error: code = NotFound desc = could not find container \"2ea92dbe009366dc56d1f894ad3797af9dee8d7eb2ea0e6261de7d183f7efdb2\": container with ID starting with 2ea92dbe009366dc56d1f894ad3797af9dee8d7eb2ea0e6261de7d183f7efdb2 not found: ID does not exist" Jul 06 22:39:30.256766 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:39:30.256738 2581 scope.go:117] "RemoveContainer" containerID="2c106871c0c2c575e87a7c2f336938281c83b52fdf253a5cd3d636036a8e5f22" Jul 06 22:39:30.256922 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:39:30.256902 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2c106871c0c2c575e87a7c2f336938281c83b52fdf253a5cd3d636036a8e5f22\": container with ID starting with 2c106871c0c2c575e87a7c2f336938281c83b52fdf253a5cd3d636036a8e5f22 not found: ID does not exist" containerID="2c106871c0c2c575e87a7c2f336938281c83b52fdf253a5cd3d636036a8e5f22" Jul 06 22:39:30.256982 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:39:30.256934 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c106871c0c2c575e87a7c2f336938281c83b52fdf253a5cd3d636036a8e5f22"} err="failed to get container status \"2c106871c0c2c575e87a7c2f336938281c83b52fdf253a5cd3d636036a8e5f22\": rpc error: code = NotFound desc = could not find container \"2c106871c0c2c575e87a7c2f336938281c83b52fdf253a5cd3d636036a8e5f22\": container with ID starting with 2c106871c0c2c575e87a7c2f336938281c83b52fdf253a5cd3d636036a8e5f22 not found: ID does not exist" Jul 06 22:39:30.261734 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:39:30.261710 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-b6d6e159-openshift-default-754cc48dd7-qdnpq"] Jul 06 22:39:30.263883 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:39:30.263859 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-b6d6e159-openshift-default-754cc48dd7-qdnpq"] Jul 06 22:39:30.976812 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:39:30.976770 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eecc11c5-ab05-4449-85ae-cd684f2774af" path="/var/lib/kubelet/pods/eecc11c5-ab05-4449-85ae-cd684f2774af/volumes" Jul 06 22:39:33.752205 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:39:33.752165 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:39:33.818508 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:39:33.818476 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:39:34.435540 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:39:34.435506 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:39:40.725200 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:39:40.725171 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:39:40.830549 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:39:40.830494 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:39:40.943755 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:39:40.943724 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:39:51.471683 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:39:51.471638 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:39:51.825327 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:39:51.825242 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:40:12.809347 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:12.809268 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:40:18.645322 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:18.645284 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:40:18.714248 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:18.714213 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:40:19.013286 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:19.013254 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:40:19.309134 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:19.309034 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-17b26a-openshift-default-9d9b7c8ff-g79kb"] Jul 06 22:40:19.309450 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:19.309437 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eecc11c5-ab05-4449-85ae-cd684f2774af" containerName="istio-proxy" Jul 06 22:40:19.309496 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:19.309452 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="eecc11c5-ab05-4449-85ae-cd684f2774af" containerName="istio-proxy" Jul 06 22:40:19.309496 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:19.309465 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eecc11c5-ab05-4449-85ae-cd684f2774af" containerName="istio-proxy" Jul 06 22:40:19.309496 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:19.309470 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="eecc11c5-ab05-4449-85ae-cd684f2774af" containerName="istio-proxy" Jul 06 22:40:19.309589 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:19.309535 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="eecc11c5-ab05-4449-85ae-cd684f2774af" containerName="istio-proxy" Jul 06 22:40:19.309589 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:19.309542 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="eecc11c5-ab05-4449-85ae-cd684f2774af" containerName="istio-proxy" Jul 06 22:40:19.312311 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:19.312292 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-17b26a-openshift-default-9d9b7c8ff-g79kb" Jul 06 22:40:19.315139 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:19.315116 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-17b26a-openshift-default-dockercfg-d6zsm\"" Jul 06 22:40:19.323801 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:19.323779 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-17b26a-openshift-default-9d9b7c8ff-g79kb"] Jul 06 22:40:19.335427 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:19.335404 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:40:19.497452 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:19.497419 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/76850ce2-1749-4478-8a5e-6ba64d7c9b80-credential-socket\") pod \"e2e-inf-a-17b26a-openshift-default-9d9b7c8ff-g79kb\" (UID: \"76850ce2-1749-4478-8a5e-6ba64d7c9b80\") " pod="openshift-ingress/e2e-inf-a-17b26a-openshift-default-9d9b7c8ff-g79kb" Jul 06 22:40:19.497658 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:19.497489 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/76850ce2-1749-4478-8a5e-6ba64d7c9b80-istio-token\") pod \"e2e-inf-a-17b26a-openshift-default-9d9b7c8ff-g79kb\" (UID: \"76850ce2-1749-4478-8a5e-6ba64d7c9b80\") " pod="openshift-ingress/e2e-inf-a-17b26a-openshift-default-9d9b7c8ff-g79kb" Jul 06 22:40:19.497658 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:19.497531 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/76850ce2-1749-4478-8a5e-6ba64d7c9b80-workload-certs\") pod \"e2e-inf-a-17b26a-openshift-default-9d9b7c8ff-g79kb\" (UID: \"76850ce2-1749-4478-8a5e-6ba64d7c9b80\") " pod="openshift-ingress/e2e-inf-a-17b26a-openshift-default-9d9b7c8ff-g79kb" Jul 06 22:40:19.497658 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:19.497554 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/76850ce2-1749-4478-8a5e-6ba64d7c9b80-istio-podinfo\") pod \"e2e-inf-a-17b26a-openshift-default-9d9b7c8ff-g79kb\" (UID: \"76850ce2-1749-4478-8a5e-6ba64d7c9b80\") " pod="openshift-ingress/e2e-inf-a-17b26a-openshift-default-9d9b7c8ff-g79kb" Jul 06 22:40:19.497658 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:19.497597 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/76850ce2-1749-4478-8a5e-6ba64d7c9b80-workload-socket\") pod \"e2e-inf-a-17b26a-openshift-default-9d9b7c8ff-g79kb\" (UID: \"76850ce2-1749-4478-8a5e-6ba64d7c9b80\") " pod="openshift-ingress/e2e-inf-a-17b26a-openshift-default-9d9b7c8ff-g79kb" Jul 06 22:40:19.497658 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:19.497620 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/76850ce2-1749-4478-8a5e-6ba64d7c9b80-istio-data\") pod \"e2e-inf-a-17b26a-openshift-default-9d9b7c8ff-g79kb\" (UID: \"76850ce2-1749-4478-8a5e-6ba64d7c9b80\") " pod="openshift-ingress/e2e-inf-a-17b26a-openshift-default-9d9b7c8ff-g79kb" Jul 06 22:40:19.497859 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:19.497664 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b9k87\" (UniqueName: \"kubernetes.io/projected/76850ce2-1749-4478-8a5e-6ba64d7c9b80-kube-api-access-b9k87\") pod \"e2e-inf-a-17b26a-openshift-default-9d9b7c8ff-g79kb\" (UID: \"76850ce2-1749-4478-8a5e-6ba64d7c9b80\") " pod="openshift-ingress/e2e-inf-a-17b26a-openshift-default-9d9b7c8ff-g79kb" Jul 06 22:40:19.497859 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:19.497714 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/76850ce2-1749-4478-8a5e-6ba64d7c9b80-istio-envoy\") pod \"e2e-inf-a-17b26a-openshift-default-9d9b7c8ff-g79kb\" (UID: \"76850ce2-1749-4478-8a5e-6ba64d7c9b80\") " pod="openshift-ingress/e2e-inf-a-17b26a-openshift-default-9d9b7c8ff-g79kb" Jul 06 22:40:19.497859 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:19.497761 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/76850ce2-1749-4478-8a5e-6ba64d7c9b80-istiod-ca-cert\") pod \"e2e-inf-a-17b26a-openshift-default-9d9b7c8ff-g79kb\" (UID: \"76850ce2-1749-4478-8a5e-6ba64d7c9b80\") " pod="openshift-ingress/e2e-inf-a-17b26a-openshift-default-9d9b7c8ff-g79kb" Jul 06 22:40:19.598958 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:19.598867 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/76850ce2-1749-4478-8a5e-6ba64d7c9b80-credential-socket\") pod \"e2e-inf-a-17b26a-openshift-default-9d9b7c8ff-g79kb\" (UID: \"76850ce2-1749-4478-8a5e-6ba64d7c9b80\") " pod="openshift-ingress/e2e-inf-a-17b26a-openshift-default-9d9b7c8ff-g79kb" Jul 06 22:40:19.598958 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:19.598928 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/76850ce2-1749-4478-8a5e-6ba64d7c9b80-istio-token\") pod \"e2e-inf-a-17b26a-openshift-default-9d9b7c8ff-g79kb\" (UID: \"76850ce2-1749-4478-8a5e-6ba64d7c9b80\") " pod="openshift-ingress/e2e-inf-a-17b26a-openshift-default-9d9b7c8ff-g79kb" Jul 06 22:40:19.598958 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:19.598950 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/76850ce2-1749-4478-8a5e-6ba64d7c9b80-workload-certs\") pod \"e2e-inf-a-17b26a-openshift-default-9d9b7c8ff-g79kb\" (UID: \"76850ce2-1749-4478-8a5e-6ba64d7c9b80\") " pod="openshift-ingress/e2e-inf-a-17b26a-openshift-default-9d9b7c8ff-g79kb" Jul 06 22:40:19.599283 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:19.598968 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/76850ce2-1749-4478-8a5e-6ba64d7c9b80-istio-podinfo\") pod \"e2e-inf-a-17b26a-openshift-default-9d9b7c8ff-g79kb\" (UID: \"76850ce2-1749-4478-8a5e-6ba64d7c9b80\") " pod="openshift-ingress/e2e-inf-a-17b26a-openshift-default-9d9b7c8ff-g79kb" Jul 06 22:40:19.599283 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:19.598997 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/76850ce2-1749-4478-8a5e-6ba64d7c9b80-workload-socket\") pod \"e2e-inf-a-17b26a-openshift-default-9d9b7c8ff-g79kb\" (UID: \"76850ce2-1749-4478-8a5e-6ba64d7c9b80\") " pod="openshift-ingress/e2e-inf-a-17b26a-openshift-default-9d9b7c8ff-g79kb" Jul 06 22:40:19.599283 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:19.599014 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/76850ce2-1749-4478-8a5e-6ba64d7c9b80-istio-data\") pod \"e2e-inf-a-17b26a-openshift-default-9d9b7c8ff-g79kb\" (UID: \"76850ce2-1749-4478-8a5e-6ba64d7c9b80\") " pod="openshift-ingress/e2e-inf-a-17b26a-openshift-default-9d9b7c8ff-g79kb" Jul 06 22:40:19.599283 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:19.599030 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b9k87\" (UniqueName: \"kubernetes.io/projected/76850ce2-1749-4478-8a5e-6ba64d7c9b80-kube-api-access-b9k87\") pod \"e2e-inf-a-17b26a-openshift-default-9d9b7c8ff-g79kb\" (UID: \"76850ce2-1749-4478-8a5e-6ba64d7c9b80\") " pod="openshift-ingress/e2e-inf-a-17b26a-openshift-default-9d9b7c8ff-g79kb" Jul 06 22:40:19.599283 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:19.599077 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/76850ce2-1749-4478-8a5e-6ba64d7c9b80-istio-envoy\") pod \"e2e-inf-a-17b26a-openshift-default-9d9b7c8ff-g79kb\" (UID: \"76850ce2-1749-4478-8a5e-6ba64d7c9b80\") " pod="openshift-ingress/e2e-inf-a-17b26a-openshift-default-9d9b7c8ff-g79kb" Jul 06 22:40:19.599283 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:19.599139 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/76850ce2-1749-4478-8a5e-6ba64d7c9b80-istiod-ca-cert\") pod \"e2e-inf-a-17b26a-openshift-default-9d9b7c8ff-g79kb\" (UID: \"76850ce2-1749-4478-8a5e-6ba64d7c9b80\") " pod="openshift-ingress/e2e-inf-a-17b26a-openshift-default-9d9b7c8ff-g79kb" Jul 06 22:40:19.599597 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:19.599365 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/76850ce2-1749-4478-8a5e-6ba64d7c9b80-credential-socket\") pod \"e2e-inf-a-17b26a-openshift-default-9d9b7c8ff-g79kb\" (UID: \"76850ce2-1749-4478-8a5e-6ba64d7c9b80\") " pod="openshift-ingress/e2e-inf-a-17b26a-openshift-default-9d9b7c8ff-g79kb" Jul 06 22:40:19.599597 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:19.599417 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/76850ce2-1749-4478-8a5e-6ba64d7c9b80-workload-certs\") pod \"e2e-inf-a-17b26a-openshift-default-9d9b7c8ff-g79kb\" (UID: \"76850ce2-1749-4478-8a5e-6ba64d7c9b80\") " pod="openshift-ingress/e2e-inf-a-17b26a-openshift-default-9d9b7c8ff-g79kb" Jul 06 22:40:19.599597 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:19.599484 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/76850ce2-1749-4478-8a5e-6ba64d7c9b80-istio-data\") pod \"e2e-inf-a-17b26a-openshift-default-9d9b7c8ff-g79kb\" (UID: \"76850ce2-1749-4478-8a5e-6ba64d7c9b80\") " pod="openshift-ingress/e2e-inf-a-17b26a-openshift-default-9d9b7c8ff-g79kb" Jul 06 22:40:19.599597 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:19.599555 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/76850ce2-1749-4478-8a5e-6ba64d7c9b80-workload-socket\") pod \"e2e-inf-a-17b26a-openshift-default-9d9b7c8ff-g79kb\" (UID: \"76850ce2-1749-4478-8a5e-6ba64d7c9b80\") " pod="openshift-ingress/e2e-inf-a-17b26a-openshift-default-9d9b7c8ff-g79kb" Jul 06 22:40:19.599985 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:19.599966 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/76850ce2-1749-4478-8a5e-6ba64d7c9b80-istiod-ca-cert\") pod \"e2e-inf-a-17b26a-openshift-default-9d9b7c8ff-g79kb\" (UID: \"76850ce2-1749-4478-8a5e-6ba64d7c9b80\") " pod="openshift-ingress/e2e-inf-a-17b26a-openshift-default-9d9b7c8ff-g79kb" Jul 06 22:40:19.601520 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:19.601497 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/76850ce2-1749-4478-8a5e-6ba64d7c9b80-istio-envoy\") pod \"e2e-inf-a-17b26a-openshift-default-9d9b7c8ff-g79kb\" (UID: \"76850ce2-1749-4478-8a5e-6ba64d7c9b80\") " pod="openshift-ingress/e2e-inf-a-17b26a-openshift-default-9d9b7c8ff-g79kb" Jul 06 22:40:19.601830 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:19.601811 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/76850ce2-1749-4478-8a5e-6ba64d7c9b80-istio-podinfo\") pod \"e2e-inf-a-17b26a-openshift-default-9d9b7c8ff-g79kb\" (UID: \"76850ce2-1749-4478-8a5e-6ba64d7c9b80\") " pod="openshift-ingress/e2e-inf-a-17b26a-openshift-default-9d9b7c8ff-g79kb" Jul 06 22:40:19.607084 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:19.607042 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/76850ce2-1749-4478-8a5e-6ba64d7c9b80-istio-token\") pod \"e2e-inf-a-17b26a-openshift-default-9d9b7c8ff-g79kb\" (UID: \"76850ce2-1749-4478-8a5e-6ba64d7c9b80\") " pod="openshift-ingress/e2e-inf-a-17b26a-openshift-default-9d9b7c8ff-g79kb" Jul 06 22:40:19.607452 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:19.607429 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b9k87\" (UniqueName: \"kubernetes.io/projected/76850ce2-1749-4478-8a5e-6ba64d7c9b80-kube-api-access-b9k87\") pod \"e2e-inf-a-17b26a-openshift-default-9d9b7c8ff-g79kb\" (UID: \"76850ce2-1749-4478-8a5e-6ba64d7c9b80\") " pod="openshift-ingress/e2e-inf-a-17b26a-openshift-default-9d9b7c8ff-g79kb" Jul 06 22:40:19.624186 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:19.624162 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-17b26a-openshift-default-9d9b7c8ff-g79kb" Jul 06 22:40:19.752915 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:19.752888 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:40:19.770804 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:19.770779 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-17b26a-openshift-default-9d9b7c8ff-g79kb"] Jul 06 22:40:19.773089 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:40:19.773042 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76850ce2_1749_4478_8a5e_6ba64d7c9b80.slice/crio-b63269d251e7cf47c4c437272ec2c021d27d6d5a4af9a1c033ba6037bf8010e5 WatchSource:0}: Error finding container b63269d251e7cf47c4c437272ec2c021d27d6d5a4af9a1c033ba6037bf8010e5: Status 404 returned error can't find the container with id b63269d251e7cf47c4c437272ec2c021d27d6d5a4af9a1c033ba6037bf8010e5 Jul 06 22:40:19.775284 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:19.775250 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 22:40:19.775393 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:19.775331 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 22:40:19.775393 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:19.775377 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 22:40:20.483990 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:20.483946 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-17b26a-openshift-default-9d9b7c8ff-g79kb" event={"ID":"76850ce2-1749-4478-8a5e-6ba64d7c9b80","Type":"ContainerStarted","Data":"bf1e8bfa6dce95a2f20e6be629536ce1de3b34d7eb9dea57f70c9b97d0def0b6"} Jul 06 22:40:20.484207 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:20.484000 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-17b26a-openshift-default-9d9b7c8ff-g79kb" event={"ID":"76850ce2-1749-4478-8a5e-6ba64d7c9b80","Type":"ContainerStarted","Data":"b63269d251e7cf47c4c437272ec2c021d27d6d5a4af9a1c033ba6037bf8010e5"} Jul 06 22:40:20.504895 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:20.504834 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-17b26a-openshift-default-9d9b7c8ff-g79kb" podStartSLOduration=1.504818047 podStartE2EDuration="1.504818047s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:40:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:40:20.503606724 +0000 UTC m=+1900.062402045" watchObservedRunningTime="2026-07-06 22:40:20.504818047 +0000 UTC m=+1900.063613367" Jul 06 22:40:20.625267 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:20.625232 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-17b26a-openshift-default-9d9b7c8ff-g79kb" Jul 06 22:40:20.630277 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:20.630253 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-17b26a-openshift-default-9d9b7c8ff-g79kb" Jul 06 22:40:21.488347 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:21.488306 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-17b26a-openshift-default-9d9b7c8ff-g79kb" Jul 06 22:40:21.489320 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:21.489299 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-17b26a-openshift-default-9d9b7c8ff-g79kb" Jul 06 22:40:22.477011 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:22.476980 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:40:22.579980 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:22.579946 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:40:22.700637 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:22.700604 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:40:22.883189 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:22.883101 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:40:23.245364 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:23.245330 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:40:23.348219 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:23.348185 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:40:27.013242 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:27.013196 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:40:27.360991 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:27.360900 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:40:27.650214 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:27.650127 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-5d913f-openshift-default-5d756c7c6c-nfcfz"] Jul 06 22:40:27.654461 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:27.654434 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-5d913f-openshift-default-5d756c7c6c-nfcfz" Jul 06 22:40:27.657002 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:27.656975 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-5d913f-openshift-default-dockercfg-7fx5w\"" Jul 06 22:40:27.664194 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:27.664163 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-5d913f-openshift-default-5d756c7c6c-nfcfz"] Jul 06 22:40:27.677428 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:27.677399 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/69185b8f-db46-4b55-85a1-27bfc2027672-istio-data\") pod \"e2e-inf-b-5d913f-openshift-default-5d756c7c6c-nfcfz\" (UID: \"69185b8f-db46-4b55-85a1-27bfc2027672\") " pod="openshift-ingress/e2e-inf-b-5d913f-openshift-default-5d756c7c6c-nfcfz" Jul 06 22:40:27.677592 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:27.677538 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xdx58\" (UniqueName: \"kubernetes.io/projected/69185b8f-db46-4b55-85a1-27bfc2027672-kube-api-access-xdx58\") pod \"e2e-inf-b-5d913f-openshift-default-5d756c7c6c-nfcfz\" (UID: \"69185b8f-db46-4b55-85a1-27bfc2027672\") " pod="openshift-ingress/e2e-inf-b-5d913f-openshift-default-5d756c7c6c-nfcfz" Jul 06 22:40:27.677662 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:27.677638 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/69185b8f-db46-4b55-85a1-27bfc2027672-istio-envoy\") pod \"e2e-inf-b-5d913f-openshift-default-5d756c7c6c-nfcfz\" (UID: \"69185b8f-db46-4b55-85a1-27bfc2027672\") " pod="openshift-ingress/e2e-inf-b-5d913f-openshift-default-5d756c7c6c-nfcfz" Jul 06 22:40:27.677724 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:27.677687 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/69185b8f-db46-4b55-85a1-27bfc2027672-istio-token\") pod \"e2e-inf-b-5d913f-openshift-default-5d756c7c6c-nfcfz\" (UID: \"69185b8f-db46-4b55-85a1-27bfc2027672\") " pod="openshift-ingress/e2e-inf-b-5d913f-openshift-default-5d756c7c6c-nfcfz" Jul 06 22:40:27.677781 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:27.677722 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/69185b8f-db46-4b55-85a1-27bfc2027672-istio-podinfo\") pod \"e2e-inf-b-5d913f-openshift-default-5d756c7c6c-nfcfz\" (UID: \"69185b8f-db46-4b55-85a1-27bfc2027672\") " pod="openshift-ingress/e2e-inf-b-5d913f-openshift-default-5d756c7c6c-nfcfz" Jul 06 22:40:27.677781 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:27.677762 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/69185b8f-db46-4b55-85a1-27bfc2027672-istiod-ca-cert\") pod \"e2e-inf-b-5d913f-openshift-default-5d756c7c6c-nfcfz\" (UID: \"69185b8f-db46-4b55-85a1-27bfc2027672\") " pod="openshift-ingress/e2e-inf-b-5d913f-openshift-default-5d756c7c6c-nfcfz" Jul 06 22:40:27.677885 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:27.677800 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/69185b8f-db46-4b55-85a1-27bfc2027672-credential-socket\") pod \"e2e-inf-b-5d913f-openshift-default-5d756c7c6c-nfcfz\" (UID: \"69185b8f-db46-4b55-85a1-27bfc2027672\") " pod="openshift-ingress/e2e-inf-b-5d913f-openshift-default-5d756c7c6c-nfcfz" Jul 06 22:40:27.677885 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:27.677850 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/69185b8f-db46-4b55-85a1-27bfc2027672-workload-socket\") pod \"e2e-inf-b-5d913f-openshift-default-5d756c7c6c-nfcfz\" (UID: \"69185b8f-db46-4b55-85a1-27bfc2027672\") " pod="openshift-ingress/e2e-inf-b-5d913f-openshift-default-5d756c7c6c-nfcfz" Jul 06 22:40:27.678000 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:27.677883 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/69185b8f-db46-4b55-85a1-27bfc2027672-workload-certs\") pod \"e2e-inf-b-5d913f-openshift-default-5d756c7c6c-nfcfz\" (UID: \"69185b8f-db46-4b55-85a1-27bfc2027672\") " pod="openshift-ingress/e2e-inf-b-5d913f-openshift-default-5d756c7c6c-nfcfz" Jul 06 22:40:27.678670 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:27.678632 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:40:27.778803 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:27.778768 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xdx58\" (UniqueName: \"kubernetes.io/projected/69185b8f-db46-4b55-85a1-27bfc2027672-kube-api-access-xdx58\") pod \"e2e-inf-b-5d913f-openshift-default-5d756c7c6c-nfcfz\" (UID: \"69185b8f-db46-4b55-85a1-27bfc2027672\") " pod="openshift-ingress/e2e-inf-b-5d913f-openshift-default-5d756c7c6c-nfcfz" Jul 06 22:40:27.779012 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:27.778820 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/69185b8f-db46-4b55-85a1-27bfc2027672-istio-envoy\") pod \"e2e-inf-b-5d913f-openshift-default-5d756c7c6c-nfcfz\" (UID: \"69185b8f-db46-4b55-85a1-27bfc2027672\") " pod="openshift-ingress/e2e-inf-b-5d913f-openshift-default-5d756c7c6c-nfcfz" Jul 06 22:40:27.779012 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:27.778848 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/69185b8f-db46-4b55-85a1-27bfc2027672-istio-token\") pod \"e2e-inf-b-5d913f-openshift-default-5d756c7c6c-nfcfz\" (UID: \"69185b8f-db46-4b55-85a1-27bfc2027672\") " pod="openshift-ingress/e2e-inf-b-5d913f-openshift-default-5d756c7c6c-nfcfz" Jul 06 22:40:27.779012 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:27.778880 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/69185b8f-db46-4b55-85a1-27bfc2027672-istio-podinfo\") pod \"e2e-inf-b-5d913f-openshift-default-5d756c7c6c-nfcfz\" (UID: \"69185b8f-db46-4b55-85a1-27bfc2027672\") " pod="openshift-ingress/e2e-inf-b-5d913f-openshift-default-5d756c7c6c-nfcfz" Jul 06 22:40:27.779012 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:27.778910 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/69185b8f-db46-4b55-85a1-27bfc2027672-istiod-ca-cert\") pod \"e2e-inf-b-5d913f-openshift-default-5d756c7c6c-nfcfz\" (UID: \"69185b8f-db46-4b55-85a1-27bfc2027672\") " pod="openshift-ingress/e2e-inf-b-5d913f-openshift-default-5d756c7c6c-nfcfz" Jul 06 22:40:27.779012 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:27.778943 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/69185b8f-db46-4b55-85a1-27bfc2027672-credential-socket\") pod \"e2e-inf-b-5d913f-openshift-default-5d756c7c6c-nfcfz\" (UID: \"69185b8f-db46-4b55-85a1-27bfc2027672\") " pod="openshift-ingress/e2e-inf-b-5d913f-openshift-default-5d756c7c6c-nfcfz" Jul 06 22:40:27.779012 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:27.778980 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/69185b8f-db46-4b55-85a1-27bfc2027672-workload-socket\") pod \"e2e-inf-b-5d913f-openshift-default-5d756c7c6c-nfcfz\" (UID: \"69185b8f-db46-4b55-85a1-27bfc2027672\") " pod="openshift-ingress/e2e-inf-b-5d913f-openshift-default-5d756c7c6c-nfcfz" Jul 06 22:40:27.779362 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:27.779012 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/69185b8f-db46-4b55-85a1-27bfc2027672-workload-certs\") pod \"e2e-inf-b-5d913f-openshift-default-5d756c7c6c-nfcfz\" (UID: \"69185b8f-db46-4b55-85a1-27bfc2027672\") " pod="openshift-ingress/e2e-inf-b-5d913f-openshift-default-5d756c7c6c-nfcfz" Jul 06 22:40:27.779362 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:27.779040 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/69185b8f-db46-4b55-85a1-27bfc2027672-istio-data\") pod \"e2e-inf-b-5d913f-openshift-default-5d756c7c6c-nfcfz\" (UID: \"69185b8f-db46-4b55-85a1-27bfc2027672\") " pod="openshift-ingress/e2e-inf-b-5d913f-openshift-default-5d756c7c6c-nfcfz" Jul 06 22:40:27.779464 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:27.779377 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/69185b8f-db46-4b55-85a1-27bfc2027672-credential-socket\") pod \"e2e-inf-b-5d913f-openshift-default-5d756c7c6c-nfcfz\" (UID: \"69185b8f-db46-4b55-85a1-27bfc2027672\") " pod="openshift-ingress/e2e-inf-b-5d913f-openshift-default-5d756c7c6c-nfcfz" Jul 06 22:40:27.779464 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:27.779449 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/69185b8f-db46-4b55-85a1-27bfc2027672-istio-data\") pod \"e2e-inf-b-5d913f-openshift-default-5d756c7c6c-nfcfz\" (UID: \"69185b8f-db46-4b55-85a1-27bfc2027672\") " pod="openshift-ingress/e2e-inf-b-5d913f-openshift-default-5d756c7c6c-nfcfz" Jul 06 22:40:27.779574 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:27.779513 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/69185b8f-db46-4b55-85a1-27bfc2027672-workload-certs\") pod \"e2e-inf-b-5d913f-openshift-default-5d756c7c6c-nfcfz\" (UID: \"69185b8f-db46-4b55-85a1-27bfc2027672\") " pod="openshift-ingress/e2e-inf-b-5d913f-openshift-default-5d756c7c6c-nfcfz" Jul 06 22:40:27.779772 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:27.779753 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/69185b8f-db46-4b55-85a1-27bfc2027672-workload-socket\") pod \"e2e-inf-b-5d913f-openshift-default-5d756c7c6c-nfcfz\" (UID: \"69185b8f-db46-4b55-85a1-27bfc2027672\") " pod="openshift-ingress/e2e-inf-b-5d913f-openshift-default-5d756c7c6c-nfcfz" Jul 06 22:40:27.779840 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:27.779759 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/69185b8f-db46-4b55-85a1-27bfc2027672-istiod-ca-cert\") pod \"e2e-inf-b-5d913f-openshift-default-5d756c7c6c-nfcfz\" (UID: \"69185b8f-db46-4b55-85a1-27bfc2027672\") " pod="openshift-ingress/e2e-inf-b-5d913f-openshift-default-5d756c7c6c-nfcfz" Jul 06 22:40:27.781267 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:27.781245 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/69185b8f-db46-4b55-85a1-27bfc2027672-istio-envoy\") pod \"e2e-inf-b-5d913f-openshift-default-5d756c7c6c-nfcfz\" (UID: \"69185b8f-db46-4b55-85a1-27bfc2027672\") " pod="openshift-ingress/e2e-inf-b-5d913f-openshift-default-5d756c7c6c-nfcfz" Jul 06 22:40:27.781484 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:27.781467 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/69185b8f-db46-4b55-85a1-27bfc2027672-istio-podinfo\") pod \"e2e-inf-b-5d913f-openshift-default-5d756c7c6c-nfcfz\" (UID: \"69185b8f-db46-4b55-85a1-27bfc2027672\") " pod="openshift-ingress/e2e-inf-b-5d913f-openshift-default-5d756c7c6c-nfcfz" Jul 06 22:40:27.787777 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:27.787759 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/69185b8f-db46-4b55-85a1-27bfc2027672-istio-token\") pod \"e2e-inf-b-5d913f-openshift-default-5d756c7c6c-nfcfz\" (UID: \"69185b8f-db46-4b55-85a1-27bfc2027672\") " pod="openshift-ingress/e2e-inf-b-5d913f-openshift-default-5d756c7c6c-nfcfz" Jul 06 22:40:27.787907 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:27.787888 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xdx58\" (UniqueName: \"kubernetes.io/projected/69185b8f-db46-4b55-85a1-27bfc2027672-kube-api-access-xdx58\") pod \"e2e-inf-b-5d913f-openshift-default-5d756c7c6c-nfcfz\" (UID: \"69185b8f-db46-4b55-85a1-27bfc2027672\") " pod="openshift-ingress/e2e-inf-b-5d913f-openshift-default-5d756c7c6c-nfcfz" Jul 06 22:40:27.969153 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:27.969114 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-5d913f-openshift-default-5d756c7c6c-nfcfz" Jul 06 22:40:28.125494 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:28.125468 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-5d913f-openshift-default-5d756c7c6c-nfcfz"] Jul 06 22:40:28.127769 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:40:28.127742 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69185b8f_db46_4b55_85a1_27bfc2027672.slice/crio-9578b0c1204a1f8648c2dac5fb1ff9f9dc31e6f964617a6c3f5ec4003922060e WatchSource:0}: Error finding container 9578b0c1204a1f8648c2dac5fb1ff9f9dc31e6f964617a6c3f5ec4003922060e: Status 404 returned error can't find the container with id 9578b0c1204a1f8648c2dac5fb1ff9f9dc31e6f964617a6c3f5ec4003922060e Jul 06 22:40:28.130410 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:28.130372 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 22:40:28.130499 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:28.130458 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 22:40:28.130539 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:28.130500 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 22:40:28.345120 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:28.345013 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:40:28.531283 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:28.531246 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-5d913f-openshift-default-5d756c7c6c-nfcfz" event={"ID":"69185b8f-db46-4b55-85a1-27bfc2027672","Type":"ContainerStarted","Data":"999e90e300f2958a66e56f2499a47d377a72f26a04e39c63d0c71456d15fa4b9"} Jul 06 22:40:28.531434 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:28.531328 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-5d913f-openshift-default-5d756c7c6c-nfcfz" event={"ID":"69185b8f-db46-4b55-85a1-27bfc2027672","Type":"ContainerStarted","Data":"9578b0c1204a1f8648c2dac5fb1ff9f9dc31e6f964617a6c3f5ec4003922060e"} Jul 06 22:40:28.558853 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:28.558801 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-5d913f-openshift-default-5d756c7c6c-nfcfz" podStartSLOduration=1.5587861090000001 podStartE2EDuration="1.558786109s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:40:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:40:28.55692129 +0000 UTC m=+1908.115716622" watchObservedRunningTime="2026-07-06 22:40:28.558786109 +0000 UTC m=+1908.117581431" Jul 06 22:40:28.976631 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:28.976600 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-5d913f-openshift-default-5d756c7c6c-nfcfz" Jul 06 22:40:29.975033 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:29.974999 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-5d913f-openshift-default-5d756c7c6c-nfcfz" Jul 06 22:40:30.490863 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:30.490832 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:40:30.543334 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:30.543289 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-5d913f-openshift-default-5d756c7c6c-nfcfz" Jul 06 22:40:30.544568 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:30.544542 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-5d913f-openshift-default-5d756c7c6c-nfcfz" Jul 06 22:40:30.578754 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:30.578716 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:40:30.690268 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:30.690237 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:40:30.829601 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:30.829518 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:40:31.154125 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:31.154023 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:40:31.261370 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:31.261325 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:40:35.039858 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:35.039820 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:40:37.530271 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:37.530225 2581 patch_prober.go:28] interesting pod/e2e-inf-a-17b26a-openshift-default-9d9b7c8ff-g79kb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 06 22:40:37.530775 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:37.530308 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-17b26a-openshift-default-9d9b7c8ff-g79kb" podUID="76850ce2-1749-4478-8a5e-6ba64d7c9b80" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 06 22:40:38.593871 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:38.593783 2581 generic.go:356] "Generic (PLEG): container finished" podID="76850ce2-1749-4478-8a5e-6ba64d7c9b80" containerID="bf1e8bfa6dce95a2f20e6be629536ce1de3b34d7eb9dea57f70c9b97d0def0b6" exitCode=0 Jul 06 22:40:38.593871 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:38.593825 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-17b26a-openshift-default-9d9b7c8ff-g79kb" event={"ID":"76850ce2-1749-4478-8a5e-6ba64d7c9b80","Type":"ContainerDied","Data":"bf1e8bfa6dce95a2f20e6be629536ce1de3b34d7eb9dea57f70c9b97d0def0b6"} Jul 06 22:40:38.594368 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:38.594253 2581 scope.go:117] "RemoveContainer" containerID="bf1e8bfa6dce95a2f20e6be629536ce1de3b34d7eb9dea57f70c9b97d0def0b6" Jul 06 22:40:38.595370 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:38.595333 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 22:40:38.595441 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:38.595406 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 22:40:38.595477 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:38.595438 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 22:40:39.600691 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:39.600658 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-17b26a-openshift-default-9d9b7c8ff-g79kb" event={"ID":"76850ce2-1749-4478-8a5e-6ba64d7c9b80","Type":"ContainerStarted","Data":"9c5526d94e5ae73119225e5387ccebe4d9df93911f6c15dae3a578f551b9be18"} Jul 06 22:40:39.624925 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:39.624900 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-17b26a-openshift-default-9d9b7c8ff-g79kb" Jul 06 22:40:39.625100 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:39.624942 2581 patch_prober.go:28] interesting pod/e2e-inf-a-17b26a-openshift-default-9d9b7c8ff-g79kb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Jul 06 22:40:39.625100 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:39.624993 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-17b26a-openshift-default-9d9b7c8ff-g79kb" podUID="76850ce2-1749-4478-8a5e-6ba64d7c9b80" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Jul 06 22:40:40.624815 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:40.624785 2581 patch_prober.go:28] interesting pod/e2e-inf-a-17b26a-openshift-default-9d9b7c8ff-g79kb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Jul 06 22:40:40.625221 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:40.624835 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-17b26a-openshift-default-9d9b7c8ff-g79kb" podUID="76850ce2-1749-4478-8a5e-6ba64d7c9b80" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Jul 06 22:40:41.625394 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:41.625354 2581 patch_prober.go:28] interesting pod/e2e-inf-a-17b26a-openshift-default-9d9b7c8ff-g79kb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Jul 06 22:40:41.625786 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:41.625420 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-17b26a-openshift-default-9d9b7c8ff-g79kb" podUID="76850ce2-1749-4478-8a5e-6ba64d7c9b80" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Jul 06 22:40:42.625548 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:42.625513 2581 patch_prober.go:28] interesting pod/e2e-inf-a-17b26a-openshift-default-9d9b7c8ff-g79kb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Jul 06 22:40:42.625939 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:42.625590 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-17b26a-openshift-default-9d9b7c8ff-g79kb" podUID="76850ce2-1749-4478-8a5e-6ba64d7c9b80" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Jul 06 22:40:43.628559 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:43.628531 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-17b26a-openshift-default-9d9b7c8ff-g79kb" Jul 06 22:40:43.629034 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:43.628985 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-17b26a-openshift-default-9d9b7c8ff-g79kb" Jul 06 22:40:43.629774 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:43.629758 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-17b26a-openshift-default-9d9b7c8ff-g79kb" Jul 06 22:40:44.766060 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:44.766009 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:40:44.866780 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:44.866745 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:40:45.695920 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:45.694621 2581 patch_prober.go:28] interesting pod/e2e-inf-b-5d913f-openshift-default-5d756c7c6c-nfcfz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": read tcp 10.132.0.2:37724->10.132.0.62:15021: read: connection reset by peer" start-of-body= Jul 06 22:40:45.695920 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:45.694682 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-5d913f-openshift-default-5d756c7c6c-nfcfz" podUID="69185b8f-db46-4b55-85a1-27bfc2027672" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": read tcp 10.132.0.2:37724->10.132.0.62:15021: read: connection reset by peer" Jul 06 22:40:46.642962 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:46.642928 2581 generic.go:356] "Generic (PLEG): container finished" podID="69185b8f-db46-4b55-85a1-27bfc2027672" containerID="999e90e300f2958a66e56f2499a47d377a72f26a04e39c63d0c71456d15fa4b9" exitCode=0 Jul 06 22:40:46.643381 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:46.642954 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-5d913f-openshift-default-5d756c7c6c-nfcfz" event={"ID":"69185b8f-db46-4b55-85a1-27bfc2027672","Type":"ContainerDied","Data":"999e90e300f2958a66e56f2499a47d377a72f26a04e39c63d0c71456d15fa4b9"} Jul 06 22:40:46.643425 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:46.643415 2581 scope.go:117] "RemoveContainer" containerID="999e90e300f2958a66e56f2499a47d377a72f26a04e39c63d0c71456d15fa4b9" Jul 06 22:40:46.644452 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:46.644426 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 22:40:46.644510 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:46.644490 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 22:40:46.644552 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:46.644521 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 22:40:47.653433 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:47.653390 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-5d913f-openshift-default-5d756c7c6c-nfcfz" event={"ID":"69185b8f-db46-4b55-85a1-27bfc2027672","Type":"ContainerStarted","Data":"09dac622fc9323a9a5253f63d2035a5a09622b233f800544ab2dc8bb48179ecc"} Jul 06 22:40:47.969499 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:47.969455 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-5d913f-openshift-default-5d756c7c6c-nfcfz" Jul 06 22:40:47.969698 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:47.969653 2581 patch_prober.go:28] interesting pod/e2e-inf-b-5d913f-openshift-default-5d756c7c6c-nfcfz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body= Jul 06 22:40:47.969767 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:47.969713 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-5d913f-openshift-default-5d756c7c6c-nfcfz" podUID="69185b8f-db46-4b55-85a1-27bfc2027672" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" Jul 06 22:40:48.256214 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:48.256085 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:40:48.969807 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:48.969771 2581 patch_prober.go:28] interesting pod/e2e-inf-b-5d913f-openshift-default-5d756c7c6c-nfcfz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body= Jul 06 22:40:48.970254 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:48.969841 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-5d913f-openshift-default-5d756c7c6c-nfcfz" podUID="69185b8f-db46-4b55-85a1-27bfc2027672" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" Jul 06 22:40:49.970324 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:49.970288 2581 patch_prober.go:28] interesting pod/e2e-inf-b-5d913f-openshift-default-5d756c7c6c-nfcfz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body= Jul 06 22:40:49.970720 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:49.970342 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-5d913f-openshift-default-5d756c7c6c-nfcfz" podUID="69185b8f-db46-4b55-85a1-27bfc2027672" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" Jul 06 22:40:50.969623 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:50.969584 2581 patch_prober.go:28] interesting pod/e2e-inf-b-5d913f-openshift-default-5d756c7c6c-nfcfz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body= Jul 06 22:40:50.969834 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:50.969637 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-5d913f-openshift-default-5d756c7c6c-nfcfz" podUID="69185b8f-db46-4b55-85a1-27bfc2027672" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" Jul 06 22:40:51.973526 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:51.973494 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-5d913f-openshift-default-5d756c7c6c-nfcfz" Jul 06 22:40:51.973920 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:51.973824 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-5d913f-openshift-default-5d756c7c6c-nfcfz" Jul 06 22:40:51.974713 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:51.974694 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-5d913f-openshift-default-5d756c7c6c-nfcfz" Jul 06 22:40:52.723083 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:52.723041 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:40:52.828379 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:40:52.828351 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:41:08.194285 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:41:08.194238 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:41:33.524302 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:41:33.524223 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:41:34.469490 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:41:34.469460 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:41:34.621850 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:41:34.621814 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:41:35.033917 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:41:35.033871 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:41:35.133474 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:41:35.133441 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:41:35.233743 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:41:35.233696 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:41:35.300289 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:41:35.300216 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:41:40.111402 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:41:40.111370 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:42:16.906716 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:16.906638 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:42:17.177539 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:17.177505 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-17b26a-openshift-default-9d9b7c8ff-g79kb"] Jul 06 22:42:17.177794 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:17.177770 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-17b26a-openshift-default-9d9b7c8ff-g79kb" podUID="76850ce2-1749-4478-8a5e-6ba64d7c9b80" containerName="istio-proxy" containerID="cri-o://9c5526d94e5ae73119225e5387ccebe4d9df93911f6c15dae3a578f551b9be18" gracePeriod=30 Jul 06 22:42:17.204995 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:17.204966 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:42:17.448982 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:17.448894 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:42:17.711461 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:17.711375 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:42:18.146571 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:18.146482 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:42:18.246469 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:18.246432 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:42:18.347801 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:18.347769 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:42:18.479417 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:18.479382 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:42:22.439085 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:22.438995 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-17b26a-openshift-default-9d9b7c8ff-g79kb" Jul 06 22:42:22.501465 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:22.501434 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/76850ce2-1749-4478-8a5e-6ba64d7c9b80-credential-socket\") pod \"76850ce2-1749-4478-8a5e-6ba64d7c9b80\" (UID: \"76850ce2-1749-4478-8a5e-6ba64d7c9b80\") " Jul 06 22:42:22.501644 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:22.501493 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/76850ce2-1749-4478-8a5e-6ba64d7c9b80-istiod-ca-cert\") pod \"76850ce2-1749-4478-8a5e-6ba64d7c9b80\" (UID: \"76850ce2-1749-4478-8a5e-6ba64d7c9b80\") " Jul 06 22:42:22.501644 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:22.501536 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/76850ce2-1749-4478-8a5e-6ba64d7c9b80-istio-envoy\") pod \"76850ce2-1749-4478-8a5e-6ba64d7c9b80\" (UID: \"76850ce2-1749-4478-8a5e-6ba64d7c9b80\") " Jul 06 22:42:22.501644 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:22.501560 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/76850ce2-1749-4478-8a5e-6ba64d7c9b80-workload-certs\") pod \"76850ce2-1749-4478-8a5e-6ba64d7c9b80\" (UID: \"76850ce2-1749-4478-8a5e-6ba64d7c9b80\") " Jul 06 22:42:22.501644 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:22.501612 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/76850ce2-1749-4478-8a5e-6ba64d7c9b80-istio-token\") pod \"76850ce2-1749-4478-8a5e-6ba64d7c9b80\" (UID: \"76850ce2-1749-4478-8a5e-6ba64d7c9b80\") " Jul 06 22:42:22.501644 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:22.501645 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/76850ce2-1749-4478-8a5e-6ba64d7c9b80-workload-socket\") pod \"76850ce2-1749-4478-8a5e-6ba64d7c9b80\" (UID: \"76850ce2-1749-4478-8a5e-6ba64d7c9b80\") " Jul 06 22:42:22.501916 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:22.501676 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/76850ce2-1749-4478-8a5e-6ba64d7c9b80-istio-data\") pod \"76850ce2-1749-4478-8a5e-6ba64d7c9b80\" (UID: \"76850ce2-1749-4478-8a5e-6ba64d7c9b80\") " Jul 06 22:42:22.501916 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:22.501705 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b9k87\" (UniqueName: \"kubernetes.io/projected/76850ce2-1749-4478-8a5e-6ba64d7c9b80-kube-api-access-b9k87\") pod \"76850ce2-1749-4478-8a5e-6ba64d7c9b80\" (UID: \"76850ce2-1749-4478-8a5e-6ba64d7c9b80\") " Jul 06 22:42:22.501916 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:22.501699 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76850ce2-1749-4478-8a5e-6ba64d7c9b80-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "76850ce2-1749-4478-8a5e-6ba64d7c9b80" (UID: "76850ce2-1749-4478-8a5e-6ba64d7c9b80"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:42:22.501916 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:22.501869 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76850ce2-1749-4478-8a5e-6ba64d7c9b80-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "76850ce2-1749-4478-8a5e-6ba64d7c9b80" (UID: "76850ce2-1749-4478-8a5e-6ba64d7c9b80"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:42:22.502157 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:22.501951 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/76850ce2-1749-4478-8a5e-6ba64d7c9b80-istio-podinfo\") pod \"76850ce2-1749-4478-8a5e-6ba64d7c9b80\" (UID: \"76850ce2-1749-4478-8a5e-6ba64d7c9b80\") " Jul 06 22:42:22.502157 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:22.501981 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76850ce2-1749-4478-8a5e-6ba64d7c9b80-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "76850ce2-1749-4478-8a5e-6ba64d7c9b80" (UID: "76850ce2-1749-4478-8a5e-6ba64d7c9b80"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:42:22.502157 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:22.502030 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/76850ce2-1749-4478-8a5e-6ba64d7c9b80-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "76850ce2-1749-4478-8a5e-6ba64d7c9b80" (UID: "76850ce2-1749-4478-8a5e-6ba64d7c9b80"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 22:42:22.502395 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:22.502374 2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/76850ce2-1749-4478-8a5e-6ba64d7c9b80-workload-certs\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:42:22.502444 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:22.502401 2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/76850ce2-1749-4478-8a5e-6ba64d7c9b80-workload-socket\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:42:22.502444 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:22.502414 2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/76850ce2-1749-4478-8a5e-6ba64d7c9b80-credential-socket\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:42:22.502444 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:22.502429 2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/76850ce2-1749-4478-8a5e-6ba64d7c9b80-istiod-ca-cert\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:42:22.502750 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:22.502727 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76850ce2-1749-4478-8a5e-6ba64d7c9b80-istio-data" (OuterVolumeSpecName: "istio-data") pod "76850ce2-1749-4478-8a5e-6ba64d7c9b80" (UID: "76850ce2-1749-4478-8a5e-6ba64d7c9b80"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:42:22.503899 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:22.503866 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/76850ce2-1749-4478-8a5e-6ba64d7c9b80-istio-token" (OuterVolumeSpecName: "istio-token") pod "76850ce2-1749-4478-8a5e-6ba64d7c9b80" (UID: "76850ce2-1749-4478-8a5e-6ba64d7c9b80"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 22:42:22.503899 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:22.503883 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/76850ce2-1749-4478-8a5e-6ba64d7c9b80-kube-api-access-b9k87" (OuterVolumeSpecName: "kube-api-access-b9k87") pod "76850ce2-1749-4478-8a5e-6ba64d7c9b80" (UID: "76850ce2-1749-4478-8a5e-6ba64d7c9b80"). InnerVolumeSpecName "kube-api-access-b9k87". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 22:42:22.504129 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:22.504109 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76850ce2-1749-4478-8a5e-6ba64d7c9b80-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "76850ce2-1749-4478-8a5e-6ba64d7c9b80" (UID: "76850ce2-1749-4478-8a5e-6ba64d7c9b80"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:42:22.504215 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:22.504195 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/76850ce2-1749-4478-8a5e-6ba64d7c9b80-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "76850ce2-1749-4478-8a5e-6ba64d7c9b80" (UID: "76850ce2-1749-4478-8a5e-6ba64d7c9b80"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 22:42:22.602860 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:22.602827 2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/76850ce2-1749-4478-8a5e-6ba64d7c9b80-istio-envoy\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:42:22.602860 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:22.602852 2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/76850ce2-1749-4478-8a5e-6ba64d7c9b80-istio-token\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:42:22.602860 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:22.602861 2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/76850ce2-1749-4478-8a5e-6ba64d7c9b80-istio-data\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:42:22.602860 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:22.602869 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b9k87\" (UniqueName: \"kubernetes.io/projected/76850ce2-1749-4478-8a5e-6ba64d7c9b80-kube-api-access-b9k87\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:42:22.603146 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:22.602878 2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/76850ce2-1749-4478-8a5e-6ba64d7c9b80-istio-podinfo\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:42:23.146303 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:23.146265 2581 generic.go:356] "Generic (PLEG): container finished" podID="76850ce2-1749-4478-8a5e-6ba64d7c9b80" containerID="9c5526d94e5ae73119225e5387ccebe4d9df93911f6c15dae3a578f551b9be18" exitCode=0 Jul 06 22:42:23.146478 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:23.146316 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-17b26a-openshift-default-9d9b7c8ff-g79kb" Jul 06 22:42:23.146478 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:23.146352 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-17b26a-openshift-default-9d9b7c8ff-g79kb" event={"ID":"76850ce2-1749-4478-8a5e-6ba64d7c9b80","Type":"ContainerDied","Data":"9c5526d94e5ae73119225e5387ccebe4d9df93911f6c15dae3a578f551b9be18"} Jul 06 22:42:23.146478 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:23.146404 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-17b26a-openshift-default-9d9b7c8ff-g79kb" event={"ID":"76850ce2-1749-4478-8a5e-6ba64d7c9b80","Type":"ContainerDied","Data":"b63269d251e7cf47c4c437272ec2c021d27d6d5a4af9a1c033ba6037bf8010e5"} Jul 06 22:42:23.146478 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:23.146425 2581 scope.go:117] "RemoveContainer" containerID="9c5526d94e5ae73119225e5387ccebe4d9df93911f6c15dae3a578f551b9be18" Jul 06 22:42:23.156763 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:23.156745 2581 scope.go:117] "RemoveContainer" containerID="bf1e8bfa6dce95a2f20e6be629536ce1de3b34d7eb9dea57f70c9b97d0def0b6" Jul 06 22:42:23.165961 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:23.164514 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-17b26a-openshift-default-9d9b7c8ff-g79kb"] Jul 06 22:42:23.168210 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:23.168189 2581 scope.go:117] "RemoveContainer" containerID="9c5526d94e5ae73119225e5387ccebe4d9df93911f6c15dae3a578f551b9be18" Jul 06 22:42:23.168521 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:42:23.168501 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9c5526d94e5ae73119225e5387ccebe4d9df93911f6c15dae3a578f551b9be18\": container with ID starting with 9c5526d94e5ae73119225e5387ccebe4d9df93911f6c15dae3a578f551b9be18 not found: ID does not exist" containerID="9c5526d94e5ae73119225e5387ccebe4d9df93911f6c15dae3a578f551b9be18" Jul 06 22:42:23.168576 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:23.168530 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9c5526d94e5ae73119225e5387ccebe4d9df93911f6c15dae3a578f551b9be18"} err="failed to get container status \"9c5526d94e5ae73119225e5387ccebe4d9df93911f6c15dae3a578f551b9be18\": rpc error: code = NotFound desc = could not find container \"9c5526d94e5ae73119225e5387ccebe4d9df93911f6c15dae3a578f551b9be18\": container with ID starting with 9c5526d94e5ae73119225e5387ccebe4d9df93911f6c15dae3a578f551b9be18 not found: ID does not exist" Jul 06 22:42:23.168576 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:23.168548 2581 scope.go:117] "RemoveContainer" containerID="bf1e8bfa6dce95a2f20e6be629536ce1de3b34d7eb9dea57f70c9b97d0def0b6" Jul 06 22:42:23.168821 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:42:23.168798 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bf1e8bfa6dce95a2f20e6be629536ce1de3b34d7eb9dea57f70c9b97d0def0b6\": container with ID starting with bf1e8bfa6dce95a2f20e6be629536ce1de3b34d7eb9dea57f70c9b97d0def0b6 not found: ID does not exist" containerID="bf1e8bfa6dce95a2f20e6be629536ce1de3b34d7eb9dea57f70c9b97d0def0b6" Jul 06 22:42:23.168870 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:23.168831 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf1e8bfa6dce95a2f20e6be629536ce1de3b34d7eb9dea57f70c9b97d0def0b6"} err="failed to get container status \"bf1e8bfa6dce95a2f20e6be629536ce1de3b34d7eb9dea57f70c9b97d0def0b6\": rpc error: code = NotFound desc = could not find container \"bf1e8bfa6dce95a2f20e6be629536ce1de3b34d7eb9dea57f70c9b97d0def0b6\": container with ID starting with bf1e8bfa6dce95a2f20e6be629536ce1de3b34d7eb9dea57f70c9b97d0def0b6 not found: ID does not exist" Jul 06 22:42:23.168959 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:23.168944 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-17b26a-openshift-default-9d9b7c8ff-g79kb"] Jul 06 22:42:24.207686 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:24.207660 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:42:24.271092 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:24.271040 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:42:24.336763 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:24.336727 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:42:24.454276 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:24.454245 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-5d913f-openshift-default-5d756c7c6c-nfcfz"] Jul 06 22:42:24.454486 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:24.454462 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-5d913f-openshift-default-5d756c7c6c-nfcfz" podUID="69185b8f-db46-4b55-85a1-27bfc2027672" containerName="istio-proxy" containerID="cri-o://09dac622fc9323a9a5253f63d2035a5a09622b233f800544ab2dc8bb48179ecc" gracePeriod=30 Jul 06 22:42:24.474945 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:24.474875 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:42:24.692082 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:24.692042 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:42:24.978145 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:24.978111 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="76850ce2-1749-4478-8a5e-6ba64d7c9b80" path="/var/lib/kubelet/pods/76850ce2-1749-4478-8a5e-6ba64d7c9b80/volumes" Jul 06 22:42:25.035456 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:25.035423 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:42:25.412802 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:25.412721 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-fbe640-openshift-default-6b9cb89c65-45gx6"] Jul 06 22:42:25.413183 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:25.413168 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="76850ce2-1749-4478-8a5e-6ba64d7c9b80" containerName="istio-proxy" Jul 06 22:42:25.413231 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:25.413185 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="76850ce2-1749-4478-8a5e-6ba64d7c9b80" containerName="istio-proxy" Jul 06 22:42:25.413231 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:25.413200 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="76850ce2-1749-4478-8a5e-6ba64d7c9b80" containerName="istio-proxy" Jul 06 22:42:25.413231 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:25.413205 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="76850ce2-1749-4478-8a5e-6ba64d7c9b80" containerName="istio-proxy" Jul 06 22:42:25.413328 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:25.413283 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="76850ce2-1749-4478-8a5e-6ba64d7c9b80" containerName="istio-proxy" Jul 06 22:42:25.413328 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:25.413293 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="76850ce2-1749-4478-8a5e-6ba64d7c9b80" containerName="istio-proxy" Jul 06 22:42:25.417842 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:25.417824 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-fbe640-openshift-default-6b9cb89c65-45gx6" Jul 06 22:42:25.420518 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:25.420497 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-fbe640-openshift-default-dockercfg-w7vzf\"" Jul 06 22:42:25.427944 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:25.427923 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-fbe640-openshift-default-6b9cb89c65-45gx6"] Jul 06 22:42:25.441685 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:25.441652 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:42:25.541051 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:25.541004 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca959e8b-6125-4469-9a30-727411fa7c8f-workload-socket\") pod \"e2e-api-a-fbe640-openshift-default-6b9cb89c65-45gx6\" (UID: \"ca959e8b-6125-4469-9a30-727411fa7c8f\") " pod="openshift-ingress/e2e-api-a-fbe640-openshift-default-6b9cb89c65-45gx6" Jul 06 22:42:25.541258 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:25.541111 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ca959e8b-6125-4469-9a30-727411fa7c8f-istio-token\") pod \"e2e-api-a-fbe640-openshift-default-6b9cb89c65-45gx6\" (UID: \"ca959e8b-6125-4469-9a30-727411fa7c8f\") " pod="openshift-ingress/e2e-api-a-fbe640-openshift-default-6b9cb89c65-45gx6" Jul 06 22:42:25.541258 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:25.541144 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ca959e8b-6125-4469-9a30-727411fa7c8f-istiod-ca-cert\") pod \"e2e-api-a-fbe640-openshift-default-6b9cb89c65-45gx6\" (UID: \"ca959e8b-6125-4469-9a30-727411fa7c8f\") " pod="openshift-ingress/e2e-api-a-fbe640-openshift-default-6b9cb89c65-45gx6" Jul 06 22:42:25.541258 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:25.541174 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ca959e8b-6125-4469-9a30-727411fa7c8f-istio-envoy\") pod \"e2e-api-a-fbe640-openshift-default-6b9cb89c65-45gx6\" (UID: \"ca959e8b-6125-4469-9a30-727411fa7c8f\") " pod="openshift-ingress/e2e-api-a-fbe640-openshift-default-6b9cb89c65-45gx6" Jul 06 22:42:25.541258 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:25.541243 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ca959e8b-6125-4469-9a30-727411fa7c8f-workload-certs\") pod \"e2e-api-a-fbe640-openshift-default-6b9cb89c65-45gx6\" (UID: \"ca959e8b-6125-4469-9a30-727411fa7c8f\") " pod="openshift-ingress/e2e-api-a-fbe640-openshift-default-6b9cb89c65-45gx6" Jul 06 22:42:25.541463 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:25.541279 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f5fhs\" (UniqueName: \"kubernetes.io/projected/ca959e8b-6125-4469-9a30-727411fa7c8f-kube-api-access-f5fhs\") pod \"e2e-api-a-fbe640-openshift-default-6b9cb89c65-45gx6\" (UID: \"ca959e8b-6125-4469-9a30-727411fa7c8f\") " pod="openshift-ingress/e2e-api-a-fbe640-openshift-default-6b9cb89c65-45gx6" Jul 06 22:42:25.541463 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:25.541315 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ca959e8b-6125-4469-9a30-727411fa7c8f-istio-podinfo\") pod \"e2e-api-a-fbe640-openshift-default-6b9cb89c65-45gx6\" (UID: \"ca959e8b-6125-4469-9a30-727411fa7c8f\") " pod="openshift-ingress/e2e-api-a-fbe640-openshift-default-6b9cb89c65-45gx6" Jul 06 22:42:25.541463 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:25.541379 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ca959e8b-6125-4469-9a30-727411fa7c8f-istio-data\") pod \"e2e-api-a-fbe640-openshift-default-6b9cb89c65-45gx6\" (UID: \"ca959e8b-6125-4469-9a30-727411fa7c8f\") " pod="openshift-ingress/e2e-api-a-fbe640-openshift-default-6b9cb89c65-45gx6" Jul 06 22:42:25.541463 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:25.541413 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca959e8b-6125-4469-9a30-727411fa7c8f-credential-socket\") pod \"e2e-api-a-fbe640-openshift-default-6b9cb89c65-45gx6\" (UID: \"ca959e8b-6125-4469-9a30-727411fa7c8f\") " pod="openshift-ingress/e2e-api-a-fbe640-openshift-default-6b9cb89c65-45gx6" Jul 06 22:42:25.642246 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:25.642205 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ca959e8b-6125-4469-9a30-727411fa7c8f-istio-token\") pod \"e2e-api-a-fbe640-openshift-default-6b9cb89c65-45gx6\" (UID: \"ca959e8b-6125-4469-9a30-727411fa7c8f\") " pod="openshift-ingress/e2e-api-a-fbe640-openshift-default-6b9cb89c65-45gx6" Jul 06 22:42:25.642246 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:25.642243 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ca959e8b-6125-4469-9a30-727411fa7c8f-istiod-ca-cert\") pod \"e2e-api-a-fbe640-openshift-default-6b9cb89c65-45gx6\" (UID: \"ca959e8b-6125-4469-9a30-727411fa7c8f\") " pod="openshift-ingress/e2e-api-a-fbe640-openshift-default-6b9cb89c65-45gx6" Jul 06 22:42:25.642486 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:25.642264 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ca959e8b-6125-4469-9a30-727411fa7c8f-istio-envoy\") pod \"e2e-api-a-fbe640-openshift-default-6b9cb89c65-45gx6\" (UID: \"ca959e8b-6125-4469-9a30-727411fa7c8f\") " pod="openshift-ingress/e2e-api-a-fbe640-openshift-default-6b9cb89c65-45gx6" Jul 06 22:42:25.642486 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:25.642301 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ca959e8b-6125-4469-9a30-727411fa7c8f-workload-certs\") pod \"e2e-api-a-fbe640-openshift-default-6b9cb89c65-45gx6\" (UID: \"ca959e8b-6125-4469-9a30-727411fa7c8f\") " pod="openshift-ingress/e2e-api-a-fbe640-openshift-default-6b9cb89c65-45gx6" Jul 06 22:42:25.642486 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:25.642331 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f5fhs\" (UniqueName: \"kubernetes.io/projected/ca959e8b-6125-4469-9a30-727411fa7c8f-kube-api-access-f5fhs\") pod \"e2e-api-a-fbe640-openshift-default-6b9cb89c65-45gx6\" (UID: \"ca959e8b-6125-4469-9a30-727411fa7c8f\") " pod="openshift-ingress/e2e-api-a-fbe640-openshift-default-6b9cb89c65-45gx6" Jul 06 22:42:25.642486 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:25.642362 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ca959e8b-6125-4469-9a30-727411fa7c8f-istio-podinfo\") pod \"e2e-api-a-fbe640-openshift-default-6b9cb89c65-45gx6\" (UID: \"ca959e8b-6125-4469-9a30-727411fa7c8f\") " pod="openshift-ingress/e2e-api-a-fbe640-openshift-default-6b9cb89c65-45gx6" Jul 06 22:42:25.642486 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:25.642426 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ca959e8b-6125-4469-9a30-727411fa7c8f-istio-data\") pod \"e2e-api-a-fbe640-openshift-default-6b9cb89c65-45gx6\" (UID: \"ca959e8b-6125-4469-9a30-727411fa7c8f\") " pod="openshift-ingress/e2e-api-a-fbe640-openshift-default-6b9cb89c65-45gx6" Jul 06 22:42:25.642486 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:25.642471 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca959e8b-6125-4469-9a30-727411fa7c8f-credential-socket\") pod \"e2e-api-a-fbe640-openshift-default-6b9cb89c65-45gx6\" (UID: \"ca959e8b-6125-4469-9a30-727411fa7c8f\") " pod="openshift-ingress/e2e-api-a-fbe640-openshift-default-6b9cb89c65-45gx6" Jul 06 22:42:25.642801 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:25.642520 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca959e8b-6125-4469-9a30-727411fa7c8f-workload-socket\") pod \"e2e-api-a-fbe640-openshift-default-6b9cb89c65-45gx6\" (UID: \"ca959e8b-6125-4469-9a30-727411fa7c8f\") " pod="openshift-ingress/e2e-api-a-fbe640-openshift-default-6b9cb89c65-45gx6" Jul 06 22:42:25.642801 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:25.642777 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ca959e8b-6125-4469-9a30-727411fa7c8f-workload-certs\") pod \"e2e-api-a-fbe640-openshift-default-6b9cb89c65-45gx6\" (UID: \"ca959e8b-6125-4469-9a30-727411fa7c8f\") " pod="openshift-ingress/e2e-api-a-fbe640-openshift-default-6b9cb89c65-45gx6" Jul 06 22:42:25.642902 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:25.642832 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ca959e8b-6125-4469-9a30-727411fa7c8f-istio-data\") pod \"e2e-api-a-fbe640-openshift-default-6b9cb89c65-45gx6\" (UID: \"ca959e8b-6125-4469-9a30-727411fa7c8f\") " pod="openshift-ingress/e2e-api-a-fbe640-openshift-default-6b9cb89c65-45gx6" Jul 06 22:42:25.642902 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:25.642876 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca959e8b-6125-4469-9a30-727411fa7c8f-workload-socket\") pod \"e2e-api-a-fbe640-openshift-default-6b9cb89c65-45gx6\" (UID: \"ca959e8b-6125-4469-9a30-727411fa7c8f\") " pod="openshift-ingress/e2e-api-a-fbe640-openshift-default-6b9cb89c65-45gx6" Jul 06 22:42:25.643008 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:25.642989 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca959e8b-6125-4469-9a30-727411fa7c8f-credential-socket\") pod \"e2e-api-a-fbe640-openshift-default-6b9cb89c65-45gx6\" (UID: \"ca959e8b-6125-4469-9a30-727411fa7c8f\") " pod="openshift-ingress/e2e-api-a-fbe640-openshift-default-6b9cb89c65-45gx6" Jul 06 22:42:25.643111 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:25.643052 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ca959e8b-6125-4469-9a30-727411fa7c8f-istiod-ca-cert\") pod \"e2e-api-a-fbe640-openshift-default-6b9cb89c65-45gx6\" (UID: \"ca959e8b-6125-4469-9a30-727411fa7c8f\") " pod="openshift-ingress/e2e-api-a-fbe640-openshift-default-6b9cb89c65-45gx6" Jul 06 22:42:25.645118 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:25.645096 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ca959e8b-6125-4469-9a30-727411fa7c8f-istio-envoy\") pod \"e2e-api-a-fbe640-openshift-default-6b9cb89c65-45gx6\" (UID: \"ca959e8b-6125-4469-9a30-727411fa7c8f\") " pod="openshift-ingress/e2e-api-a-fbe640-openshift-default-6b9cb89c65-45gx6" Jul 06 22:42:25.645294 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:25.645271 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ca959e8b-6125-4469-9a30-727411fa7c8f-istio-podinfo\") pod \"e2e-api-a-fbe640-openshift-default-6b9cb89c65-45gx6\" (UID: \"ca959e8b-6125-4469-9a30-727411fa7c8f\") " pod="openshift-ingress/e2e-api-a-fbe640-openshift-default-6b9cb89c65-45gx6" Jul 06 22:42:25.649854 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:25.649824 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ca959e8b-6125-4469-9a30-727411fa7c8f-istio-token\") pod \"e2e-api-a-fbe640-openshift-default-6b9cb89c65-45gx6\" (UID: \"ca959e8b-6125-4469-9a30-727411fa7c8f\") " pod="openshift-ingress/e2e-api-a-fbe640-openshift-default-6b9cb89c65-45gx6" Jul 06 22:42:25.649962 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:25.649919 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f5fhs\" (UniqueName: \"kubernetes.io/projected/ca959e8b-6125-4469-9a30-727411fa7c8f-kube-api-access-f5fhs\") pod \"e2e-api-a-fbe640-openshift-default-6b9cb89c65-45gx6\" (UID: \"ca959e8b-6125-4469-9a30-727411fa7c8f\") " pod="openshift-ingress/e2e-api-a-fbe640-openshift-default-6b9cb89c65-45gx6" Jul 06 22:42:25.678007 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:25.677964 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:42:25.728451 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:25.728424 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-fbe640-openshift-default-6b9cb89c65-45gx6" Jul 06 22:42:25.874322 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:25.874293 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-fbe640-openshift-default-6b9cb89c65-45gx6"] Jul 06 22:42:25.876363 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:42:25.876339 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca959e8b_6125_4469_9a30_727411fa7c8f.slice/crio-ee411fed06e2516723ab9d15b04ffdf394039c5789c2ed6f2fd01b42bc982007 WatchSource:0}: Error finding container ee411fed06e2516723ab9d15b04ffdf394039c5789c2ed6f2fd01b42bc982007: Status 404 returned error can't find the container with id ee411fed06e2516723ab9d15b04ffdf394039c5789c2ed6f2fd01b42bc982007 Jul 06 22:42:25.878549 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:25.878506 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 22:42:25.878633 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:25.878598 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 22:42:25.878675 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:25.878639 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 22:42:26.163355 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:26.163322 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-fbe640-openshift-default-6b9cb89c65-45gx6" event={"ID":"ca959e8b-6125-4469-9a30-727411fa7c8f","Type":"ContainerStarted","Data":"0d62aef909c58c76bcef114f4f1a8148d6ebaa960735e61f1cc9173e15b991db"} Jul 06 22:42:26.163552 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:26.163379 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-fbe640-openshift-default-6b9cb89c65-45gx6" event={"ID":"ca959e8b-6125-4469-9a30-727411fa7c8f","Type":"ContainerStarted","Data":"ee411fed06e2516723ab9d15b04ffdf394039c5789c2ed6f2fd01b42bc982007"} Jul 06 22:42:26.183408 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:26.183296 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-fbe640-openshift-default-6b9cb89c65-45gx6" podStartSLOduration=1.183274983 podStartE2EDuration="1.183274983s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:42:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:42:26.182342107 +0000 UTC m=+2025.741137439" watchObservedRunningTime="2026-07-06 22:42:26.183274983 +0000 UTC m=+2025.742070306" Jul 06 22:42:26.729140 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:26.729104 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-fbe640-openshift-default-6b9cb89c65-45gx6" Jul 06 22:42:26.735370 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:26.735346 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-fbe640-openshift-default-6b9cb89c65-45gx6" Jul 06 22:42:27.168851 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:27.168767 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-fbe640-openshift-default-6b9cb89c65-45gx6" Jul 06 22:42:27.169819 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:27.169795 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-fbe640-openshift-default-6b9cb89c65-45gx6" Jul 06 22:42:28.091453 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:28.091419 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:42:28.184351 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:28.184319 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:42:28.263219 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:28.263184 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:42:28.362606 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:28.362527 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:42:28.623935 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:28.623843 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:42:28.725210 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:28.725152 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:42:29.702223 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:29.702202 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-5d913f-openshift-default-5d756c7c6c-nfcfz" Jul 06 22:42:29.780880 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:29.780803 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/69185b8f-db46-4b55-85a1-27bfc2027672-istiod-ca-cert\") pod \"69185b8f-db46-4b55-85a1-27bfc2027672\" (UID: \"69185b8f-db46-4b55-85a1-27bfc2027672\") " Jul 06 22:42:29.780880 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:29.780849 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/69185b8f-db46-4b55-85a1-27bfc2027672-istio-data\") pod \"69185b8f-db46-4b55-85a1-27bfc2027672\" (UID: \"69185b8f-db46-4b55-85a1-27bfc2027672\") " Jul 06 22:42:29.780880 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:29.780873 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/69185b8f-db46-4b55-85a1-27bfc2027672-istio-podinfo\") pod \"69185b8f-db46-4b55-85a1-27bfc2027672\" (UID: \"69185b8f-db46-4b55-85a1-27bfc2027672\") " Jul 06 22:42:29.781188 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:29.780895 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/69185b8f-db46-4b55-85a1-27bfc2027672-workload-socket\") pod \"69185b8f-db46-4b55-85a1-27bfc2027672\" (UID: \"69185b8f-db46-4b55-85a1-27bfc2027672\") " Jul 06 22:42:29.781188 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:29.780921 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/69185b8f-db46-4b55-85a1-27bfc2027672-istio-envoy\") pod \"69185b8f-db46-4b55-85a1-27bfc2027672\" (UID: \"69185b8f-db46-4b55-85a1-27bfc2027672\") " Jul 06 22:42:29.781188 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:29.780946 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/69185b8f-db46-4b55-85a1-27bfc2027672-istio-token\") pod \"69185b8f-db46-4b55-85a1-27bfc2027672\" (UID: \"69185b8f-db46-4b55-85a1-27bfc2027672\") " Jul 06 22:42:29.781188 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:29.780991 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/69185b8f-db46-4b55-85a1-27bfc2027672-workload-certs\") pod \"69185b8f-db46-4b55-85a1-27bfc2027672\" (UID: \"69185b8f-db46-4b55-85a1-27bfc2027672\") " Jul 06 22:42:29.781188 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:29.781102 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xdx58\" (UniqueName: \"kubernetes.io/projected/69185b8f-db46-4b55-85a1-27bfc2027672-kube-api-access-xdx58\") pod \"69185b8f-db46-4b55-85a1-27bfc2027672\" (UID: \"69185b8f-db46-4b55-85a1-27bfc2027672\") " Jul 06 22:42:29.781188 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:29.781163 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/69185b8f-db46-4b55-85a1-27bfc2027672-credential-socket\") pod \"69185b8f-db46-4b55-85a1-27bfc2027672\" (UID: \"69185b8f-db46-4b55-85a1-27bfc2027672\") " Jul 06 22:42:29.781464 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:29.781266 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69185b8f-db46-4b55-85a1-27bfc2027672-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "69185b8f-db46-4b55-85a1-27bfc2027672" (UID: "69185b8f-db46-4b55-85a1-27bfc2027672"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:42:29.781464 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:29.781319 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/69185b8f-db46-4b55-85a1-27bfc2027672-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "69185b8f-db46-4b55-85a1-27bfc2027672" (UID: "69185b8f-db46-4b55-85a1-27bfc2027672"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 22:42:29.781464 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:29.781349 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69185b8f-db46-4b55-85a1-27bfc2027672-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "69185b8f-db46-4b55-85a1-27bfc2027672" (UID: "69185b8f-db46-4b55-85a1-27bfc2027672"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:42:29.781464 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:29.781452 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69185b8f-db46-4b55-85a1-27bfc2027672-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "69185b8f-db46-4b55-85a1-27bfc2027672" (UID: "69185b8f-db46-4b55-85a1-27bfc2027672"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:42:29.781639 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:29.781572 2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/69185b8f-db46-4b55-85a1-27bfc2027672-credential-socket\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:42:29.781639 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:29.781597 2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/69185b8f-db46-4b55-85a1-27bfc2027672-istiod-ca-cert\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:42:29.781639 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:29.781612 2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/69185b8f-db46-4b55-85a1-27bfc2027672-workload-socket\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:42:29.781639 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:29.781628 2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/69185b8f-db46-4b55-85a1-27bfc2027672-workload-certs\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:42:29.781964 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:29.781939 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69185b8f-db46-4b55-85a1-27bfc2027672-istio-data" (OuterVolumeSpecName: "istio-data") pod "69185b8f-db46-4b55-85a1-27bfc2027672" (UID: "69185b8f-db46-4b55-85a1-27bfc2027672"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:42:29.783192 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:29.783171 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/69185b8f-db46-4b55-85a1-27bfc2027672-istio-token" (OuterVolumeSpecName: "istio-token") pod "69185b8f-db46-4b55-85a1-27bfc2027672" (UID: "69185b8f-db46-4b55-85a1-27bfc2027672"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 22:42:29.783284 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:29.783263 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69185b8f-db46-4b55-85a1-27bfc2027672-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "69185b8f-db46-4b55-85a1-27bfc2027672" (UID: "69185b8f-db46-4b55-85a1-27bfc2027672"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:42:29.783774 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:29.783752 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/69185b8f-db46-4b55-85a1-27bfc2027672-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "69185b8f-db46-4b55-85a1-27bfc2027672" (UID: "69185b8f-db46-4b55-85a1-27bfc2027672"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 22:42:29.783774 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:29.783762 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/69185b8f-db46-4b55-85a1-27bfc2027672-kube-api-access-xdx58" (OuterVolumeSpecName: "kube-api-access-xdx58") pod "69185b8f-db46-4b55-85a1-27bfc2027672" (UID: "69185b8f-db46-4b55-85a1-27bfc2027672"). InnerVolumeSpecName "kube-api-access-xdx58". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 22:42:29.882183 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:29.882146 2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/69185b8f-db46-4b55-85a1-27bfc2027672-istio-data\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:42:29.882183 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:29.882176 2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/69185b8f-db46-4b55-85a1-27bfc2027672-istio-podinfo\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:42:29.882183 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:29.882187 2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/69185b8f-db46-4b55-85a1-27bfc2027672-istio-envoy\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:42:29.882420 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:29.882196 2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/69185b8f-db46-4b55-85a1-27bfc2027672-istio-token\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:42:29.882420 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:29.882205 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xdx58\" (UniqueName: \"kubernetes.io/projected/69185b8f-db46-4b55-85a1-27bfc2027672-kube-api-access-xdx58\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:42:30.187527 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:30.187491 2581 generic.go:356] "Generic (PLEG): container finished" podID="69185b8f-db46-4b55-85a1-27bfc2027672" containerID="09dac622fc9323a9a5253f63d2035a5a09622b233f800544ab2dc8bb48179ecc" exitCode=0 Jul 06 22:42:30.187700 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:30.187539 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-5d913f-openshift-default-5d756c7c6c-nfcfz" Jul 06 22:42:30.187700 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:30.187578 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-5d913f-openshift-default-5d756c7c6c-nfcfz" event={"ID":"69185b8f-db46-4b55-85a1-27bfc2027672","Type":"ContainerDied","Data":"09dac622fc9323a9a5253f63d2035a5a09622b233f800544ab2dc8bb48179ecc"} Jul 06 22:42:30.187700 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:30.187634 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-5d913f-openshift-default-5d756c7c6c-nfcfz" event={"ID":"69185b8f-db46-4b55-85a1-27bfc2027672","Type":"ContainerDied","Data":"9578b0c1204a1f8648c2dac5fb1ff9f9dc31e6f964617a6c3f5ec4003922060e"} Jul 06 22:42:30.187700 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:30.187659 2581 scope.go:117] "RemoveContainer" containerID="09dac622fc9323a9a5253f63d2035a5a09622b233f800544ab2dc8bb48179ecc" Jul 06 22:42:30.196861 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:30.196843 2581 scope.go:117] "RemoveContainer" containerID="999e90e300f2958a66e56f2499a47d377a72f26a04e39c63d0c71456d15fa4b9" Jul 06 22:42:30.204831 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:30.204814 2581 scope.go:117] "RemoveContainer" containerID="09dac622fc9323a9a5253f63d2035a5a09622b233f800544ab2dc8bb48179ecc" Jul 06 22:42:30.205099 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:42:30.205080 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"09dac622fc9323a9a5253f63d2035a5a09622b233f800544ab2dc8bb48179ecc\": container with ID starting with 09dac622fc9323a9a5253f63d2035a5a09622b233f800544ab2dc8bb48179ecc not found: ID does not exist" containerID="09dac622fc9323a9a5253f63d2035a5a09622b233f800544ab2dc8bb48179ecc" Jul 06 22:42:30.205154 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:30.205107 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"09dac622fc9323a9a5253f63d2035a5a09622b233f800544ab2dc8bb48179ecc"} err="failed to get container status \"09dac622fc9323a9a5253f63d2035a5a09622b233f800544ab2dc8bb48179ecc\": rpc error: code = NotFound desc = could not find container \"09dac622fc9323a9a5253f63d2035a5a09622b233f800544ab2dc8bb48179ecc\": container with ID starting with 09dac622fc9323a9a5253f63d2035a5a09622b233f800544ab2dc8bb48179ecc not found: ID does not exist" Jul 06 22:42:30.205154 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:30.205125 2581 scope.go:117] "RemoveContainer" containerID="999e90e300f2958a66e56f2499a47d377a72f26a04e39c63d0c71456d15fa4b9" Jul 06 22:42:30.205369 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:42:30.205353 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"999e90e300f2958a66e56f2499a47d377a72f26a04e39c63d0c71456d15fa4b9\": container with ID starting with 999e90e300f2958a66e56f2499a47d377a72f26a04e39c63d0c71456d15fa4b9 not found: ID does not exist" containerID="999e90e300f2958a66e56f2499a47d377a72f26a04e39c63d0c71456d15fa4b9" Jul 06 22:42:30.205423 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:30.205373 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"999e90e300f2958a66e56f2499a47d377a72f26a04e39c63d0c71456d15fa4b9"} err="failed to get container status \"999e90e300f2958a66e56f2499a47d377a72f26a04e39c63d0c71456d15fa4b9\": rpc error: code = NotFound desc = could not find container \"999e90e300f2958a66e56f2499a47d377a72f26a04e39c63d0c71456d15fa4b9\": container with ID starting with 999e90e300f2958a66e56f2499a47d377a72f26a04e39c63d0c71456d15fa4b9 not found: ID does not exist" Jul 06 22:42:30.211595 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:30.211575 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-5d913f-openshift-default-5d756c7c6c-nfcfz"] Jul 06 22:42:30.216784 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:30.216765 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-5d913f-openshift-default-5d756c7c6c-nfcfz"] Jul 06 22:42:30.977259 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:30.977218 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="69185b8f-db46-4b55-85a1-27bfc2027672" path="/var/lib/kubelet/pods/69185b8f-db46-4b55-85a1-27bfc2027672/volumes" Jul 06 22:42:32.583514 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:32.583474 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:42:33.031093 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:33.031046 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:42:33.313014 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:33.312913 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-63b31a-openshift-default-7f4945587c-jgxdj"] Jul 06 22:42:33.313707 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:33.313680 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="69185b8f-db46-4b55-85a1-27bfc2027672" containerName="istio-proxy" Jul 06 22:42:33.313878 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:33.313855 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="69185b8f-db46-4b55-85a1-27bfc2027672" containerName="istio-proxy" Jul 06 22:42:33.314001 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:33.313901 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="69185b8f-db46-4b55-85a1-27bfc2027672" containerName="istio-proxy" Jul 06 22:42:33.314001 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:33.313911 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="69185b8f-db46-4b55-85a1-27bfc2027672" containerName="istio-proxy" Jul 06 22:42:33.314152 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:33.314042 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="69185b8f-db46-4b55-85a1-27bfc2027672" containerName="istio-proxy" Jul 06 22:42:33.314152 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:33.314060 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="69185b8f-db46-4b55-85a1-27bfc2027672" containerName="istio-proxy" Jul 06 22:42:33.318901 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:33.318882 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-63b31a-openshift-default-7f4945587c-jgxdj" Jul 06 22:42:33.321275 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:33.321254 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-63b31a-openshift-default-dockercfg-dhcwz\"" Jul 06 22:42:33.328878 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:33.328855 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-63b31a-openshift-default-7f4945587c-jgxdj"] Jul 06 22:42:33.345338 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:33.345301 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:42:33.413720 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:33.413686 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ac73802a-b5e4-47bb-9f6e-d938e8705f13-istio-envoy\") pod \"e2e-api-b-63b31a-openshift-default-7f4945587c-jgxdj\" (UID: \"ac73802a-b5e4-47bb-9f6e-d938e8705f13\") " pod="openshift-ingress/e2e-api-b-63b31a-openshift-default-7f4945587c-jgxdj" Jul 06 22:42:33.413922 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:33.413744 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ac73802a-b5e4-47bb-9f6e-d938e8705f13-istio-data\") pod \"e2e-api-b-63b31a-openshift-default-7f4945587c-jgxdj\" (UID: \"ac73802a-b5e4-47bb-9f6e-d938e8705f13\") " pod="openshift-ingress/e2e-api-b-63b31a-openshift-default-7f4945587c-jgxdj" Jul 06 22:42:33.413922 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:33.413823 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ac73802a-b5e4-47bb-9f6e-d938e8705f13-istio-token\") pod \"e2e-api-b-63b31a-openshift-default-7f4945587c-jgxdj\" (UID: \"ac73802a-b5e4-47bb-9f6e-d938e8705f13\") " pod="openshift-ingress/e2e-api-b-63b31a-openshift-default-7f4945587c-jgxdj" Jul 06 22:42:33.413922 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:33.413904 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ac73802a-b5e4-47bb-9f6e-d938e8705f13-istiod-ca-cert\") pod \"e2e-api-b-63b31a-openshift-default-7f4945587c-jgxdj\" (UID: \"ac73802a-b5e4-47bb-9f6e-d938e8705f13\") " pod="openshift-ingress/e2e-api-b-63b31a-openshift-default-7f4945587c-jgxdj" Jul 06 22:42:33.414104 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:33.413950 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ac73802a-b5e4-47bb-9f6e-d938e8705f13-workload-certs\") pod \"e2e-api-b-63b31a-openshift-default-7f4945587c-jgxdj\" (UID: \"ac73802a-b5e4-47bb-9f6e-d938e8705f13\") " pod="openshift-ingress/e2e-api-b-63b31a-openshift-default-7f4945587c-jgxdj" Jul 06 22:42:33.414104 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:33.414003 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac73802a-b5e4-47bb-9f6e-d938e8705f13-workload-socket\") pod \"e2e-api-b-63b31a-openshift-default-7f4945587c-jgxdj\" (UID: \"ac73802a-b5e4-47bb-9f6e-d938e8705f13\") " pod="openshift-ingress/e2e-api-b-63b31a-openshift-default-7f4945587c-jgxdj" Jul 06 22:42:33.414104 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:33.414052 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-knvb7\" (UniqueName: \"kubernetes.io/projected/ac73802a-b5e4-47bb-9f6e-d938e8705f13-kube-api-access-knvb7\") pod \"e2e-api-b-63b31a-openshift-default-7f4945587c-jgxdj\" (UID: \"ac73802a-b5e4-47bb-9f6e-d938e8705f13\") " pod="openshift-ingress/e2e-api-b-63b31a-openshift-default-7f4945587c-jgxdj" Jul 06 22:42:33.414246 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:33.414168 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac73802a-b5e4-47bb-9f6e-d938e8705f13-credential-socket\") pod \"e2e-api-b-63b31a-openshift-default-7f4945587c-jgxdj\" (UID: \"ac73802a-b5e4-47bb-9f6e-d938e8705f13\") " pod="openshift-ingress/e2e-api-b-63b31a-openshift-default-7f4945587c-jgxdj" Jul 06 22:42:33.414246 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:33.414205 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ac73802a-b5e4-47bb-9f6e-d938e8705f13-istio-podinfo\") pod \"e2e-api-b-63b31a-openshift-default-7f4945587c-jgxdj\" (UID: \"ac73802a-b5e4-47bb-9f6e-d938e8705f13\") " pod="openshift-ingress/e2e-api-b-63b31a-openshift-default-7f4945587c-jgxdj" Jul 06 22:42:33.515373 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:33.515332 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ac73802a-b5e4-47bb-9f6e-d938e8705f13-istio-token\") pod \"e2e-api-b-63b31a-openshift-default-7f4945587c-jgxdj\" (UID: \"ac73802a-b5e4-47bb-9f6e-d938e8705f13\") " pod="openshift-ingress/e2e-api-b-63b31a-openshift-default-7f4945587c-jgxdj" Jul 06 22:42:33.515568 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:33.515382 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ac73802a-b5e4-47bb-9f6e-d938e8705f13-istiod-ca-cert\") pod \"e2e-api-b-63b31a-openshift-default-7f4945587c-jgxdj\" (UID: \"ac73802a-b5e4-47bb-9f6e-d938e8705f13\") " pod="openshift-ingress/e2e-api-b-63b31a-openshift-default-7f4945587c-jgxdj" Jul 06 22:42:33.515568 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:33.515405 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ac73802a-b5e4-47bb-9f6e-d938e8705f13-workload-certs\") pod \"e2e-api-b-63b31a-openshift-default-7f4945587c-jgxdj\" (UID: \"ac73802a-b5e4-47bb-9f6e-d938e8705f13\") " pod="openshift-ingress/e2e-api-b-63b31a-openshift-default-7f4945587c-jgxdj" Jul 06 22:42:33.515568 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:33.515432 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac73802a-b5e4-47bb-9f6e-d938e8705f13-workload-socket\") pod \"e2e-api-b-63b31a-openshift-default-7f4945587c-jgxdj\" (UID: \"ac73802a-b5e4-47bb-9f6e-d938e8705f13\") " pod="openshift-ingress/e2e-api-b-63b31a-openshift-default-7f4945587c-jgxdj" Jul 06 22:42:33.515568 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:33.515461 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-knvb7\" (UniqueName: \"kubernetes.io/projected/ac73802a-b5e4-47bb-9f6e-d938e8705f13-kube-api-access-knvb7\") pod \"e2e-api-b-63b31a-openshift-default-7f4945587c-jgxdj\" (UID: \"ac73802a-b5e4-47bb-9f6e-d938e8705f13\") " pod="openshift-ingress/e2e-api-b-63b31a-openshift-default-7f4945587c-jgxdj" Jul 06 22:42:33.515792 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:33.515611 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac73802a-b5e4-47bb-9f6e-d938e8705f13-credential-socket\") pod \"e2e-api-b-63b31a-openshift-default-7f4945587c-jgxdj\" (UID: \"ac73802a-b5e4-47bb-9f6e-d938e8705f13\") " pod="openshift-ingress/e2e-api-b-63b31a-openshift-default-7f4945587c-jgxdj" Jul 06 22:42:33.515792 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:33.515664 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ac73802a-b5e4-47bb-9f6e-d938e8705f13-istio-podinfo\") pod \"e2e-api-b-63b31a-openshift-default-7f4945587c-jgxdj\" (UID: \"ac73802a-b5e4-47bb-9f6e-d938e8705f13\") " pod="openshift-ingress/e2e-api-b-63b31a-openshift-default-7f4945587c-jgxdj" Jul 06 22:42:33.515792 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:33.515698 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ac73802a-b5e4-47bb-9f6e-d938e8705f13-istio-envoy\") pod \"e2e-api-b-63b31a-openshift-default-7f4945587c-jgxdj\" (UID: \"ac73802a-b5e4-47bb-9f6e-d938e8705f13\") " pod="openshift-ingress/e2e-api-b-63b31a-openshift-default-7f4945587c-jgxdj" Jul 06 22:42:33.515792 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:33.515751 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ac73802a-b5e4-47bb-9f6e-d938e8705f13-istio-data\") pod \"e2e-api-b-63b31a-openshift-default-7f4945587c-jgxdj\" (UID: \"ac73802a-b5e4-47bb-9f6e-d938e8705f13\") " pod="openshift-ingress/e2e-api-b-63b31a-openshift-default-7f4945587c-jgxdj" Jul 06 22:42:33.515999 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:33.515876 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ac73802a-b5e4-47bb-9f6e-d938e8705f13-workload-certs\") pod \"e2e-api-b-63b31a-openshift-default-7f4945587c-jgxdj\" (UID: \"ac73802a-b5e4-47bb-9f6e-d938e8705f13\") " pod="openshift-ingress/e2e-api-b-63b31a-openshift-default-7f4945587c-jgxdj" Jul 06 22:42:33.515999 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:33.515904 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac73802a-b5e4-47bb-9f6e-d938e8705f13-workload-socket\") pod \"e2e-api-b-63b31a-openshift-default-7f4945587c-jgxdj\" (UID: \"ac73802a-b5e4-47bb-9f6e-d938e8705f13\") " pod="openshift-ingress/e2e-api-b-63b31a-openshift-default-7f4945587c-jgxdj" Jul 06 22:42:33.516130 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:33.516050 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac73802a-b5e4-47bb-9f6e-d938e8705f13-credential-socket\") pod \"e2e-api-b-63b31a-openshift-default-7f4945587c-jgxdj\" (UID: \"ac73802a-b5e4-47bb-9f6e-d938e8705f13\") " pod="openshift-ingress/e2e-api-b-63b31a-openshift-default-7f4945587c-jgxdj" Jul 06 22:42:33.516197 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:33.516124 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ac73802a-b5e4-47bb-9f6e-d938e8705f13-istio-data\") pod \"e2e-api-b-63b31a-openshift-default-7f4945587c-jgxdj\" (UID: \"ac73802a-b5e4-47bb-9f6e-d938e8705f13\") " pod="openshift-ingress/e2e-api-b-63b31a-openshift-default-7f4945587c-jgxdj" Jul 06 22:42:33.516335 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:33.516314 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ac73802a-b5e4-47bb-9f6e-d938e8705f13-istiod-ca-cert\") pod \"e2e-api-b-63b31a-openshift-default-7f4945587c-jgxdj\" (UID: \"ac73802a-b5e4-47bb-9f6e-d938e8705f13\") " pod="openshift-ingress/e2e-api-b-63b31a-openshift-default-7f4945587c-jgxdj" Jul 06 22:42:33.518096 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:33.518051 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ac73802a-b5e4-47bb-9f6e-d938e8705f13-istio-envoy\") pod \"e2e-api-b-63b31a-openshift-default-7f4945587c-jgxdj\" (UID: \"ac73802a-b5e4-47bb-9f6e-d938e8705f13\") " pod="openshift-ingress/e2e-api-b-63b31a-openshift-default-7f4945587c-jgxdj" Jul 06 22:42:33.518216 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:33.518201 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ac73802a-b5e4-47bb-9f6e-d938e8705f13-istio-podinfo\") pod \"e2e-api-b-63b31a-openshift-default-7f4945587c-jgxdj\" (UID: \"ac73802a-b5e4-47bb-9f6e-d938e8705f13\") " pod="openshift-ingress/e2e-api-b-63b31a-openshift-default-7f4945587c-jgxdj" Jul 06 22:42:33.528471 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:33.528448 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-knvb7\" (UniqueName: \"kubernetes.io/projected/ac73802a-b5e4-47bb-9f6e-d938e8705f13-kube-api-access-knvb7\") pod \"e2e-api-b-63b31a-openshift-default-7f4945587c-jgxdj\" (UID: \"ac73802a-b5e4-47bb-9f6e-d938e8705f13\") " pod="openshift-ingress/e2e-api-b-63b31a-openshift-default-7f4945587c-jgxdj" Jul 06 22:42:33.532121 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:33.532099 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ac73802a-b5e4-47bb-9f6e-d938e8705f13-istio-token\") pod \"e2e-api-b-63b31a-openshift-default-7f4945587c-jgxdj\" (UID: \"ac73802a-b5e4-47bb-9f6e-d938e8705f13\") " pod="openshift-ingress/e2e-api-b-63b31a-openshift-default-7f4945587c-jgxdj" Jul 06 22:42:33.631121 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:33.631013 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-63b31a-openshift-default-7f4945587c-jgxdj" Jul 06 22:42:33.689508 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:33.689416 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:42:33.773009 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:33.772986 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-63b31a-openshift-default-7f4945587c-jgxdj"] Jul 06 22:42:33.775231 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:42:33.775206 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac73802a_b5e4_47bb_9f6e_d938e8705f13.slice/crio-9426136c434411bd6f3805d39150ee540295b52bf30f48eb59f57fb6685da60e WatchSource:0}: Error finding container 9426136c434411bd6f3805d39150ee540295b52bf30f48eb59f57fb6685da60e: Status 404 returned error can't find the container with id 9426136c434411bd6f3805d39150ee540295b52bf30f48eb59f57fb6685da60e Jul 06 22:42:33.777299 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:33.777267 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 22:42:33.777379 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:33.777332 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 22:42:33.777379 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:33.777358 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 22:42:34.209696 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:34.209659 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-63b31a-openshift-default-7f4945587c-jgxdj" event={"ID":"ac73802a-b5e4-47bb-9f6e-d938e8705f13","Type":"ContainerStarted","Data":"3c7d4404974c155303bf46376d5959f29bb9c6866c4ebc7978e4714843c147dc"} Jul 06 22:42:34.209863 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:34.209722 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-63b31a-openshift-default-7f4945587c-jgxdj" event={"ID":"ac73802a-b5e4-47bb-9f6e-d938e8705f13","Type":"ContainerStarted","Data":"9426136c434411bd6f3805d39150ee540295b52bf30f48eb59f57fb6685da60e"} Jul 06 22:42:34.229926 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:34.229874 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-63b31a-openshift-default-7f4945587c-jgxdj" podStartSLOduration=1.229851823 podStartE2EDuration="1.229851823s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:42:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:42:34.227269838 +0000 UTC m=+2033.786065184" watchObservedRunningTime="2026-07-06 22:42:34.229851823 +0000 UTC m=+2033.788647147" Jul 06 22:42:34.631511 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:34.631433 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-63b31a-openshift-default-7f4945587c-jgxdj" Jul 06 22:42:34.636438 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:34.636413 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-63b31a-openshift-default-7f4945587c-jgxdj" Jul 06 22:42:35.221487 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:35.221453 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-63b31a-openshift-default-7f4945587c-jgxdj" Jul 06 22:42:35.222532 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:35.222505 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-63b31a-openshift-default-7f4945587c-jgxdj" Jul 06 22:42:36.173098 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:36.173042 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:42:36.274596 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:36.274566 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:42:36.369743 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:36.369710 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:42:36.513047 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:36.513007 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:42:36.728818 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:36.728786 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:42:36.830345 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:36.830264 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:42:40.664691 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:40.664649 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:42:43.210211 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:43.210170 2581 patch_prober.go:28] interesting pod/e2e-api-a-fbe640-openshift-default-6b9cb89c65-45gx6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 06 22:42:43.210609 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:43.210243 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-fbe640-openshift-default-6b9cb89c65-45gx6" podUID="ca959e8b-6125-4469-9a30-727411fa7c8f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 06 22:42:45.286659 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:45.286622 2581 generic.go:356] "Generic (PLEG): container finished" podID="ca959e8b-6125-4469-9a30-727411fa7c8f" containerID="0d62aef909c58c76bcef114f4f1a8148d6ebaa960735e61f1cc9173e15b991db" exitCode=0 Jul 06 22:42:45.287131 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:45.286700 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-fbe640-openshift-default-6b9cb89c65-45gx6" event={"ID":"ca959e8b-6125-4469-9a30-727411fa7c8f","Type":"ContainerDied","Data":"0d62aef909c58c76bcef114f4f1a8148d6ebaa960735e61f1cc9173e15b991db"} Jul 06 22:42:45.287183 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:45.287143 2581 scope.go:117] "RemoveContainer" containerID="0d62aef909c58c76bcef114f4f1a8148d6ebaa960735e61f1cc9173e15b991db" Jul 06 22:42:45.288258 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:45.288230 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 22:42:45.288341 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:45.288298 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 22:42:45.288341 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:45.288333 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 22:42:46.295408 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:46.295363 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-fbe640-openshift-default-6b9cb89c65-45gx6" event={"ID":"ca959e8b-6125-4469-9a30-727411fa7c8f","Type":"ContainerStarted","Data":"4bf636c7972e4c4804c4665a984747fd9f49ca7444f0d0b366641bc5deb48373"} Jul 06 22:42:46.729693 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:46.729662 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-fbe640-openshift-default-6b9cb89c65-45gx6" Jul 06 22:42:46.729851 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:46.729787 2581 patch_prober.go:28] interesting pod/e2e-api-a-fbe640-openshift-default-6b9cb89c65-45gx6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body= Jul 06 22:42:46.729851 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:46.729823 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-fbe640-openshift-default-6b9cb89c65-45gx6" podUID="ca959e8b-6125-4469-9a30-727411fa7c8f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" Jul 06 22:42:47.728955 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:47.728918 2581 patch_prober.go:28] interesting pod/e2e-api-a-fbe640-openshift-default-6b9cb89c65-45gx6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body= Jul 06 22:42:47.729478 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:47.728978 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-fbe640-openshift-default-6b9cb89c65-45gx6" podUID="ca959e8b-6125-4469-9a30-727411fa7c8f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" Jul 06 22:42:48.729276 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:48.729233 2581 patch_prober.go:28] interesting pod/e2e-api-a-fbe640-openshift-default-6b9cb89c65-45gx6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body= Jul 06 22:42:48.729830 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:48.729308 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-fbe640-openshift-default-6b9cb89c65-45gx6" podUID="ca959e8b-6125-4469-9a30-727411fa7c8f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" Jul 06 22:42:48.777012 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:48.776923 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:42:49.172475 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:49.172371 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:42:49.276463 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:49.276425 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:42:49.379636 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:49.379583 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:42:49.605705 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:49.605654 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:42:49.729330 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:49.729296 2581 patch_prober.go:28] interesting pod/e2e-api-a-fbe640-openshift-default-6b9cb89c65-45gx6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body= Jul 06 22:42:49.729702 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:49.729392 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-fbe640-openshift-default-6b9cb89c65-45gx6" podUID="ca959e8b-6125-4469-9a30-727411fa7c8f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" Jul 06 22:42:50.732778 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:50.732749 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-fbe640-openshift-default-6b9cb89c65-45gx6" Jul 06 22:42:50.733465 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:50.733315 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-fbe640-openshift-default-6b9cb89c65-45gx6" Jul 06 22:42:50.734199 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:50.734177 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-fbe640-openshift-default-6b9cb89c65-45gx6" Jul 06 22:42:51.263276 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:51.263234 2581 patch_prober.go:28] interesting pod/e2e-api-b-63b31a-openshift-default-7f4945587c-jgxdj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 06 22:42:51.263464 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:51.263312 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-63b31a-openshift-default-7f4945587c-jgxdj" podUID="ac73802a-b5e4-47bb-9f6e-d938e8705f13" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 06 22:42:53.334003 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:53.333960 2581 generic.go:356] "Generic (PLEG): container finished" podID="ac73802a-b5e4-47bb-9f6e-d938e8705f13" containerID="3c7d4404974c155303bf46376d5959f29bb9c6866c4ebc7978e4714843c147dc" exitCode=0 Jul 06 22:42:53.334457 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:53.334038 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-63b31a-openshift-default-7f4945587c-jgxdj" event={"ID":"ac73802a-b5e4-47bb-9f6e-d938e8705f13","Type":"ContainerDied","Data":"3c7d4404974c155303bf46376d5959f29bb9c6866c4ebc7978e4714843c147dc"} Jul 06 22:42:53.334505 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:53.334491 2581 scope.go:117] "RemoveContainer" containerID="3c7d4404974c155303bf46376d5959f29bb9c6866c4ebc7978e4714843c147dc" Jul 06 22:42:53.335533 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:53.335501 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 22:42:53.335643 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:53.335587 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 22:42:53.335643 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:53.335618 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 22:42:54.350808 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:54.350761 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-63b31a-openshift-default-7f4945587c-jgxdj" event={"ID":"ac73802a-b5e4-47bb-9f6e-d938e8705f13","Type":"ContainerStarted","Data":"11fc7b341eeb4d430e3bb27e9ac0d4c77d6b647c6d8546bfb21c46ee32a38074"} Jul 06 22:42:54.631374 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:54.631292 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-63b31a-openshift-default-7f4945587c-jgxdj" Jul 06 22:42:54.631522 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:54.631465 2581 patch_prober.go:28] interesting pod/e2e-api-b-63b31a-openshift-default-7f4945587c-jgxdj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body= Jul 06 22:42:54.631565 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:54.631512 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-63b31a-openshift-default-7f4945587c-jgxdj" podUID="ac73802a-b5e4-47bb-9f6e-d938e8705f13" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" Jul 06 22:42:55.040580 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:55.040549 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:42:55.141649 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:55.141606 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:42:55.286790 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:55.286756 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-fbe640-openshift-default-6b9cb89c65-45gx6"] Jul 06 22:42:55.287018 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:55.286993 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-fbe640-openshift-default-6b9cb89c65-45gx6" podUID="ca959e8b-6125-4469-9a30-727411fa7c8f" containerName="istio-proxy" containerID="cri-o://4bf636c7972e4c4804c4665a984747fd9f49ca7444f0d0b366641bc5deb48373" gracePeriod=30 Jul 06 22:42:55.302829 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:55.302755 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:42:55.527181 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:55.527131 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:42:55.631976 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:55.631901 2581 patch_prober.go:28] interesting pod/e2e-api-b-63b31a-openshift-default-7f4945587c-jgxdj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body= Jul 06 22:42:55.631976 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:55.631958 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-63b31a-openshift-default-7f4945587c-jgxdj" podUID="ac73802a-b5e4-47bb-9f6e-d938e8705f13" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" Jul 06 22:42:55.696901 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:55.696873 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:42:56.077925 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:56.077892 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:42:56.178277 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:56.178244 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:42:56.279403 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:56.279371 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:42:56.420769 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:56.420685 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:42:56.631682 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:56.631647 2581 patch_prober.go:28] interesting pod/e2e-api-b-63b31a-openshift-default-7f4945587c-jgxdj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body= Jul 06 22:42:56.632059 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:56.631702 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-63b31a-openshift-default-7f4945587c-jgxdj" podUID="ac73802a-b5e4-47bb-9f6e-d938e8705f13" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" Jul 06 22:42:57.261792 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:57.261745 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:42:57.363000 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:57.362965 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:42:57.632453 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:57.632367 2581 patch_prober.go:28] interesting pod/e2e-api-b-63b31a-openshift-default-7f4945587c-jgxdj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body= Jul 06 22:42:57.632453 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:57.632419 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-63b31a-openshift-default-7f4945587c-jgxdj" podUID="ac73802a-b5e4-47bb-9f6e-d938e8705f13" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" Jul 06 22:42:58.635238 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:58.635201 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-63b31a-openshift-default-7f4945587c-jgxdj" Jul 06 22:42:58.635703 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:58.635610 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-63b31a-openshift-default-7f4945587c-jgxdj" Jul 06 22:42:58.636344 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:42:58.636324 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-63b31a-openshift-default-7f4945587c-jgxdj" Jul 06 22:43:00.558753 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:00.558678 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-fbe640-openshift-default-6b9cb89c65-45gx6" Jul 06 22:43:00.613467 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:00.613439 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ca959e8b-6125-4469-9a30-727411fa7c8f-workload-certs\") pod \"ca959e8b-6125-4469-9a30-727411fa7c8f\" (UID: \"ca959e8b-6125-4469-9a30-727411fa7c8f\") " Jul 06 22:43:00.613649 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:00.613475 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ca959e8b-6125-4469-9a30-727411fa7c8f-istio-envoy\") pod \"ca959e8b-6125-4469-9a30-727411fa7c8f\" (UID: \"ca959e8b-6125-4469-9a30-727411fa7c8f\") " Jul 06 22:43:00.613649 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:00.613528 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ca959e8b-6125-4469-9a30-727411fa7c8f-istio-data\") pod \"ca959e8b-6125-4469-9a30-727411fa7c8f\" (UID: \"ca959e8b-6125-4469-9a30-727411fa7c8f\") " Jul 06 22:43:00.613649 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:00.613577 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca959e8b-6125-4469-9a30-727411fa7c8f-workload-socket\") pod \"ca959e8b-6125-4469-9a30-727411fa7c8f\" (UID: \"ca959e8b-6125-4469-9a30-727411fa7c8f\") " Jul 06 22:43:00.613649 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:00.613598 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ca959e8b-6125-4469-9a30-727411fa7c8f-istio-token\") pod \"ca959e8b-6125-4469-9a30-727411fa7c8f\" (UID: \"ca959e8b-6125-4469-9a30-727411fa7c8f\") " Jul 06 22:43:00.613649 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:00.613625 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f5fhs\" (UniqueName: \"kubernetes.io/projected/ca959e8b-6125-4469-9a30-727411fa7c8f-kube-api-access-f5fhs\") pod \"ca959e8b-6125-4469-9a30-727411fa7c8f\" (UID: \"ca959e8b-6125-4469-9a30-727411fa7c8f\") " Jul 06 22:43:00.613649 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:00.613649 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ca959e8b-6125-4469-9a30-727411fa7c8f-istio-podinfo\") pod \"ca959e8b-6125-4469-9a30-727411fa7c8f\" (UID: \"ca959e8b-6125-4469-9a30-727411fa7c8f\") " Jul 06 22:43:00.613959 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:00.613669 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca959e8b-6125-4469-9a30-727411fa7c8f-credential-socket\") pod \"ca959e8b-6125-4469-9a30-727411fa7c8f\" (UID: \"ca959e8b-6125-4469-9a30-727411fa7c8f\") " Jul 06 22:43:00.613959 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:00.613695 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ca959e8b-6125-4469-9a30-727411fa7c8f-istiod-ca-cert\") pod \"ca959e8b-6125-4469-9a30-727411fa7c8f\" (UID: \"ca959e8b-6125-4469-9a30-727411fa7c8f\") " Jul 06 22:43:00.613959 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:00.613743 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca959e8b-6125-4469-9a30-727411fa7c8f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ca959e8b-6125-4469-9a30-727411fa7c8f" (UID: "ca959e8b-6125-4469-9a30-727411fa7c8f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:43:00.613959 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:00.613858 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca959e8b-6125-4469-9a30-727411fa7c8f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ca959e8b-6125-4469-9a30-727411fa7c8f" (UID: "ca959e8b-6125-4469-9a30-727411fa7c8f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:43:00.614156 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:00.614088 2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca959e8b-6125-4469-9a30-727411fa7c8f-workload-socket\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:43:00.614156 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:00.614094 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ca959e8b-6125-4469-9a30-727411fa7c8f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ca959e8b-6125-4469-9a30-727411fa7c8f" (UID: "ca959e8b-6125-4469-9a30-727411fa7c8f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 22:43:00.614156 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:00.614114 2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ca959e8b-6125-4469-9a30-727411fa7c8f-workload-certs\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:43:00.614375 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:00.614349 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca959e8b-6125-4469-9a30-727411fa7c8f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ca959e8b-6125-4469-9a30-727411fa7c8f" (UID: "ca959e8b-6125-4469-9a30-727411fa7c8f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:43:00.614618 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:00.614589 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca959e8b-6125-4469-9a30-727411fa7c8f-istio-data" (OuterVolumeSpecName: "istio-data") pod "ca959e8b-6125-4469-9a30-727411fa7c8f" (UID: "ca959e8b-6125-4469-9a30-727411fa7c8f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:43:00.615837 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:00.615810 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca959e8b-6125-4469-9a30-727411fa7c8f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ca959e8b-6125-4469-9a30-727411fa7c8f" (UID: "ca959e8b-6125-4469-9a30-727411fa7c8f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:43:00.615918 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:00.615865 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ca959e8b-6125-4469-9a30-727411fa7c8f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ca959e8b-6125-4469-9a30-727411fa7c8f" (UID: "ca959e8b-6125-4469-9a30-727411fa7c8f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 22:43:00.616020 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:00.616001 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca959e8b-6125-4469-9a30-727411fa7c8f-istio-token" (OuterVolumeSpecName: "istio-token") pod "ca959e8b-6125-4469-9a30-727411fa7c8f" (UID: "ca959e8b-6125-4469-9a30-727411fa7c8f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 22:43:00.616103 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:00.616032 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca959e8b-6125-4469-9a30-727411fa7c8f-kube-api-access-f5fhs" (OuterVolumeSpecName: "kube-api-access-f5fhs") pod "ca959e8b-6125-4469-9a30-727411fa7c8f" (UID: "ca959e8b-6125-4469-9a30-727411fa7c8f"). InnerVolumeSpecName "kube-api-access-f5fhs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 22:43:00.714876 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:00.714825 2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ca959e8b-6125-4469-9a30-727411fa7c8f-istio-envoy\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:43:00.714876 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:00.714868 2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ca959e8b-6125-4469-9a30-727411fa7c8f-istio-data\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:43:00.714876 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:00.714880 2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ca959e8b-6125-4469-9a30-727411fa7c8f-istio-token\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:43:00.715142 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:00.714892 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f5fhs\" (UniqueName: \"kubernetes.io/projected/ca959e8b-6125-4469-9a30-727411fa7c8f-kube-api-access-f5fhs\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:43:00.715142 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:00.714906 2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ca959e8b-6125-4469-9a30-727411fa7c8f-istio-podinfo\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:43:00.715142 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:00.714917 2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca959e8b-6125-4469-9a30-727411fa7c8f-credential-socket\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:43:00.715142 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:00.714929 2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ca959e8b-6125-4469-9a30-727411fa7c8f-istiod-ca-cert\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:43:01.395795 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:01.395704 2581 generic.go:356] "Generic (PLEG): container finished" podID="ca959e8b-6125-4469-9a30-727411fa7c8f" containerID="4bf636c7972e4c4804c4665a984747fd9f49ca7444f0d0b366641bc5deb48373" exitCode=0 Jul 06 22:43:01.395795 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:01.395754 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-fbe640-openshift-default-6b9cb89c65-45gx6" Jul 06 22:43:01.395795 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:01.395780 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-fbe640-openshift-default-6b9cb89c65-45gx6" event={"ID":"ca959e8b-6125-4469-9a30-727411fa7c8f","Type":"ContainerDied","Data":"4bf636c7972e4c4804c4665a984747fd9f49ca7444f0d0b366641bc5deb48373"} Jul 06 22:43:01.396033 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:01.395830 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-fbe640-openshift-default-6b9cb89c65-45gx6" event={"ID":"ca959e8b-6125-4469-9a30-727411fa7c8f","Type":"ContainerDied","Data":"ee411fed06e2516723ab9d15b04ffdf394039c5789c2ed6f2fd01b42bc982007"} Jul 06 22:43:01.396033 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:01.395853 2581 scope.go:117] "RemoveContainer" containerID="4bf636c7972e4c4804c4665a984747fd9f49ca7444f0d0b366641bc5deb48373" Jul 06 22:43:01.404855 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:01.404685 2581 scope.go:117] "RemoveContainer" containerID="0d62aef909c58c76bcef114f4f1a8148d6ebaa960735e61f1cc9173e15b991db" Jul 06 22:43:01.412977 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:01.412959 2581 scope.go:117] "RemoveContainer" containerID="4bf636c7972e4c4804c4665a984747fd9f49ca7444f0d0b366641bc5deb48373" Jul 06 22:43:01.413292 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:43:01.413273 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4bf636c7972e4c4804c4665a984747fd9f49ca7444f0d0b366641bc5deb48373\": container with ID starting with 4bf636c7972e4c4804c4665a984747fd9f49ca7444f0d0b366641bc5deb48373 not found: ID does not exist" containerID="4bf636c7972e4c4804c4665a984747fd9f49ca7444f0d0b366641bc5deb48373" Jul 06 22:43:01.413365 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:01.413299 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4bf636c7972e4c4804c4665a984747fd9f49ca7444f0d0b366641bc5deb48373"} err="failed to get container status \"4bf636c7972e4c4804c4665a984747fd9f49ca7444f0d0b366641bc5deb48373\": rpc error: code = NotFound desc = could not find container \"4bf636c7972e4c4804c4665a984747fd9f49ca7444f0d0b366641bc5deb48373\": container with ID starting with 4bf636c7972e4c4804c4665a984747fd9f49ca7444f0d0b366641bc5deb48373 not found: ID does not exist" Jul 06 22:43:01.413365 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:01.413315 2581 scope.go:117] "RemoveContainer" containerID="0d62aef909c58c76bcef114f4f1a8148d6ebaa960735e61f1cc9173e15b991db" Jul 06 22:43:01.413541 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:43:01.413525 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0d62aef909c58c76bcef114f4f1a8148d6ebaa960735e61f1cc9173e15b991db\": container with ID starting with 0d62aef909c58c76bcef114f4f1a8148d6ebaa960735e61f1cc9173e15b991db not found: ID does not exist" containerID="0d62aef909c58c76bcef114f4f1a8148d6ebaa960735e61f1cc9173e15b991db" Jul 06 22:43:01.413578 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:01.413544 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0d62aef909c58c76bcef114f4f1a8148d6ebaa960735e61f1cc9173e15b991db"} err="failed to get container status \"0d62aef909c58c76bcef114f4f1a8148d6ebaa960735e61f1cc9173e15b991db\": rpc error: code = NotFound desc = could not find container \"0d62aef909c58c76bcef114f4f1a8148d6ebaa960735e61f1cc9173e15b991db\": container with ID starting with 0d62aef909c58c76bcef114f4f1a8148d6ebaa960735e61f1cc9173e15b991db not found: ID does not exist" Jul 06 22:43:01.417778 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:01.417753 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-fbe640-openshift-default-6b9cb89c65-45gx6"] Jul 06 22:43:01.421293 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:01.421267 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-fbe640-openshift-default-6b9cb89c65-45gx6"] Jul 06 22:43:01.931613 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:01.931579 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:43:02.003785 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:02.003728 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:43:02.169317 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:02.169286 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-63b31a-openshift-default-7f4945587c-jgxdj"] Jul 06 22:43:02.169539 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:02.169509 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-63b31a-openshift-default-7f4945587c-jgxdj" podUID="ac73802a-b5e4-47bb-9f6e-d938e8705f13" containerName="istio-proxy" containerID="cri-o://11fc7b341eeb4d430e3bb27e9ac0d4c77d6b647c6d8546bfb21c46ee32a38074" gracePeriod=30 Jul 06 22:43:02.188521 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:02.188432 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:43:02.407043 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:02.406985 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:43:02.738773 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:02.738736 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:43:02.976858 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:02.976827 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ca959e8b-6125-4469-9a30-727411fa7c8f" path="/var/lib/kubelet/pods/ca959e8b-6125-4469-9a30-727411fa7c8f/volumes" Jul 06 22:43:03.004453 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:03.004364 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-65387d-openshift-default-7594fd6b75-2br8j"] Jul 06 22:43:03.004895 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:03.004861 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca959e8b-6125-4469-9a30-727411fa7c8f" containerName="istio-proxy" Jul 06 22:43:03.004895 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:03.004885 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca959e8b-6125-4469-9a30-727411fa7c8f" containerName="istio-proxy" Jul 06 22:43:03.005114 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:03.004901 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca959e8b-6125-4469-9a30-727411fa7c8f" containerName="istio-proxy" Jul 06 22:43:03.005114 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:03.004912 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca959e8b-6125-4469-9a30-727411fa7c8f" containerName="istio-proxy" Jul 06 22:43:03.005114 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:03.004998 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="ca959e8b-6125-4469-9a30-727411fa7c8f" containerName="istio-proxy" Jul 06 22:43:03.005114 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:03.005013 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="ca959e8b-6125-4469-9a30-727411fa7c8f" containerName="istio-proxy" Jul 06 22:43:03.009581 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:03.009559 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-65387d-openshift-default-7594fd6b75-2br8j" Jul 06 22:43:03.012227 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:03.012210 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-65387d-openshift-default-dockercfg-lrlr2\"" Jul 06 22:43:03.019994 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:03.019967 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-65387d-openshift-default-7594fd6b75-2br8j"] Jul 06 22:43:03.025120 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:03.025097 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:43:03.137996 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:03.137951 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9kf8s\" (UniqueName: \"kubernetes.io/projected/f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9-kube-api-access-9kf8s\") pod \"e2e-shared-a-65387d-openshift-default-7594fd6b75-2br8j\" (UID: \"f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9\") " pod="openshift-ingress/e2e-shared-a-65387d-openshift-default-7594fd6b75-2br8j" Jul 06 22:43:03.137996 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:03.138001 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9-istio-token\") pod \"e2e-shared-a-65387d-openshift-default-7594fd6b75-2br8j\" (UID: \"f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9\") " pod="openshift-ingress/e2e-shared-a-65387d-openshift-default-7594fd6b75-2br8j" Jul 06 22:43:03.138285 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:03.138021 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9-istiod-ca-cert\") pod \"e2e-shared-a-65387d-openshift-default-7594fd6b75-2br8j\" (UID: \"f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9\") " pod="openshift-ingress/e2e-shared-a-65387d-openshift-default-7594fd6b75-2br8j" Jul 06 22:43:03.138285 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:03.138103 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9-istio-podinfo\") pod \"e2e-shared-a-65387d-openshift-default-7594fd6b75-2br8j\" (UID: \"f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9\") " pod="openshift-ingress/e2e-shared-a-65387d-openshift-default-7594fd6b75-2br8j" Jul 06 22:43:03.138285 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:03.138142 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9-workload-socket\") pod \"e2e-shared-a-65387d-openshift-default-7594fd6b75-2br8j\" (UID: \"f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9\") " pod="openshift-ingress/e2e-shared-a-65387d-openshift-default-7594fd6b75-2br8j" Jul 06 22:43:03.138285 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:03.138186 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9-istio-envoy\") pod \"e2e-shared-a-65387d-openshift-default-7594fd6b75-2br8j\" (UID: \"f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9\") " pod="openshift-ingress/e2e-shared-a-65387d-openshift-default-7594fd6b75-2br8j" Jul 06 22:43:03.138285 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:03.138217 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9-workload-certs\") pod \"e2e-shared-a-65387d-openshift-default-7594fd6b75-2br8j\" (UID: \"f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9\") " pod="openshift-ingress/e2e-shared-a-65387d-openshift-default-7594fd6b75-2br8j" Jul 06 22:43:03.138285 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:03.138235 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9-credential-socket\") pod \"e2e-shared-a-65387d-openshift-default-7594fd6b75-2br8j\" (UID: \"f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9\") " pod="openshift-ingress/e2e-shared-a-65387d-openshift-default-7594fd6b75-2br8j" Jul 06 22:43:03.138597 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:03.138296 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9-istio-data\") pod \"e2e-shared-a-65387d-openshift-default-7594fd6b75-2br8j\" (UID: \"f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9\") " pod="openshift-ingress/e2e-shared-a-65387d-openshift-default-7594fd6b75-2br8j" Jul 06 22:43:03.239913 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:03.239824 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9-istio-envoy\") pod \"e2e-shared-a-65387d-openshift-default-7594fd6b75-2br8j\" (UID: \"f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9\") " pod="openshift-ingress/e2e-shared-a-65387d-openshift-default-7594fd6b75-2br8j" Jul 06 22:43:03.239913 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:03.239876 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9-workload-certs\") pod \"e2e-shared-a-65387d-openshift-default-7594fd6b75-2br8j\" (UID: \"f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9\") " pod="openshift-ingress/e2e-shared-a-65387d-openshift-default-7594fd6b75-2br8j" Jul 06 22:43:03.239913 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:03.239902 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9-credential-socket\") pod \"e2e-shared-a-65387d-openshift-default-7594fd6b75-2br8j\" (UID: \"f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9\") " pod="openshift-ingress/e2e-shared-a-65387d-openshift-default-7594fd6b75-2br8j" Jul 06 22:43:03.240240 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:03.239923 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9-istio-data\") pod \"e2e-shared-a-65387d-openshift-default-7594fd6b75-2br8j\" (UID: \"f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9\") " pod="openshift-ingress/e2e-shared-a-65387d-openshift-default-7594fd6b75-2br8j" Jul 06 22:43:03.240240 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:03.239985 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9kf8s\" (UniqueName: \"kubernetes.io/projected/f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9-kube-api-access-9kf8s\") pod \"e2e-shared-a-65387d-openshift-default-7594fd6b75-2br8j\" (UID: \"f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9\") " pod="openshift-ingress/e2e-shared-a-65387d-openshift-default-7594fd6b75-2br8j" Jul 06 22:43:03.240240 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:03.240013 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9-istio-token\") pod \"e2e-shared-a-65387d-openshift-default-7594fd6b75-2br8j\" (UID: \"f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9\") " pod="openshift-ingress/e2e-shared-a-65387d-openshift-default-7594fd6b75-2br8j" Jul 06 22:43:03.240240 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:03.240034 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9-istiod-ca-cert\") pod \"e2e-shared-a-65387d-openshift-default-7594fd6b75-2br8j\" (UID: \"f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9\") " pod="openshift-ingress/e2e-shared-a-65387d-openshift-default-7594fd6b75-2br8j" Jul 06 22:43:03.240240 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:03.240092 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9-istio-podinfo\") pod \"e2e-shared-a-65387d-openshift-default-7594fd6b75-2br8j\" (UID: \"f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9\") " pod="openshift-ingress/e2e-shared-a-65387d-openshift-default-7594fd6b75-2br8j" Jul 06 22:43:03.240240 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:03.240129 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9-workload-socket\") pod \"e2e-shared-a-65387d-openshift-default-7594fd6b75-2br8j\" (UID: \"f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9\") " pod="openshift-ingress/e2e-shared-a-65387d-openshift-default-7594fd6b75-2br8j" Jul 06 22:43:03.240544 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:03.240384 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9-workload-certs\") pod \"e2e-shared-a-65387d-openshift-default-7594fd6b75-2br8j\" (UID: \"f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9\") " pod="openshift-ingress/e2e-shared-a-65387d-openshift-default-7594fd6b75-2br8j" Jul 06 22:43:03.240544 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:03.240487 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9-workload-socket\") pod \"e2e-shared-a-65387d-openshift-default-7594fd6b75-2br8j\" (UID: \"f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9\") " pod="openshift-ingress/e2e-shared-a-65387d-openshift-default-7594fd6b75-2br8j" Jul 06 22:43:03.240686 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:03.240661 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9-istio-data\") pod \"e2e-shared-a-65387d-openshift-default-7594fd6b75-2br8j\" (UID: \"f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9\") " pod="openshift-ingress/e2e-shared-a-65387d-openshift-default-7594fd6b75-2br8j" Jul 06 22:43:03.240802 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:03.240777 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9-istiod-ca-cert\") pod \"e2e-shared-a-65387d-openshift-default-7594fd6b75-2br8j\" (UID: \"f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9\") " pod="openshift-ingress/e2e-shared-a-65387d-openshift-default-7594fd6b75-2br8j" Jul 06 22:43:03.240895 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:03.240879 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9-credential-socket\") pod \"e2e-shared-a-65387d-openshift-default-7594fd6b75-2br8j\" (UID: \"f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9\") " pod="openshift-ingress/e2e-shared-a-65387d-openshift-default-7594fd6b75-2br8j" Jul 06 22:43:03.242284 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:03.242266 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9-istio-envoy\") pod \"e2e-shared-a-65387d-openshift-default-7594fd6b75-2br8j\" (UID: \"f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9\") " pod="openshift-ingress/e2e-shared-a-65387d-openshift-default-7594fd6b75-2br8j" Jul 06 22:43:03.242513 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:03.242493 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9-istio-podinfo\") pod \"e2e-shared-a-65387d-openshift-default-7594fd6b75-2br8j\" (UID: \"f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9\") " pod="openshift-ingress/e2e-shared-a-65387d-openshift-default-7594fd6b75-2br8j" Jul 06 22:43:03.247891 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:03.247867 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9kf8s\" (UniqueName: \"kubernetes.io/projected/f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9-kube-api-access-9kf8s\") pod \"e2e-shared-a-65387d-openshift-default-7594fd6b75-2br8j\" (UID: \"f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9\") " pod="openshift-ingress/e2e-shared-a-65387d-openshift-default-7594fd6b75-2br8j" Jul 06 22:43:03.247991 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:03.247952 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9-istio-token\") pod \"e2e-shared-a-65387d-openshift-default-7594fd6b75-2br8j\" (UID: \"f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9\") " pod="openshift-ingress/e2e-shared-a-65387d-openshift-default-7594fd6b75-2br8j" Jul 06 22:43:03.322035 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:03.321950 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-65387d-openshift-default-7594fd6b75-2br8j" Jul 06 22:43:03.341460 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:03.341426 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:43:03.467135 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:03.467105 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-65387d-openshift-default-7594fd6b75-2br8j"] Jul 06 22:43:03.468928 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:43:03.468901 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5b203b5_de40_4fb0_a2a7_d0cfe5f52aa9.slice/crio-27a07b539dd3346d1a61c5712db16503927a2d19fc4fe7304b53235f5197d72c WatchSource:0}: Error finding container 27a07b539dd3346d1a61c5712db16503927a2d19fc4fe7304b53235f5197d72c: Status 404 returned error can't find the container with id 27a07b539dd3346d1a61c5712db16503927a2d19fc4fe7304b53235f5197d72c Jul 06 22:43:03.471224 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:03.471185 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 22:43:03.471300 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:03.471271 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 22:43:03.471339 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:03.471313 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 22:43:04.413515 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:04.413482 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-65387d-openshift-default-7594fd6b75-2br8j" event={"ID":"f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9","Type":"ContainerStarted","Data":"cfa1306b96a3a87a56ce37d1b8d446d267cdab31205448e322a58da44652707c"} Jul 06 22:43:04.413890 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:04.413531 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-65387d-openshift-default-7594fd6b75-2br8j" event={"ID":"f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9","Type":"ContainerStarted","Data":"27a07b539dd3346d1a61c5712db16503927a2d19fc4fe7304b53235f5197d72c"} Jul 06 22:43:04.436048 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:04.435999 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-65387d-openshift-default-7594fd6b75-2br8j" podStartSLOduration=2.43598334 podStartE2EDuration="2.43598334s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:43:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:43:04.434011478 +0000 UTC m=+2063.992806800" watchObservedRunningTime="2026-07-06 22:43:04.43598334 +0000 UTC m=+2063.994778661" Jul 06 22:43:05.322947 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:05.322914 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-65387d-openshift-default-7594fd6b75-2br8j" Jul 06 22:43:05.327921 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:05.327896 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-65387d-openshift-default-7594fd6b75-2br8j" Jul 06 22:43:05.418977 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:05.418939 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-65387d-openshift-default-7594fd6b75-2br8j" Jul 06 22:43:05.419855 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:05.419838 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-65387d-openshift-default-7594fd6b75-2br8j" Jul 06 22:43:06.319969 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:06.319935 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:43:06.420280 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:06.420135 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:43:06.520647 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:06.520613 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:43:06.845875 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:06.845842 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:43:06.954459 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:06.954426 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:43:07.055464 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:07.055430 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:43:07.434176 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:07.433939 2581 generic.go:356] "Generic (PLEG): container finished" podID="ac73802a-b5e4-47bb-9f6e-d938e8705f13" containerID="11fc7b341eeb4d430e3bb27e9ac0d4c77d6b647c6d8546bfb21c46ee32a38074" exitCode=0 Jul 06 22:43:07.434176 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:07.433986 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-63b31a-openshift-default-7f4945587c-jgxdj" event={"ID":"ac73802a-b5e4-47bb-9f6e-d938e8705f13","Type":"ContainerDied","Data":"11fc7b341eeb4d430e3bb27e9ac0d4c77d6b647c6d8546bfb21c46ee32a38074"} Jul 06 22:43:07.434176 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:07.434087 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-63b31a-openshift-default-7f4945587c-jgxdj" event={"ID":"ac73802a-b5e4-47bb-9f6e-d938e8705f13","Type":"ContainerDied","Data":"9426136c434411bd6f3805d39150ee540295b52bf30f48eb59f57fb6685da60e"} Jul 06 22:43:07.434176 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:07.434119 2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9426136c434411bd6f3805d39150ee540295b52bf30f48eb59f57fb6685da60e" Jul 06 22:43:07.434176 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:07.434142 2581 scope.go:117] "RemoveContainer" containerID="3c7d4404974c155303bf46376d5959f29bb9c6866c4ebc7978e4714843c147dc" Jul 06 22:43:07.444881 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:07.442845 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-63b31a-openshift-default-7f4945587c-jgxdj" Jul 06 22:43:07.584001 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:07.583967 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ac73802a-b5e4-47bb-9f6e-d938e8705f13-istiod-ca-cert\") pod \"ac73802a-b5e4-47bb-9f6e-d938e8705f13\" (UID: \"ac73802a-b5e4-47bb-9f6e-d938e8705f13\") " Jul 06 22:43:07.584223 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:07.584050 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ac73802a-b5e4-47bb-9f6e-d938e8705f13-workload-certs\") pod \"ac73802a-b5e4-47bb-9f6e-d938e8705f13\" (UID: \"ac73802a-b5e4-47bb-9f6e-d938e8705f13\") " Jul 06 22:43:07.584223 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:07.584108 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ac73802a-b5e4-47bb-9f6e-d938e8705f13-istio-envoy\") pod \"ac73802a-b5e4-47bb-9f6e-d938e8705f13\" (UID: \"ac73802a-b5e4-47bb-9f6e-d938e8705f13\") " Jul 06 22:43:07.584223 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:07.584147 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-knvb7\" (UniqueName: \"kubernetes.io/projected/ac73802a-b5e4-47bb-9f6e-d938e8705f13-kube-api-access-knvb7\") pod \"ac73802a-b5e4-47bb-9f6e-d938e8705f13\" (UID: \"ac73802a-b5e4-47bb-9f6e-d938e8705f13\") " Jul 06 22:43:07.584223 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:07.584193 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ac73802a-b5e4-47bb-9f6e-d938e8705f13-istio-token\") pod \"ac73802a-b5e4-47bb-9f6e-d938e8705f13\" (UID: \"ac73802a-b5e4-47bb-9f6e-d938e8705f13\") " Jul 06 22:43:07.584223 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:07.584218 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ac73802a-b5e4-47bb-9f6e-d938e8705f13-istio-podinfo\") pod \"ac73802a-b5e4-47bb-9f6e-d938e8705f13\" (UID: \"ac73802a-b5e4-47bb-9f6e-d938e8705f13\") " Jul 06 22:43:07.584480 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:07.584262 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac73802a-b5e4-47bb-9f6e-d938e8705f13-workload-socket\") pod \"ac73802a-b5e4-47bb-9f6e-d938e8705f13\" (UID: \"ac73802a-b5e4-47bb-9f6e-d938e8705f13\") " Jul 06 22:43:07.584480 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:07.584300 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac73802a-b5e4-47bb-9f6e-d938e8705f13-credential-socket\") pod \"ac73802a-b5e4-47bb-9f6e-d938e8705f13\" (UID: \"ac73802a-b5e4-47bb-9f6e-d938e8705f13\") " Jul 06 22:43:07.584480 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:07.584366 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ac73802a-b5e4-47bb-9f6e-d938e8705f13-istio-data\") pod \"ac73802a-b5e4-47bb-9f6e-d938e8705f13\" (UID: \"ac73802a-b5e4-47bb-9f6e-d938e8705f13\") " Jul 06 22:43:07.584480 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:07.584381 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ac73802a-b5e4-47bb-9f6e-d938e8705f13-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ac73802a-b5e4-47bb-9f6e-d938e8705f13" (UID: "ac73802a-b5e4-47bb-9f6e-d938e8705f13"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:43:07.584480 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:07.584399 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ac73802a-b5e4-47bb-9f6e-d938e8705f13-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ac73802a-b5e4-47bb-9f6e-d938e8705f13" (UID: "ac73802a-b5e4-47bb-9f6e-d938e8705f13"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 22:43:07.584715 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:07.584658 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ac73802a-b5e4-47bb-9f6e-d938e8705f13-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ac73802a-b5e4-47bb-9f6e-d938e8705f13" (UID: "ac73802a-b5e4-47bb-9f6e-d938e8705f13"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:43:07.584715 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:07.584679 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ac73802a-b5e4-47bb-9f6e-d938e8705f13-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ac73802a-b5e4-47bb-9f6e-d938e8705f13" (UID: "ac73802a-b5e4-47bb-9f6e-d938e8705f13"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:43:07.584822 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:07.584807 2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac73802a-b5e4-47bb-9f6e-d938e8705f13-workload-socket\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:43:07.584885 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:07.584829 2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac73802a-b5e4-47bb-9f6e-d938e8705f13-credential-socket\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:43:07.584885 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:07.584845 2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ac73802a-b5e4-47bb-9f6e-d938e8705f13-istiod-ca-cert\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:43:07.584885 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:07.584858 2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ac73802a-b5e4-47bb-9f6e-d938e8705f13-workload-certs\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:43:07.585649 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:07.585623 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ac73802a-b5e4-47bb-9f6e-d938e8705f13-istio-data" (OuterVolumeSpecName: "istio-data") pod "ac73802a-b5e4-47bb-9f6e-d938e8705f13" (UID: "ac73802a-b5e4-47bb-9f6e-d938e8705f13"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:43:07.586591 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:07.586569 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ac73802a-b5e4-47bb-9f6e-d938e8705f13-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ac73802a-b5e4-47bb-9f6e-d938e8705f13" (UID: "ac73802a-b5e4-47bb-9f6e-d938e8705f13"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 22:43:07.586693 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:07.586640 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ac73802a-b5e4-47bb-9f6e-d938e8705f13-kube-api-access-knvb7" (OuterVolumeSpecName: "kube-api-access-knvb7") pod "ac73802a-b5e4-47bb-9f6e-d938e8705f13" (UID: "ac73802a-b5e4-47bb-9f6e-d938e8705f13"). InnerVolumeSpecName "kube-api-access-knvb7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 22:43:07.586759 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:07.586712 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ac73802a-b5e4-47bb-9f6e-d938e8705f13-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ac73802a-b5e4-47bb-9f6e-d938e8705f13" (UID: "ac73802a-b5e4-47bb-9f6e-d938e8705f13"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:43:07.586971 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:07.586955 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ac73802a-b5e4-47bb-9f6e-d938e8705f13-istio-token" (OuterVolumeSpecName: "istio-token") pod "ac73802a-b5e4-47bb-9f6e-d938e8705f13" (UID: "ac73802a-b5e4-47bb-9f6e-d938e8705f13"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 22:43:07.685484 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:07.685408 2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ac73802a-b5e4-47bb-9f6e-d938e8705f13-istio-token\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:43:07.685484 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:07.685433 2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ac73802a-b5e4-47bb-9f6e-d938e8705f13-istio-podinfo\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:43:07.685484 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:07.685442 2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ac73802a-b5e4-47bb-9f6e-d938e8705f13-istio-data\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:43:07.685484 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:07.685450 2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ac73802a-b5e4-47bb-9f6e-d938e8705f13-istio-envoy\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:43:07.685484 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:07.685459 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-knvb7\" (UniqueName: \"kubernetes.io/projected/ac73802a-b5e4-47bb-9f6e-d938e8705f13-kube-api-access-knvb7\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:43:08.441506 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:08.441479 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-63b31a-openshift-default-7f4945587c-jgxdj" Jul 06 22:43:08.477980 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:08.477950 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-63b31a-openshift-default-7f4945587c-jgxdj"] Jul 06 22:43:08.482272 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:08.482250 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-63b31a-openshift-default-7f4945587c-jgxdj"] Jul 06 22:43:08.977466 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:08.977436 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ac73802a-b5e4-47bb-9f6e-d938e8705f13" path="/var/lib/kubelet/pods/ac73802a-b5e4-47bb-9f6e-d938e8705f13/volumes" Jul 06 22:43:10.807191 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:10.807159 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:43:11.135397 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:11.135319 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:43:11.413980 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:11.413888 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-944f9f-openshift-default-847d84d4c9-6v6r9"] Jul 06 22:43:11.414516 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:11.414495 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac73802a-b5e4-47bb-9f6e-d938e8705f13" containerName="istio-proxy" Jul 06 22:43:11.414516 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:11.414516 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac73802a-b5e4-47bb-9f6e-d938e8705f13" containerName="istio-proxy" Jul 06 22:43:11.414689 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:11.414610 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="ac73802a-b5e4-47bb-9f6e-d938e8705f13" containerName="istio-proxy" Jul 06 22:43:11.419307 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:11.419285 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-944f9f-openshift-default-847d84d4c9-6v6r9" Jul 06 22:43:11.421762 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:11.421742 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-944f9f-openshift-default-dockercfg-grb4p\"" Jul 06 22:43:11.429395 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:11.429367 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-944f9f-openshift-default-847d84d4c9-6v6r9"] Jul 06 22:43:11.447415 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:11.447377 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:43:11.519805 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:11.519772 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6c3e1570-bd19-430d-8065-19aeb291f3ad-workload-certs\") pod \"e2e-shared-b-944f9f-openshift-default-847d84d4c9-6v6r9\" (UID: \"6c3e1570-bd19-430d-8065-19aeb291f3ad\") " pod="openshift-ingress/e2e-shared-b-944f9f-openshift-default-847d84d4c9-6v6r9" Jul 06 22:43:11.520016 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:11.519833 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6c3e1570-bd19-430d-8065-19aeb291f3ad-istio-envoy\") pod \"e2e-shared-b-944f9f-openshift-default-847d84d4c9-6v6r9\" (UID: \"6c3e1570-bd19-430d-8065-19aeb291f3ad\") " pod="openshift-ingress/e2e-shared-b-944f9f-openshift-default-847d84d4c9-6v6r9" Jul 06 22:43:11.520016 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:11.519917 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6c3e1570-bd19-430d-8065-19aeb291f3ad-istio-data\") pod \"e2e-shared-b-944f9f-openshift-default-847d84d4c9-6v6r9\" (UID: \"6c3e1570-bd19-430d-8065-19aeb291f3ad\") " pod="openshift-ingress/e2e-shared-b-944f9f-openshift-default-847d84d4c9-6v6r9" Jul 06 22:43:11.520016 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:11.520004 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6c3e1570-bd19-430d-8065-19aeb291f3ad-credential-socket\") pod \"e2e-shared-b-944f9f-openshift-default-847d84d4c9-6v6r9\" (UID: \"6c3e1570-bd19-430d-8065-19aeb291f3ad\") " pod="openshift-ingress/e2e-shared-b-944f9f-openshift-default-847d84d4c9-6v6r9" Jul 06 22:43:11.520222 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:11.520034 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6c3e1570-bd19-430d-8065-19aeb291f3ad-workload-socket\") pod \"e2e-shared-b-944f9f-openshift-default-847d84d4c9-6v6r9\" (UID: \"6c3e1570-bd19-430d-8065-19aeb291f3ad\") " pod="openshift-ingress/e2e-shared-b-944f9f-openshift-default-847d84d4c9-6v6r9" Jul 06 22:43:11.520222 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:11.520158 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-66mpv\" (UniqueName: \"kubernetes.io/projected/6c3e1570-bd19-430d-8065-19aeb291f3ad-kube-api-access-66mpv\") pod \"e2e-shared-b-944f9f-openshift-default-847d84d4c9-6v6r9\" (UID: \"6c3e1570-bd19-430d-8065-19aeb291f3ad\") " pod="openshift-ingress/e2e-shared-b-944f9f-openshift-default-847d84d4c9-6v6r9" Jul 06 22:43:11.520222 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:11.520205 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6c3e1570-bd19-430d-8065-19aeb291f3ad-istio-podinfo\") pod \"e2e-shared-b-944f9f-openshift-default-847d84d4c9-6v6r9\" (UID: \"6c3e1570-bd19-430d-8065-19aeb291f3ad\") " pod="openshift-ingress/e2e-shared-b-944f9f-openshift-default-847d84d4c9-6v6r9" Jul 06 22:43:11.520377 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:11.520248 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6c3e1570-bd19-430d-8065-19aeb291f3ad-istiod-ca-cert\") pod \"e2e-shared-b-944f9f-openshift-default-847d84d4c9-6v6r9\" (UID: \"6c3e1570-bd19-430d-8065-19aeb291f3ad\") " pod="openshift-ingress/e2e-shared-b-944f9f-openshift-default-847d84d4c9-6v6r9" Jul 06 22:43:11.520377 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:11.520281 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6c3e1570-bd19-430d-8065-19aeb291f3ad-istio-token\") pod \"e2e-shared-b-944f9f-openshift-default-847d84d4c9-6v6r9\" (UID: \"6c3e1570-bd19-430d-8065-19aeb291f3ad\") " pod="openshift-ingress/e2e-shared-b-944f9f-openshift-default-847d84d4c9-6v6r9" Jul 06 22:43:11.621805 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:11.621768 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-66mpv\" (UniqueName: \"kubernetes.io/projected/6c3e1570-bd19-430d-8065-19aeb291f3ad-kube-api-access-66mpv\") pod \"e2e-shared-b-944f9f-openshift-default-847d84d4c9-6v6r9\" (UID: \"6c3e1570-bd19-430d-8065-19aeb291f3ad\") " pod="openshift-ingress/e2e-shared-b-944f9f-openshift-default-847d84d4c9-6v6r9" Jul 06 22:43:11.621979 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:11.621813 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6c3e1570-bd19-430d-8065-19aeb291f3ad-istio-podinfo\") pod \"e2e-shared-b-944f9f-openshift-default-847d84d4c9-6v6r9\" (UID: \"6c3e1570-bd19-430d-8065-19aeb291f3ad\") " pod="openshift-ingress/e2e-shared-b-944f9f-openshift-default-847d84d4c9-6v6r9" Jul 06 22:43:11.621979 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:11.621887 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6c3e1570-bd19-430d-8065-19aeb291f3ad-istiod-ca-cert\") pod \"e2e-shared-b-944f9f-openshift-default-847d84d4c9-6v6r9\" (UID: \"6c3e1570-bd19-430d-8065-19aeb291f3ad\") " pod="openshift-ingress/e2e-shared-b-944f9f-openshift-default-847d84d4c9-6v6r9" Jul 06 22:43:11.621979 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:11.621937 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6c3e1570-bd19-430d-8065-19aeb291f3ad-istio-token\") pod \"e2e-shared-b-944f9f-openshift-default-847d84d4c9-6v6r9\" (UID: \"6c3e1570-bd19-430d-8065-19aeb291f3ad\") " pod="openshift-ingress/e2e-shared-b-944f9f-openshift-default-847d84d4c9-6v6r9" Jul 06 22:43:11.621979 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:11.621962 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6c3e1570-bd19-430d-8065-19aeb291f3ad-workload-certs\") pod \"e2e-shared-b-944f9f-openshift-default-847d84d4c9-6v6r9\" (UID: \"6c3e1570-bd19-430d-8065-19aeb291f3ad\") " pod="openshift-ingress/e2e-shared-b-944f9f-openshift-default-847d84d4c9-6v6r9" Jul 06 22:43:11.622240 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:11.622124 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6c3e1570-bd19-430d-8065-19aeb291f3ad-istio-envoy\") pod \"e2e-shared-b-944f9f-openshift-default-847d84d4c9-6v6r9\" (UID: \"6c3e1570-bd19-430d-8065-19aeb291f3ad\") " pod="openshift-ingress/e2e-shared-b-944f9f-openshift-default-847d84d4c9-6v6r9" Jul 06 22:43:11.622240 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:11.622175 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6c3e1570-bd19-430d-8065-19aeb291f3ad-istio-data\") pod \"e2e-shared-b-944f9f-openshift-default-847d84d4c9-6v6r9\" (UID: \"6c3e1570-bd19-430d-8065-19aeb291f3ad\") " pod="openshift-ingress/e2e-shared-b-944f9f-openshift-default-847d84d4c9-6v6r9" Jul 06 22:43:11.622341 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:11.622265 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6c3e1570-bd19-430d-8065-19aeb291f3ad-credential-socket\") pod \"e2e-shared-b-944f9f-openshift-default-847d84d4c9-6v6r9\" (UID: \"6c3e1570-bd19-430d-8065-19aeb291f3ad\") " pod="openshift-ingress/e2e-shared-b-944f9f-openshift-default-847d84d4c9-6v6r9" Jul 06 22:43:11.622341 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:11.622301 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6c3e1570-bd19-430d-8065-19aeb291f3ad-workload-socket\") pod \"e2e-shared-b-944f9f-openshift-default-847d84d4c9-6v6r9\" (UID: \"6c3e1570-bd19-430d-8065-19aeb291f3ad\") " pod="openshift-ingress/e2e-shared-b-944f9f-openshift-default-847d84d4c9-6v6r9" Jul 06 22:43:11.622610 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:11.622547 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6c3e1570-bd19-430d-8065-19aeb291f3ad-istio-data\") pod \"e2e-shared-b-944f9f-openshift-default-847d84d4c9-6v6r9\" (UID: \"6c3e1570-bd19-430d-8065-19aeb291f3ad\") " pod="openshift-ingress/e2e-shared-b-944f9f-openshift-default-847d84d4c9-6v6r9" Jul 06 22:43:11.622749 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:11.622662 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6c3e1570-bd19-430d-8065-19aeb291f3ad-workload-socket\") pod \"e2e-shared-b-944f9f-openshift-default-847d84d4c9-6v6r9\" (UID: \"6c3e1570-bd19-430d-8065-19aeb291f3ad\") " pod="openshift-ingress/e2e-shared-b-944f9f-openshift-default-847d84d4c9-6v6r9" Jul 06 22:43:11.622749 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:11.622724 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6c3e1570-bd19-430d-8065-19aeb291f3ad-credential-socket\") pod \"e2e-shared-b-944f9f-openshift-default-847d84d4c9-6v6r9\" (UID: \"6c3e1570-bd19-430d-8065-19aeb291f3ad\") " pod="openshift-ingress/e2e-shared-b-944f9f-openshift-default-847d84d4c9-6v6r9" Jul 06 22:43:11.622871 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:11.622727 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6c3e1570-bd19-430d-8065-19aeb291f3ad-istiod-ca-cert\") pod \"e2e-shared-b-944f9f-openshift-default-847d84d4c9-6v6r9\" (UID: \"6c3e1570-bd19-430d-8065-19aeb291f3ad\") " pod="openshift-ingress/e2e-shared-b-944f9f-openshift-default-847d84d4c9-6v6r9" Jul 06 22:43:11.622871 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:11.622759 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6c3e1570-bd19-430d-8065-19aeb291f3ad-workload-certs\") pod \"e2e-shared-b-944f9f-openshift-default-847d84d4c9-6v6r9\" (UID: \"6c3e1570-bd19-430d-8065-19aeb291f3ad\") " pod="openshift-ingress/e2e-shared-b-944f9f-openshift-default-847d84d4c9-6v6r9" Jul 06 22:43:11.624392 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:11.624373 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6c3e1570-bd19-430d-8065-19aeb291f3ad-istio-envoy\") pod \"e2e-shared-b-944f9f-openshift-default-847d84d4c9-6v6r9\" (UID: \"6c3e1570-bd19-430d-8065-19aeb291f3ad\") " pod="openshift-ingress/e2e-shared-b-944f9f-openshift-default-847d84d4c9-6v6r9" Jul 06 22:43:11.624474 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:11.624401 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6c3e1570-bd19-430d-8065-19aeb291f3ad-istio-podinfo\") pod \"e2e-shared-b-944f9f-openshift-default-847d84d4c9-6v6r9\" (UID: \"6c3e1570-bd19-430d-8065-19aeb291f3ad\") " pod="openshift-ingress/e2e-shared-b-944f9f-openshift-default-847d84d4c9-6v6r9" Jul 06 22:43:11.629433 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:11.629407 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-66mpv\" (UniqueName: \"kubernetes.io/projected/6c3e1570-bd19-430d-8065-19aeb291f3ad-kube-api-access-66mpv\") pod \"e2e-shared-b-944f9f-openshift-default-847d84d4c9-6v6r9\" (UID: \"6c3e1570-bd19-430d-8065-19aeb291f3ad\") " pod="openshift-ingress/e2e-shared-b-944f9f-openshift-default-847d84d4c9-6v6r9" Jul 06 22:43:11.629503 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:11.629468 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6c3e1570-bd19-430d-8065-19aeb291f3ad-istio-token\") pod \"e2e-shared-b-944f9f-openshift-default-847d84d4c9-6v6r9\" (UID: \"6c3e1570-bd19-430d-8065-19aeb291f3ad\") " pod="openshift-ingress/e2e-shared-b-944f9f-openshift-default-847d84d4c9-6v6r9" Jul 06 22:43:11.730485 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:11.730453 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-944f9f-openshift-default-847d84d4c9-6v6r9" Jul 06 22:43:11.883919 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:11.883893 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-944f9f-openshift-default-847d84d4c9-6v6r9"] Jul 06 22:43:11.885828 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:43:11.885800 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c3e1570_bd19_430d_8065_19aeb291f3ad.slice/crio-0cccdc56e5591a5cccbf9a7306b4b4604525681472393f214c92e3099a9a7156 WatchSource:0}: Error finding container 0cccdc56e5591a5cccbf9a7306b4b4604525681472393f214c92e3099a9a7156: Status 404 returned error can't find the container with id 0cccdc56e5591a5cccbf9a7306b4b4604525681472393f214c92e3099a9a7156 Jul 06 22:43:11.888029 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:11.887991 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 22:43:11.888144 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:11.888099 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 22:43:11.888214 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:11.888168 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 22:43:12.468495 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:12.468448 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-944f9f-openshift-default-847d84d4c9-6v6r9" event={"ID":"6c3e1570-bd19-430d-8065-19aeb291f3ad","Type":"ContainerStarted","Data":"14baa758516dd9a905a60798f58eefe274404e1995f184a4032c9ab7f43f5a9c"} Jul 06 22:43:12.468657 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:12.468522 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-944f9f-openshift-default-847d84d4c9-6v6r9" event={"ID":"6c3e1570-bd19-430d-8065-19aeb291f3ad","Type":"ContainerStarted","Data":"0cccdc56e5591a5cccbf9a7306b4b4604525681472393f214c92e3099a9a7156"} Jul 06 22:43:12.490172 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:12.490128 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-944f9f-openshift-default-847d84d4c9-6v6r9" podStartSLOduration=1.490113038 podStartE2EDuration="1.490113038s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:43:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:43:12.487968505 +0000 UTC m=+2072.046763824" watchObservedRunningTime="2026-07-06 22:43:12.490113038 +0000 UTC m=+2072.048908359" Jul 06 22:43:12.561977 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:12.561940 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:43:12.731017 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:12.730923 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-944f9f-openshift-default-847d84d4c9-6v6r9" Jul 06 22:43:12.740750 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:12.740717 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-944f9f-openshift-default-847d84d4c9-6v6r9" Jul 06 22:43:13.474203 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:13.474173 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-944f9f-openshift-default-847d84d4c9-6v6r9" Jul 06 22:43:13.475088 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:13.475049 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-944f9f-openshift-default-847d84d4c9-6v6r9" Jul 06 22:43:14.377730 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:14.377695 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:43:14.477380 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:14.477349 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:43:14.644146 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:14.643387 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:43:14.842243 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:14.842208 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:43:15.269723 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:15.269692 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:43:15.370089 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:15.369984 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:43:18.878345 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:18.878304 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:43:19.507402 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:19.507365 2581 generic.go:356] "Generic (PLEG): container finished" podID="f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9" containerID="cfa1306b96a3a87a56ce37d1b8d446d267cdab31205448e322a58da44652707c" exitCode=0 Jul 06 22:43:19.507649 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:19.507457 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-65387d-openshift-default-7594fd6b75-2br8j" event={"ID":"f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9","Type":"ContainerDied","Data":"cfa1306b96a3a87a56ce37d1b8d446d267cdab31205448e322a58da44652707c"} Jul 06 22:43:19.507922 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:19.507905 2581 scope.go:117] "RemoveContainer" containerID="cfa1306b96a3a87a56ce37d1b8d446d267cdab31205448e322a58da44652707c" Jul 06 22:43:19.508932 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:19.508903 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 22:43:19.509027 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:19.508986 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 22:43:19.509121 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:19.509037 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 22:43:20.419587 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:20.419552 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-65387d-openshift-default-7594fd6b75-2br8j" Jul 06 22:43:20.514316 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:20.514281 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-65387d-openshift-default-7594fd6b75-2br8j" event={"ID":"f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9","Type":"ContainerStarted","Data":"d01409fdce09c5d675834e0fe02bad9e3a965b55d6dff9e64b2d094d4f5a0bf3"} Jul 06 22:43:21.323182 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:21.323147 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-65387d-openshift-default-7594fd6b75-2br8j" Jul 06 22:43:21.324264 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:21.324240 2581 patch_prober.go:28] interesting pod/e2e-shared-a-65387d-openshift-default-7594fd6b75-2br8j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body= Jul 06 22:43:21.324346 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:21.324292 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-65387d-openshift-default-7594fd6b75-2br8j" podUID="f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" Jul 06 22:43:22.323247 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:22.323204 2581 patch_prober.go:28] interesting pod/e2e-shared-a-65387d-openshift-default-7594fd6b75-2br8j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body= Jul 06 22:43:22.323682 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:22.323655 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-65387d-openshift-default-7594fd6b75-2br8j" podUID="f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" Jul 06 22:43:23.323306 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:23.323201 2581 patch_prober.go:28] interesting pod/e2e-shared-a-65387d-openshift-default-7594fd6b75-2br8j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body= Jul 06 22:43:23.323306 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:23.323290 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-65387d-openshift-default-7594fd6b75-2br8j" podUID="f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" Jul 06 22:43:24.322468 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:24.322420 2581 patch_prober.go:28] interesting pod/e2e-shared-a-65387d-openshift-default-7594fd6b75-2br8j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body= Jul 06 22:43:24.322730 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:24.322503 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-65387d-openshift-default-7594fd6b75-2br8j" podUID="f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" Jul 06 22:43:25.327377 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:25.327346 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-65387d-openshift-default-7594fd6b75-2br8j" Jul 06 22:43:25.327858 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:25.327845 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-65387d-openshift-default-7594fd6b75-2br8j" Jul 06 22:43:25.328467 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:25.328451 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-65387d-openshift-default-7594fd6b75-2br8j" Jul 06 22:43:25.634613 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:25.634503 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:43:26.551127 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:26.551097 2581 generic.go:356] "Generic (PLEG): container finished" podID="6c3e1570-bd19-430d-8065-19aeb291f3ad" containerID="14baa758516dd9a905a60798f58eefe274404e1995f184a4032c9ab7f43f5a9c" exitCode=0 Jul 06 22:43:26.551568 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:26.551159 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-944f9f-openshift-default-847d84d4c9-6v6r9" event={"ID":"6c3e1570-bd19-430d-8065-19aeb291f3ad","Type":"ContainerDied","Data":"14baa758516dd9a905a60798f58eefe274404e1995f184a4032c9ab7f43f5a9c"} Jul 06 22:43:26.551646 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:26.551633 2581 scope.go:117] "RemoveContainer" containerID="14baa758516dd9a905a60798f58eefe274404e1995f184a4032c9ab7f43f5a9c" Jul 06 22:43:26.552560 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:26.552529 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 22:43:26.552662 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:26.552614 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 22:43:26.552727 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:26.552704 2581 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 06 22:43:27.562472 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:27.562437 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-944f9f-openshift-default-847d84d4c9-6v6r9" event={"ID":"6c3e1570-bd19-430d-8065-19aeb291f3ad","Type":"ContainerStarted","Data":"2f39e993223d02c4756b532c4095e3b91cde395d485ebb08626d531f2512bff3"} Jul 06 22:43:27.730963 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:27.730929 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-944f9f-openshift-default-847d84d4c9-6v6r9" Jul 06 22:43:27.732139 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:27.732110 2581 patch_prober.go:28] interesting pod/e2e-shared-b-944f9f-openshift-default-847d84d4c9-6v6r9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body= Jul 06 22:43:27.732253 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:27.732169 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-944f9f-openshift-default-847d84d4c9-6v6r9" podUID="6c3e1570-bd19-430d-8065-19aeb291f3ad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" Jul 06 22:43:28.231381 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:28.231349 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:43:28.331696 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:28.331650 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:43:28.474950 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:28.474897 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-944f9f-openshift-default-847d84d4c9-6v6r9" Jul 06 22:43:28.730824 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:28.730786 2581 patch_prober.go:28] interesting pod/e2e-shared-b-944f9f-openshift-default-847d84d4c9-6v6r9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body= Jul 06 22:43:28.731251 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:28.730855 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-944f9f-openshift-default-847d84d4c9-6v6r9" podUID="6c3e1570-bd19-430d-8065-19aeb291f3ad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" Jul 06 22:43:29.731334 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:29.731298 2581 patch_prober.go:28] interesting pod/e2e-shared-b-944f9f-openshift-default-847d84d4c9-6v6r9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body= Jul 06 22:43:29.731713 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:29.731356 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-944f9f-openshift-default-847d84d4c9-6v6r9" podUID="6c3e1570-bd19-430d-8065-19aeb291f3ad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" Jul 06 22:43:30.731810 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:30.731776 2581 patch_prober.go:28] interesting pod/e2e-shared-b-944f9f-openshift-default-847d84d4c9-6v6r9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body= Jul 06 22:43:30.732230 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:30.731841 2581 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-944f9f-openshift-default-847d84d4c9-6v6r9" podUID="6c3e1570-bd19-430d-8065-19aeb291f3ad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" Jul 06 22:43:31.715363 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:31.715330 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:43:31.736124 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:31.736095 2581 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-944f9f-openshift-default-847d84d4c9-6v6r9" Jul 06 22:43:31.737180 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:31.737161 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-944f9f-openshift-default-847d84d4c9-6v6r9" Jul 06 22:43:36.561541 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:36.561510 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:43:36.667233 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:36.667185 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:43:41.058267 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:41.058237 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-rcn5q_86b07166-2f34-443b-9c61-86b7d950ad7b/console-operator/1.log" Jul 06 22:43:41.064163 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:41.064142 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-rcn5q_86b07166-2f34-443b-9c61-86b7d950ad7b/console-operator/1.log" Jul 06 22:43:57.630709 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:43:57.630674 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:45:03.841446 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:45:03.841412 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:45:05.345635 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:45:05.345598 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:45:47.601735 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:45:47.601695 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:45:48.613424 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:45:48.613394 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:45:58.516376 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:45:58.516320 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:46:11.011837 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:46:11.011758 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:46:11.065049 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:46:11.065008 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:46:22.306849 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:46:22.306811 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:47:20.927559 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:47:20.927520 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:47:21.898533 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:47:21.898500 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:47:29.328718 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:47:29.328683 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:47:47.133047 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:47:47.132964 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:47:51.107677 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:47:51.107639 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:47:51.165111 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:47:51.165053 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:47:58.175398 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:47:58.175358 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:47:58.268501 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:47:58.268392 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:48:29.509492 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:48:29.509455 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:48:29.657169 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:48:29.657131 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:48:34.947839 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:48:34.947806 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:48:35.996632 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:48:35.996602 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:48:37.140846 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:48:37.140811 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:48:37.528819 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:48:37.528781 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:48:37.598271 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:48:37.598237 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:48:38.073401 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:48:38.073368 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:48:38.174744 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:48:38.174710 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:48:41.089490 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:48:41.089465 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-rcn5q_86b07166-2f34-443b-9c61-86b7d950ad7b/console-operator/1.log" Jul 06 22:48:41.096699 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:48:41.096674 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-rcn5q_86b07166-2f34-443b-9c61-86b7d950ad7b/console-operator/1.log" Jul 06 22:48:42.600217 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:48:42.600180 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:49:19.318998 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:19.318893 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:49:19.553038 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:19.553000 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-65387d-openshift-default-7594fd6b75-2br8j"] Jul 06 22:49:19.553250 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:19.553231 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-65387d-openshift-default-7594fd6b75-2br8j" podUID="f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9" containerName="istio-proxy" containerID="cri-o://d01409fdce09c5d675834e0fe02bad9e3a965b55d6dff9e64b2d094d4f5a0bf3" gracePeriod=30 Jul 06 22:49:19.574690 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:19.574611 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:49:19.793938 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:19.793907 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:49:19.992691 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:19.992659 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:49:20.352375 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:20.352254 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:49:20.451302 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:20.451271 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:49:24.814148 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:24.814121 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-65387d-openshift-default-7594fd6b75-2br8j" Jul 06 22:49:24.941898 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:24.941863 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9-istio-data\") pod \"f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9\" (UID: \"f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9\") " Jul 06 22:49:24.942137 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:24.941905 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9-istio-podinfo\") pod \"f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9\" (UID: \"f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9\") " Jul 06 22:49:24.942137 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:24.941928 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9-istio-envoy\") pod \"f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9\" (UID: \"f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9\") " Jul 06 22:49:24.942137 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:24.941946 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9-credential-socket\") pod \"f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9\" (UID: \"f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9\") " Jul 06 22:49:24.942137 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:24.941971 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9-workload-certs\") pod \"f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9\" (UID: \"f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9\") " Jul 06 22:49:24.942137 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:24.942127 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9-istiod-ca-cert\") pod \"f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9\" (UID: \"f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9\") " Jul 06 22:49:24.942431 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:24.942182 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9-istio-token\") pod \"f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9\" (UID: \"f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9\") " Jul 06 22:49:24.942431 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:24.942215 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9kf8s\" (UniqueName: \"kubernetes.io/projected/f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9-kube-api-access-9kf8s\") pod \"f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9\" (UID: \"f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9\") " Jul 06 22:49:24.942431 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:24.942318 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9-workload-socket\") pod \"f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9\" (UID: \"f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9\") " Jul 06 22:49:24.942431 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:24.942324 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9" (UID: "f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:49:24.942431 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:24.942360 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9" (UID: "f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:49:24.942680 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:24.942501 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9" (UID: "f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 22:49:24.942733 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:24.942675 2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9-credential-socket\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:49:24.942733 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:24.942694 2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9-workload-certs\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:49:24.942733 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:24.942707 2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9-istiod-ca-cert\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:49:24.942874 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:24.942766 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9" (UID: "f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:49:24.942941 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:24.942914 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9-istio-data" (OuterVolumeSpecName: "istio-data") pod "f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9" (UID: "f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:49:24.944180 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:24.944160 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9" (UID: "f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:49:24.944388 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:24.944357 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9-kube-api-access-9kf8s" (OuterVolumeSpecName: "kube-api-access-9kf8s") pod "f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9" (UID: "f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9"). InnerVolumeSpecName "kube-api-access-9kf8s". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 22:49:24.944688 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:24.944661 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9-istio-token" (OuterVolumeSpecName: "istio-token") pod "f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9" (UID: "f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 22:49:24.944747 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:24.944715 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9" (UID: "f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 22:49:25.043945 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:25.043903 2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9-istio-data\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:49:25.043945 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:25.043932 2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9-istio-podinfo\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:49:25.043945 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:25.043940 2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9-istio-envoy\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:49:25.043945 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:25.043948 2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9-istio-token\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:49:25.043945 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:25.043958 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9kf8s\" (UniqueName: \"kubernetes.io/projected/f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9-kube-api-access-9kf8s\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:49:25.044398 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:25.043966 2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9-workload-socket\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:49:25.422243 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:25.422155 2581 generic.go:356] "Generic (PLEG): container finished" podID="f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9" containerID="d01409fdce09c5d675834e0fe02bad9e3a965b55d6dff9e64b2d094d4f5a0bf3" exitCode=0 Jul 06 22:49:25.422243 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:25.422204 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-65387d-openshift-default-7594fd6b75-2br8j" event={"ID":"f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9","Type":"ContainerDied","Data":"d01409fdce09c5d675834e0fe02bad9e3a965b55d6dff9e64b2d094d4f5a0bf3"} Jul 06 22:49:25.422426 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:25.422247 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-65387d-openshift-default-7594fd6b75-2br8j" event={"ID":"f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9","Type":"ContainerDied","Data":"27a07b539dd3346d1a61c5712db16503927a2d19fc4fe7304b53235f5197d72c"} Jul 06 22:49:25.422426 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:25.422270 2581 scope.go:117] "RemoveContainer" containerID="d01409fdce09c5d675834e0fe02bad9e3a965b55d6dff9e64b2d094d4f5a0bf3" Jul 06 22:49:25.422426 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:25.422207 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-65387d-openshift-default-7594fd6b75-2br8j" Jul 06 22:49:25.431494 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:25.431472 2581 scope.go:117] "RemoveContainer" containerID="cfa1306b96a3a87a56ce37d1b8d446d267cdab31205448e322a58da44652707c" Jul 06 22:49:25.440728 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:25.440703 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-65387d-openshift-default-7594fd6b75-2br8j"] Jul 06 22:49:25.441799 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:25.441781 2581 scope.go:117] "RemoveContainer" containerID="d01409fdce09c5d675834e0fe02bad9e3a965b55d6dff9e64b2d094d4f5a0bf3" Jul 06 22:49:25.442108 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:49:25.442087 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d01409fdce09c5d675834e0fe02bad9e3a965b55d6dff9e64b2d094d4f5a0bf3\": container with ID starting with d01409fdce09c5d675834e0fe02bad9e3a965b55d6dff9e64b2d094d4f5a0bf3 not found: ID does not exist" containerID="d01409fdce09c5d675834e0fe02bad9e3a965b55d6dff9e64b2d094d4f5a0bf3" Jul 06 22:49:25.442196 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:25.442116 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d01409fdce09c5d675834e0fe02bad9e3a965b55d6dff9e64b2d094d4f5a0bf3"} err="failed to get container status \"d01409fdce09c5d675834e0fe02bad9e3a965b55d6dff9e64b2d094d4f5a0bf3\": rpc error: code = NotFound desc = could not find container \"d01409fdce09c5d675834e0fe02bad9e3a965b55d6dff9e64b2d094d4f5a0bf3\": container with ID starting with d01409fdce09c5d675834e0fe02bad9e3a965b55d6dff9e64b2d094d4f5a0bf3 not found: ID does not exist" Jul 06 22:49:25.442196 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:25.442135 2581 scope.go:117] "RemoveContainer" containerID="cfa1306b96a3a87a56ce37d1b8d446d267cdab31205448e322a58da44652707c" Jul 06 22:49:25.442399 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:49:25.442376 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cfa1306b96a3a87a56ce37d1b8d446d267cdab31205448e322a58da44652707c\": container with ID starting with cfa1306b96a3a87a56ce37d1b8d446d267cdab31205448e322a58da44652707c not found: ID does not exist" containerID="cfa1306b96a3a87a56ce37d1b8d446d267cdab31205448e322a58da44652707c" Jul 06 22:49:25.442469 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:25.442405 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cfa1306b96a3a87a56ce37d1b8d446d267cdab31205448e322a58da44652707c"} err="failed to get container status \"cfa1306b96a3a87a56ce37d1b8d446d267cdab31205448e322a58da44652707c\": rpc error: code = NotFound desc = could not find container \"cfa1306b96a3a87a56ce37d1b8d446d267cdab31205448e322a58da44652707c\": container with ID starting with cfa1306b96a3a87a56ce37d1b8d446d267cdab31205448e322a58da44652707c not found: ID does not exist" Jul 06 22:49:25.443328 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:25.443305 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-65387d-openshift-default-7594fd6b75-2br8j"] Jul 06 22:49:26.375684 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:26.375645 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:49:26.476179 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:26.476146 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:49:26.611605 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:26.611565 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-944f9f-openshift-default-847d84d4c9-6v6r9"] Jul 06 22:49:26.611885 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:26.611860 2581 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-944f9f-openshift-default-847d84d4c9-6v6r9" podUID="6c3e1570-bd19-430d-8065-19aeb291f3ad" containerName="istio-proxy" containerID="cri-o://2f39e993223d02c4756b532c4095e3b91cde395d485ebb08626d531f2512bff3" gracePeriod=30 Jul 06 22:49:26.636918 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:26.636835 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:49:26.853257 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:26.853223 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:49:26.977354 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:26.977320 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9" path="/var/lib/kubelet/pods/f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9/volumes" Jul 06 22:49:31.738809 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:31.738776 2581 patch_prober.go:28] interesting pod/e2e-shared-b-944f9f-openshift-default-847d84d4c9-6v6r9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body= Jul 06 22:49:31.739292 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:31.738848 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-944f9f-openshift-default-847d84d4c9-6v6r9" podUID="6c3e1570-bd19-430d-8065-19aeb291f3ad" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" Jul 06 22:49:31.788646 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:31.788619 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-wn2sx_ab6dcafe-a3c5-41d3-92ae-55aafb915634/manager/0.log" Jul 06 22:49:31.876040 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:31.876009 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-944f9f-openshift-default-847d84d4c9-6v6r9" Jul 06 22:49:31.906939 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:31.906860 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-4swjt_5ad9d1ac-6623-4ff6-8c6d-6003ee007795/manager/0.log" Jul 06 22:49:31.907092 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:31.906938 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6c3e1570-bd19-430d-8065-19aeb291f3ad-istio-podinfo\") pod \"6c3e1570-bd19-430d-8065-19aeb291f3ad\" (UID: \"6c3e1570-bd19-430d-8065-19aeb291f3ad\") " Jul 06 22:49:31.907092 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:31.906972 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6c3e1570-bd19-430d-8065-19aeb291f3ad-credential-socket\") pod \"6c3e1570-bd19-430d-8065-19aeb291f3ad\" (UID: \"6c3e1570-bd19-430d-8065-19aeb291f3ad\") " Jul 06 22:49:31.907092 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:31.907037 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6c3e1570-bd19-430d-8065-19aeb291f3ad-workload-certs\") pod \"6c3e1570-bd19-430d-8065-19aeb291f3ad\" (UID: \"6c3e1570-bd19-430d-8065-19aeb291f3ad\") " Jul 06 22:49:31.907092 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:31.907054 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6c3e1570-bd19-430d-8065-19aeb291f3ad-istio-envoy\") pod \"6c3e1570-bd19-430d-8065-19aeb291f3ad\" (UID: \"6c3e1570-bd19-430d-8065-19aeb291f3ad\") " Jul 06 22:49:31.907309 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:31.907113 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6c3e1570-bd19-430d-8065-19aeb291f3ad-workload-socket\") pod \"6c3e1570-bd19-430d-8065-19aeb291f3ad\" (UID: \"6c3e1570-bd19-430d-8065-19aeb291f3ad\") " Jul 06 22:49:31.907309 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:31.907138 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6c3e1570-bd19-430d-8065-19aeb291f3ad-istiod-ca-cert\") pod \"6c3e1570-bd19-430d-8065-19aeb291f3ad\" (UID: \"6c3e1570-bd19-430d-8065-19aeb291f3ad\") " Jul 06 22:49:31.907309 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:31.907208 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6c3e1570-bd19-430d-8065-19aeb291f3ad-istio-data\") pod \"6c3e1570-bd19-430d-8065-19aeb291f3ad\" (UID: \"6c3e1570-bd19-430d-8065-19aeb291f3ad\") " Jul 06 22:49:31.907309 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:31.907241 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-66mpv\" (UniqueName: \"kubernetes.io/projected/6c3e1570-bd19-430d-8065-19aeb291f3ad-kube-api-access-66mpv\") pod \"6c3e1570-bd19-430d-8065-19aeb291f3ad\" (UID: \"6c3e1570-bd19-430d-8065-19aeb291f3ad\") " Jul 06 22:49:31.907309 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:31.907265 2581 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6c3e1570-bd19-430d-8065-19aeb291f3ad-istio-token\") pod \"6c3e1570-bd19-430d-8065-19aeb291f3ad\" (UID: \"6c3e1570-bd19-430d-8065-19aeb291f3ad\") " Jul 06 22:49:31.907309 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:31.907298 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6c3e1570-bd19-430d-8065-19aeb291f3ad-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6c3e1570-bd19-430d-8065-19aeb291f3ad" (UID: "6c3e1570-bd19-430d-8065-19aeb291f3ad"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:49:31.907542 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:31.907397 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6c3e1570-bd19-430d-8065-19aeb291f3ad-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6c3e1570-bd19-430d-8065-19aeb291f3ad" (UID: "6c3e1570-bd19-430d-8065-19aeb291f3ad"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:49:31.907583 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:31.907557 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6c3e1570-bd19-430d-8065-19aeb291f3ad-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6c3e1570-bd19-430d-8065-19aeb291f3ad" (UID: "6c3e1570-bd19-430d-8065-19aeb291f3ad"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 22:49:31.907645 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:31.907625 2581 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6c3e1570-bd19-430d-8065-19aeb291f3ad-credential-socket\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:49:31.907698 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:31.907650 2581 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6c3e1570-bd19-430d-8065-19aeb291f3ad-workload-socket\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:49:31.907991 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:31.907968 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6c3e1570-bd19-430d-8065-19aeb291f3ad-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6c3e1570-bd19-430d-8065-19aeb291f3ad" (UID: "6c3e1570-bd19-430d-8065-19aeb291f3ad"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:49:31.908196 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:31.908114 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6c3e1570-bd19-430d-8065-19aeb291f3ad-istio-data" (OuterVolumeSpecName: "istio-data") pod "6c3e1570-bd19-430d-8065-19aeb291f3ad" (UID: "6c3e1570-bd19-430d-8065-19aeb291f3ad"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:49:31.909392 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:31.909365 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6c3e1570-bd19-430d-8065-19aeb291f3ad-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6c3e1570-bd19-430d-8065-19aeb291f3ad" (UID: "6c3e1570-bd19-430d-8065-19aeb291f3ad"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 22:49:31.909872 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:31.909840 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6c3e1570-bd19-430d-8065-19aeb291f3ad-kube-api-access-66mpv" (OuterVolumeSpecName: "kube-api-access-66mpv") pod "6c3e1570-bd19-430d-8065-19aeb291f3ad" (UID: "6c3e1570-bd19-430d-8065-19aeb291f3ad"). InnerVolumeSpecName "kube-api-access-66mpv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 22:49:31.909964 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:31.909878 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6c3e1570-bd19-430d-8065-19aeb291f3ad-istio-token" (OuterVolumeSpecName: "istio-token") pod "6c3e1570-bd19-430d-8065-19aeb291f3ad" (UID: "6c3e1570-bd19-430d-8065-19aeb291f3ad"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 22:49:31.909964 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:31.909913 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6c3e1570-bd19-430d-8065-19aeb291f3ad-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6c3e1570-bd19-430d-8065-19aeb291f3ad" (UID: "6c3e1570-bd19-430d-8065-19aeb291f3ad"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 22:49:32.008838 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:32.008795 2581 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6c3e1570-bd19-430d-8065-19aeb291f3ad-workload-certs\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:49:32.008838 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:32.008828 2581 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6c3e1570-bd19-430d-8065-19aeb291f3ad-istio-envoy\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:49:32.008838 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:32.008837 2581 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6c3e1570-bd19-430d-8065-19aeb291f3ad-istiod-ca-cert\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:49:32.008838 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:32.008844 2581 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6c3e1570-bd19-430d-8065-19aeb291f3ad-istio-data\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:49:32.009158 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:32.008853 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-66mpv\" (UniqueName: \"kubernetes.io/projected/6c3e1570-bd19-430d-8065-19aeb291f3ad-kube-api-access-66mpv\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:49:32.009158 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:32.008863 2581 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6c3e1570-bd19-430d-8065-19aeb291f3ad-istio-token\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:49:32.009158 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:32.008871 2581 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6c3e1570-bd19-430d-8065-19aeb291f3ad-istio-podinfo\") on node \"ip-10-0-137-101.ec2.internal\" DevicePath \"\"" Jul 06 22:49:32.139717 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:32.139686 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29722950-79m5s_1a705704-330f-498f-b7a3-696be4ace5a6/cleanup/0.log" Jul 06 22:49:32.377323 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:32.377290 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5fb54f555-mbndp_aca1a96f-d595-4aec-a683-47ee9930a384/manager/0.log" Jul 06 22:49:32.462749 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:32.462710 2581 generic.go:356] "Generic (PLEG): container finished" podID="6c3e1570-bd19-430d-8065-19aeb291f3ad" containerID="2f39e993223d02c4756b532c4095e3b91cde395d485ebb08626d531f2512bff3" exitCode=0 Jul 06 22:49:32.462924 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:32.462765 2581 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-944f9f-openshift-default-847d84d4c9-6v6r9" Jul 06 22:49:32.462924 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:32.462800 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-944f9f-openshift-default-847d84d4c9-6v6r9" event={"ID":"6c3e1570-bd19-430d-8065-19aeb291f3ad","Type":"ContainerDied","Data":"2f39e993223d02c4756b532c4095e3b91cde395d485ebb08626d531f2512bff3"} Jul 06 22:49:32.462924 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:32.462858 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-944f9f-openshift-default-847d84d4c9-6v6r9" event={"ID":"6c3e1570-bd19-430d-8065-19aeb291f3ad","Type":"ContainerDied","Data":"0cccdc56e5591a5cccbf9a7306b4b4604525681472393f214c92e3099a9a7156"} Jul 06 22:49:32.462924 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:32.462883 2581 scope.go:117] "RemoveContainer" containerID="2f39e993223d02c4756b532c4095e3b91cde395d485ebb08626d531f2512bff3" Jul 06 22:49:32.472329 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:32.472311 2581 scope.go:117] "RemoveContainer" containerID="14baa758516dd9a905a60798f58eefe274404e1995f184a4032c9ab7f43f5a9c" Jul 06 22:49:32.480399 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:32.480376 2581 scope.go:117] "RemoveContainer" containerID="2f39e993223d02c4756b532c4095e3b91cde395d485ebb08626d531f2512bff3" Jul 06 22:49:32.480656 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:49:32.480635 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2f39e993223d02c4756b532c4095e3b91cde395d485ebb08626d531f2512bff3\": container with ID starting with 2f39e993223d02c4756b532c4095e3b91cde395d485ebb08626d531f2512bff3 not found: ID does not exist" containerID="2f39e993223d02c4756b532c4095e3b91cde395d485ebb08626d531f2512bff3" Jul 06 22:49:32.480723 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:32.480662 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2f39e993223d02c4756b532c4095e3b91cde395d485ebb08626d531f2512bff3"} err="failed to get container status \"2f39e993223d02c4756b532c4095e3b91cde395d485ebb08626d531f2512bff3\": rpc error: code = NotFound desc = could not find container \"2f39e993223d02c4756b532c4095e3b91cde395d485ebb08626d531f2512bff3\": container with ID starting with 2f39e993223d02c4756b532c4095e3b91cde395d485ebb08626d531f2512bff3 not found: ID does not exist" Jul 06 22:49:32.480723 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:32.480678 2581 scope.go:117] "RemoveContainer" containerID="14baa758516dd9a905a60798f58eefe274404e1995f184a4032c9ab7f43f5a9c" Jul 06 22:49:32.480879 ip-10-0-137-101 kubenswrapper[2581]: E0706 22:49:32.480864 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"14baa758516dd9a905a60798f58eefe274404e1995f184a4032c9ab7f43f5a9c\": container with ID starting with 14baa758516dd9a905a60798f58eefe274404e1995f184a4032c9ab7f43f5a9c not found: ID does not exist" containerID="14baa758516dd9a905a60798f58eefe274404e1995f184a4032c9ab7f43f5a9c" Jul 06 22:49:32.480915 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:32.480883 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"14baa758516dd9a905a60798f58eefe274404e1995f184a4032c9ab7f43f5a9c"} err="failed to get container status \"14baa758516dd9a905a60798f58eefe274404e1995f184a4032c9ab7f43f5a9c\": rpc error: code = NotFound desc = could not find container \"14baa758516dd9a905a60798f58eefe274404e1995f184a4032c9ab7f43f5a9c\": container with ID starting with 14baa758516dd9a905a60798f58eefe274404e1995f184a4032c9ab7f43f5a9c not found: ID does not exist" Jul 06 22:49:32.485939 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:32.485917 2581 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-944f9f-openshift-default-847d84d4c9-6v6r9"] Jul 06 22:49:32.489354 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:32.489330 2581 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-944f9f-openshift-default-847d84d4c9-6v6r9"] Jul 06 22:49:32.493940 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:32.493923 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-rc8jv_3f238cfe-21cc-46f6-94f7-60d2327e6af1/server/0.log" Jul 06 22:49:32.608590 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:32.608554 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-2xpwc_98c7acc6-8ae3-470e-b51b-8be2db428a7d/manager/0.log" Jul 06 22:49:32.960619 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:32.960587 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5c695b4674-zcn8n_48cb8eea-ec98-410c-9b13-84352620f7cd/copy-manifests/0.log" Jul 06 22:49:32.966561 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:32.966537 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5c695b4674-zcn8n_48cb8eea-ec98-410c-9b13-84352620f7cd/manager/0.log" Jul 06 22:49:32.976415 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:32.976388 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6c3e1570-bd19-430d-8065-19aeb291f3ad" path="/var/lib/kubelet/pods/6c3e1570-bd19-430d-8065-19aeb291f3ad/volumes" Jul 06 22:49:33.082475 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:33.082449 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-mt2gt_07104706-fbc7-4f5a-9b3f-e15ba908f46f/postgres/0.log" Jul 06 22:49:34.437239 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:34.437211 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-4m2sf_cfda177d-012f-4dff-ae40-553adf9e1cdb/manager/0.log" Jul 06 22:49:34.548682 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:34.548654 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-r8gcf_a79d66ac-7935-4124-b79c-355e0cb8b89d/manager/0.log" Jul 06 22:49:34.770461 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:34.770386 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-lswds_75cd310a-3b61-4cd2-8156-7a0b787cd2cf/registry-server/0.log" Jul 06 22:49:34.889944 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:34.889914 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-f468p_b25f696a-3383-4add-a8ff-07711818c31d/manager/0.log" Jul 06 22:49:35.476566 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:35.476537 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4vcm7z_e046a164-38cc-40ae-a412-18ad8af5b2e3/istio-proxy/0.log" Jul 06 22:49:35.700484 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:35.700458 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-658774566b-59kk5_70ec6221-d9c4-449b-b469-8725e711cae5/kube-auth-proxy/0.log" Jul 06 22:49:35.934320 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:35.934287 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-mbpp9_0fd612d1-61ff-411a-a137-dded5a45a55e/istio-proxy/1.log" Jul 06 22:49:36.046287 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:36.046258 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-9548w_85da56b6-db6b-4aa3-9478-0954a40c4105/payload-pre-processing/0.log" Jul 06 22:49:36.177188 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:36.177158 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-5mvrk_68adbb52-4e6d-4fc0-bda7-271551f24007/payload-processing/0.log" Jul 06 22:49:36.293290 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:36.293207 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-78989f99df-sxq46_6fdb1134-ed1c-4505-a741-6e99dba1e2c8/router/0.log" Jul 06 22:49:36.639172 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:36.639093 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-5ghnm_10fce2a3-efad-4bc4-8fc6-8fec4a808c92/main/0.log" Jul 06 22:49:36.758363 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:36.758331 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-7rzhc_b9536688-2588-4410-9d32-ca359b04815e/main/0.log" Jul 06 22:49:37.105647 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:37.105615 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-7pmq7_763061e4-dbed-450e-9dfa-65f51a94e41f/main/0.log" Jul 06 22:49:40.918038 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:40.918006 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-fc7g8/must-gather-kzzlk"] Jul 06 22:49:40.918593 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:40.918573 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9" containerName="istio-proxy" Jul 06 22:49:40.918677 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:40.918597 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9" containerName="istio-proxy" Jul 06 22:49:40.918677 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:40.918637 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6c3e1570-bd19-430d-8065-19aeb291f3ad" containerName="istio-proxy" Jul 06 22:49:40.918677 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:40.918647 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="6c3e1570-bd19-430d-8065-19aeb291f3ad" containerName="istio-proxy" Jul 06 22:49:40.918677 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:40.918662 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6c3e1570-bd19-430d-8065-19aeb291f3ad" containerName="istio-proxy" Jul 06 22:49:40.918677 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:40.918670 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="6c3e1570-bd19-430d-8065-19aeb291f3ad" containerName="istio-proxy" Jul 06 22:49:40.918995 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:40.918688 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac73802a-b5e4-47bb-9f6e-d938e8705f13" containerName="istio-proxy" Jul 06 22:49:40.918995 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:40.918696 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac73802a-b5e4-47bb-9f6e-d938e8705f13" containerName="istio-proxy" Jul 06 22:49:40.918995 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:40.918783 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="6c3e1570-bd19-430d-8065-19aeb291f3ad" containerName="istio-proxy" Jul 06 22:49:40.918995 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:40.918798 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9" containerName="istio-proxy" Jul 06 22:49:40.918995 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:40.918810 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9" containerName="istio-proxy" Jul 06 22:49:40.918995 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:40.918820 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="ac73802a-b5e4-47bb-9f6e-d938e8705f13" containerName="istio-proxy" Jul 06 22:49:40.918995 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:40.918936 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9" containerName="istio-proxy" Jul 06 22:49:40.918995 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:40.918948 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="f5b203b5-de40-4fb0-a2a7-d0cfe5f52aa9" containerName="istio-proxy" Jul 06 22:49:40.919414 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:40.919042 2581 memory_manager.go:368] "RemoveStaleState removing state" podUID="6c3e1570-bd19-430d-8065-19aeb291f3ad" containerName="istio-proxy" Jul 06 22:49:40.923691 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:40.923667 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-fc7g8/must-gather-kzzlk" Jul 06 22:49:40.926252 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:40.926229 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-fc7g8\"/\"openshift-service-ca.crt\"" Jul 06 22:49:40.927159 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:40.927138 2581 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-fc7g8\"/\"default-dockercfg-5gjkx\"" Jul 06 22:49:40.927262 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:40.927143 2581 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-fc7g8\"/\"kube-root-ca.crt\"" Jul 06 22:49:40.931332 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:40.931309 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-fc7g8/must-gather-kzzlk"] Jul 06 22:49:40.935564 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:40.935540 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:49:40.991019 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:40.990986 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8hrkd\" (UniqueName: \"kubernetes.io/projected/933f6e15-aa27-47cd-8050-3f1b0dbdb28d-kube-api-access-8hrkd\") pod \"must-gather-kzzlk\" (UID: \"933f6e15-aa27-47cd-8050-3f1b0dbdb28d\") " pod="openshift-must-gather-fc7g8/must-gather-kzzlk" Jul 06 22:49:40.991223 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:40.991132 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/933f6e15-aa27-47cd-8050-3f1b0dbdb28d-must-gather-output\") pod \"must-gather-kzzlk\" (UID: \"933f6e15-aa27-47cd-8050-3f1b0dbdb28d\") " pod="openshift-must-gather-fc7g8/must-gather-kzzlk" Jul 06 22:49:41.047833 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:41.047799 2581 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" limit=1024 Jul 06 22:49:41.092289 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:41.092254 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/933f6e15-aa27-47cd-8050-3f1b0dbdb28d-must-gather-output\") pod \"must-gather-kzzlk\" (UID: \"933f6e15-aa27-47cd-8050-3f1b0dbdb28d\") " pod="openshift-must-gather-fc7g8/must-gather-kzzlk" Jul 06 22:49:41.092440 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:41.092326 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8hrkd\" (UniqueName: \"kubernetes.io/projected/933f6e15-aa27-47cd-8050-3f1b0dbdb28d-kube-api-access-8hrkd\") pod \"must-gather-kzzlk\" (UID: \"933f6e15-aa27-47cd-8050-3f1b0dbdb28d\") " pod="openshift-must-gather-fc7g8/must-gather-kzzlk" Jul 06 22:49:41.092560 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:41.092541 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/933f6e15-aa27-47cd-8050-3f1b0dbdb28d-must-gather-output\") pod \"must-gather-kzzlk\" (UID: \"933f6e15-aa27-47cd-8050-3f1b0dbdb28d\") " pod="openshift-must-gather-fc7g8/must-gather-kzzlk" Jul 06 22:49:41.102379 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:41.102348 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8hrkd\" (UniqueName: \"kubernetes.io/projected/933f6e15-aa27-47cd-8050-3f1b0dbdb28d-kube-api-access-8hrkd\") pod \"must-gather-kzzlk\" (UID: \"933f6e15-aa27-47cd-8050-3f1b0dbdb28d\") " pod="openshift-must-gather-fc7g8/must-gather-kzzlk" Jul 06 22:49:41.230495 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:41.230448 2581 scope.go:117] "RemoveContainer" containerID="11fc7b341eeb4d430e3bb27e9ac0d4c77d6b647c6d8546bfb21c46ee32a38074" Jul 06 22:49:41.234462 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:41.234433 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-fc7g8/must-gather-kzzlk" Jul 06 22:49:41.372721 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:41.372692 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-fc7g8/must-gather-kzzlk"] Jul 06 22:49:41.374823 ip-10-0-137-101 kubenswrapper[2581]: W0706 22:49:41.374798 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod933f6e15_aa27_47cd_8050_3f1b0dbdb28d.slice/crio-9a96d05abaa354c700a1b06291dccd7ba5fe7c45a452f533a3c76dfdcf3fea8f WatchSource:0}: Error finding container 9a96d05abaa354c700a1b06291dccd7ba5fe7c45a452f533a3c76dfdcf3fea8f: Status 404 returned error can't find the container with id 9a96d05abaa354c700a1b06291dccd7ba5fe7c45a452f533a3c76dfdcf3fea8f Jul 06 22:49:41.377325 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:41.377306 2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 22:49:41.509313 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:41.509224 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-fc7g8/must-gather-kzzlk" event={"ID":"933f6e15-aa27-47cd-8050-3f1b0dbdb28d","Type":"ContainerStarted","Data":"9a96d05abaa354c700a1b06291dccd7ba5fe7c45a452f533a3c76dfdcf3fea8f"} Jul 06 22:49:42.519932 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:42.519888 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-fc7g8/must-gather-kzzlk" event={"ID":"933f6e15-aa27-47cd-8050-3f1b0dbdb28d","Type":"ContainerStarted","Data":"e92a98bdf1795a78eb3298d29ca1ad75974b4a760016be7b5571f3a92940b524"} Jul 06 22:49:42.520448 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:42.519945 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-fc7g8/must-gather-kzzlk" event={"ID":"933f6e15-aa27-47cd-8050-3f1b0dbdb28d","Type":"ContainerStarted","Data":"9768b5ccc562a8eca17eeb2452322486a17143c31f5288d608ff6ca57d3b0ac1"} Jul 06 22:49:42.536190 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:42.536130 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-fc7g8/must-gather-kzzlk" podStartSLOduration=1.7910340420000002 podStartE2EDuration="2.536108772s" totalImagesPullingTime="745.07473ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 22:49:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:49:42.534319273 +0000 UTC m=+2462.093114601" watchObservedRunningTime="2026-07-06 22:49:42.536108772 +0000 UTC m=+2462.094904097" Jul 06 22:49:43.739045 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:43.739011 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-klc48_b42224bc-b677-43bf-94fb-fc75223b963a/global-pull-secret-syncer/0.log" Jul 06 22:49:43.830575 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:43.830546 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-66tt7_2041467a-e1c5-4702-96c6-8be98208f3eb/konnectivity-agent/0.log" Jul 06 22:49:43.945344 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:43.945302 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-137-101.ec2.internal_2573bcaf102b258c7dbacbd9609f526e/haproxy/0.log" Jul 06 22:49:49.725438 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:49.725399 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-4m2sf_cfda177d-012f-4dff-ae40-553adf9e1cdb/manager/0.log" Jul 06 22:49:49.751494 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:49.751459 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-r8gcf_a79d66ac-7935-4124-b79c-355e0cb8b89d/manager/0.log" Jul 06 22:49:49.820952 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:49.820922 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-lswds_75cd310a-3b61-4cd2-8156-7a0b787cd2cf/registry-server/0.log" Jul 06 22:49:49.961082 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:49.961035 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-f468p_b25f696a-3383-4add-a8ff-07711818c31d/manager/0.log" Jul 06 22:49:51.876505 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:51.876468 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-24gv2_179af188-f2c6-46da-9d1d-b15ce8fcb039/cluster-monitoring-operator/0.log" Jul 06 22:49:52.102338 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:52.102297 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-b6dcb_4fe91138-7f43-4d7a-b7a7-df08b1804376/node-exporter/0.log" Jul 06 22:49:52.122686 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:52.122645 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-b6dcb_4fe91138-7f43-4d7a-b7a7-df08b1804376/kube-rbac-proxy/0.log" Jul 06 22:49:52.146716 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:52.146632 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-b6dcb_4fe91138-7f43-4d7a-b7a7-df08b1804376/init-textfile/0.log" Jul 06 22:49:53.623178 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:53.623138 2581 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-fc7g8/perf-node-gather-daemonset-wp4vf"] Jul 06 22:49:53.630044 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:53.630014 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-fc7g8/perf-node-gather-daemonset-wp4vf" Jul 06 22:49:53.633026 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:53.632998 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-fc7g8/perf-node-gather-daemonset-wp4vf"] Jul 06 22:49:53.732645 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:53.732603 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ee6ded83-acc9-4c2d-aae2-7ab6616d03f3-sys\") pod \"perf-node-gather-daemonset-wp4vf\" (UID: \"ee6ded83-acc9-4c2d-aae2-7ab6616d03f3\") " pod="openshift-must-gather-fc7g8/perf-node-gather-daemonset-wp4vf" Jul 06 22:49:53.732833 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:53.732690 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ee6ded83-acc9-4c2d-aae2-7ab6616d03f3-proc\") pod \"perf-node-gather-daemonset-wp4vf\" (UID: \"ee6ded83-acc9-4c2d-aae2-7ab6616d03f3\") " pod="openshift-must-gather-fc7g8/perf-node-gather-daemonset-wp4vf" Jul 06 22:49:53.732833 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:53.732716 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ee6ded83-acc9-4c2d-aae2-7ab6616d03f3-podres\") pod \"perf-node-gather-daemonset-wp4vf\" (UID: \"ee6ded83-acc9-4c2d-aae2-7ab6616d03f3\") " pod="openshift-must-gather-fc7g8/perf-node-gather-daemonset-wp4vf" Jul 06 22:49:53.732938 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:53.732841 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ee6ded83-acc9-4c2d-aae2-7ab6616d03f3-lib-modules\") pod \"perf-node-gather-daemonset-wp4vf\" (UID: \"ee6ded83-acc9-4c2d-aae2-7ab6616d03f3\") " pod="openshift-must-gather-fc7g8/perf-node-gather-daemonset-wp4vf" Jul 06 22:49:53.732938 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:53.732870 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sj2jz\" (UniqueName: \"kubernetes.io/projected/ee6ded83-acc9-4c2d-aae2-7ab6616d03f3-kube-api-access-sj2jz\") pod \"perf-node-gather-daemonset-wp4vf\" (UID: \"ee6ded83-acc9-4c2d-aae2-7ab6616d03f3\") " pod="openshift-must-gather-fc7g8/perf-node-gather-daemonset-wp4vf" Jul 06 22:49:53.833719 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:53.833672 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ee6ded83-acc9-4c2d-aae2-7ab6616d03f3-lib-modules\") pod \"perf-node-gather-daemonset-wp4vf\" (UID: \"ee6ded83-acc9-4c2d-aae2-7ab6616d03f3\") " pod="openshift-must-gather-fc7g8/perf-node-gather-daemonset-wp4vf" Jul 06 22:49:53.833916 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:53.833733 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sj2jz\" (UniqueName: \"kubernetes.io/projected/ee6ded83-acc9-4c2d-aae2-7ab6616d03f3-kube-api-access-sj2jz\") pod \"perf-node-gather-daemonset-wp4vf\" (UID: \"ee6ded83-acc9-4c2d-aae2-7ab6616d03f3\") " pod="openshift-must-gather-fc7g8/perf-node-gather-daemonset-wp4vf" Jul 06 22:49:53.833916 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:53.833781 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ee6ded83-acc9-4c2d-aae2-7ab6616d03f3-sys\") pod \"perf-node-gather-daemonset-wp4vf\" (UID: \"ee6ded83-acc9-4c2d-aae2-7ab6616d03f3\") " pod="openshift-must-gather-fc7g8/perf-node-gather-daemonset-wp4vf" Jul 06 22:49:53.833916 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:53.833843 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ee6ded83-acc9-4c2d-aae2-7ab6616d03f3-proc\") pod \"perf-node-gather-daemonset-wp4vf\" (UID: \"ee6ded83-acc9-4c2d-aae2-7ab6616d03f3\") " pod="openshift-must-gather-fc7g8/perf-node-gather-daemonset-wp4vf" Jul 06 22:49:53.833916 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:53.833869 2581 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ee6ded83-acc9-4c2d-aae2-7ab6616d03f3-podres\") pod \"perf-node-gather-daemonset-wp4vf\" (UID: \"ee6ded83-acc9-4c2d-aae2-7ab6616d03f3\") " pod="openshift-must-gather-fc7g8/perf-node-gather-daemonset-wp4vf" Jul 06 22:49:53.834172 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:53.834117 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ee6ded83-acc9-4c2d-aae2-7ab6616d03f3-podres\") pod \"perf-node-gather-daemonset-wp4vf\" (UID: \"ee6ded83-acc9-4c2d-aae2-7ab6616d03f3\") " pod="openshift-must-gather-fc7g8/perf-node-gather-daemonset-wp4vf" Jul 06 22:49:53.834247 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:53.834212 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ee6ded83-acc9-4c2d-aae2-7ab6616d03f3-proc\") pod \"perf-node-gather-daemonset-wp4vf\" (UID: \"ee6ded83-acc9-4c2d-aae2-7ab6616d03f3\") " pod="openshift-must-gather-fc7g8/perf-node-gather-daemonset-wp4vf" Jul 06 22:49:53.834326 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:53.834302 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ee6ded83-acc9-4c2d-aae2-7ab6616d03f3-sys\") pod \"perf-node-gather-daemonset-wp4vf\" (UID: \"ee6ded83-acc9-4c2d-aae2-7ab6616d03f3\") " pod="openshift-must-gather-fc7g8/perf-node-gather-daemonset-wp4vf" Jul 06 22:49:53.834405 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:53.834383 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ee6ded83-acc9-4c2d-aae2-7ab6616d03f3-lib-modules\") pod \"perf-node-gather-daemonset-wp4vf\" (UID: \"ee6ded83-acc9-4c2d-aae2-7ab6616d03f3\") " pod="openshift-must-gather-fc7g8/perf-node-gather-daemonset-wp4vf" Jul 06 22:49:53.842666 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:53.842634 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sj2jz\" (UniqueName: \"kubernetes.io/projected/ee6ded83-acc9-4c2d-aae2-7ab6616d03f3-kube-api-access-sj2jz\") pod \"perf-node-gather-daemonset-wp4vf\" (UID: \"ee6ded83-acc9-4c2d-aae2-7ab6616d03f3\") " pod="openshift-must-gather-fc7g8/perf-node-gather-daemonset-wp4vf" Jul 06 22:49:53.946165 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:53.946125 2581 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-fc7g8/perf-node-gather-daemonset-wp4vf" Jul 06 22:49:54.149730 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:54.149670 2581 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-fc7g8/perf-node-gather-daemonset-wp4vf"] Jul 06 22:49:54.526843 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:54.526742 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-rcn5q_86b07166-2f34-443b-9c61-86b7d950ad7b/console-operator/1.log" Jul 06 22:49:54.532759 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:54.532734 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-68b757fb67-rcn5q_86b07166-2f34-443b-9c61-86b7d950ad7b/console-operator/2.log" Jul 06 22:49:54.615957 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:54.615919 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-fc7g8/perf-node-gather-daemonset-wp4vf" event={"ID":"ee6ded83-acc9-4c2d-aae2-7ab6616d03f3","Type":"ContainerStarted","Data":"5368cf137dabf752e1edcde19b8934f91e955a3a058ef08e4dcf744a5f0d8de6"} Jul 06 22:49:54.616280 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:54.616255 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-fc7g8/perf-node-gather-daemonset-wp4vf" Jul 06 22:49:54.616428 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:54.616294 2581 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-fc7g8/perf-node-gather-daemonset-wp4vf" event={"ID":"ee6ded83-acc9-4c2d-aae2-7ab6616d03f3","Type":"ContainerStarted","Data":"adc3a5b2a91a4b37cf547bfd8fb90dce01927cc4eb13f45524bf6d803a0140d9"} Jul 06 22:49:54.634548 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:54.634487 2581 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-fc7g8/perf-node-gather-daemonset-wp4vf" podStartSLOduration=1.6344651190000001 podStartE2EDuration="1.634465119s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 22:49:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 22:49:54.631094265 +0000 UTC m=+2474.189889603" watchObservedRunningTime="2026-07-06 22:49:54.634465119 +0000 UTC m=+2474.193260442" Jul 06 22:49:55.624828 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:55.624797 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-5d456fdbcd-dftxz_c39dd055-b80e-45d4-9882-ae689d849208/volume-data-source-validator/0.log" Jul 06 22:49:56.692799 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:56.692772 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-nfklb_da5ae4fc-9377-4d40-89cf-efc75bac039e/dns/0.log" Jul 06 22:49:56.717204 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:56.717161 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-nfklb_da5ae4fc-9377-4d40-89cf-efc75bac039e/kube-rbac-proxy/0.log" Jul 06 22:49:56.794910 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:56.794880 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-4b7pr_314ffdbc-f9e6-45c0-b37a-d216700a1de5/dns-node-resolver/0.log" Jul 06 22:49:57.426864 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:57.426823 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-cmtz2_9a1fd480-e89a-4522-b82a-87b8c72124d5/node-ca/0.log" Jul 06 22:49:58.399136 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:58.399109 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4vcm7z_e046a164-38cc-40ae-a412-18ad8af5b2e3/istio-proxy/0.log" Jul 06 22:49:58.535252 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:58.535221 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-658774566b-59kk5_70ec6221-d9c4-449b-b469-8725e711cae5/kube-auth-proxy/0.log" Jul 06 22:49:58.587872 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:58.587845 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-mbpp9_0fd612d1-61ff-411a-a137-dded5a45a55e/istio-proxy/0.log" Jul 06 22:49:58.626059 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:58.626025 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-mbpp9_0fd612d1-61ff-411a-a137-dded5a45a55e/istio-proxy/1.log" Jul 06 22:49:58.660782 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:58.660700 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-9548w_85da56b6-db6b-4aa3-9478-0954a40c4105/payload-pre-processing/0.log" Jul 06 22:49:58.716099 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:58.716053 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-5mvrk_68adbb52-4e6d-4fc0-bda7-271551f24007/payload-processing/0.log" Jul 06 22:49:58.748527 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:58.748499 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-78989f99df-sxq46_6fdb1134-ed1c-4505-a741-6e99dba1e2c8/router/0.log" Jul 06 22:49:59.488605 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:59.488569 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-vf82r_638815a3-fb4e-41ac-9e74-7c3426d561f4/serve-healthcheck-canary/0.log" Jul 06 22:49:59.976129 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:59.976091 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-2pphn_fe3d4288-3a9d-413c-be44-ac70bd65b45f/insights-operator/0.log" Jul 06 22:49:59.976766 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:49:59.976744 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5c847dcfc4-2pphn_fe3d4288-3a9d-413c-be44-ac70bd65b45f/insights-operator/1.log" Jul 06 22:50:00.000031 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:50:00.000002 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-68gjp_bd42289a-105e-46c7-ad15-a6341ce43f15/kube-rbac-proxy/0.log" Jul 06 22:50:00.021375 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:50:00.021337 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-68gjp_bd42289a-105e-46c7-ad15-a6341ce43f15/exporter/0.log" Jul 06 22:50:00.044669 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:50:00.044635 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-68gjp_bd42289a-105e-46c7-ad15-a6341ce43f15/extractor/0.log" Jul 06 22:50:00.637856 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:50:00.637825 2581 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-fc7g8/perf-node-gather-daemonset-wp4vf" Jul 06 22:50:02.578713 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:50:02.578575 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-wn2sx_ab6dcafe-a3c5-41d3-92ae-55aafb915634/manager/0.log" Jul 06 22:50:02.765347 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:50:02.765316 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-4swjt_5ad9d1ac-6623-4ff6-8c6d-6003ee007795/manager/0.log" Jul 06 22:50:02.841785 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:50:02.841690 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29722950-79m5s_1a705704-330f-498f-b7a3-696be4ace5a6/cleanup/0.log" Jul 06 22:50:03.006708 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:50:03.006669 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5fb54f555-mbndp_aca1a96f-d595-4aec-a683-47ee9930a384/manager/0.log" Jul 06 22:50:03.028938 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:50:03.028902 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-rc8jv_3f238cfe-21cc-46f6-94f7-60d2327e6af1/server/0.log" Jul 06 22:50:03.113652 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:50:03.113562 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-2xpwc_98c7acc6-8ae3-470e-b51b-8be2db428a7d/manager/0.log" Jul 06 22:50:03.295426 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:50:03.295386 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5c695b4674-zcn8n_48cb8eea-ec98-410c-9b13-84352620f7cd/manager/0.log" Jul 06 22:50:03.318169 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:50:03.318136 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5c695b4674-zcn8n_48cb8eea-ec98-410c-9b13-84352620f7cd/copy-manifests/0.log" Jul 06 22:50:03.343334 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:50:03.343296 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-mt2gt_07104706-fbc7-4f5a-9b3f-e15ba908f46f/postgres/0.log" Jul 06 22:50:04.655402 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:50:04.655355 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4vcm7z_e046a164-38cc-40ae-a412-18ad8af5b2e3/istio-proxy/0.log" Jul 06 22:50:04.762179 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:50:04.762152 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-658774566b-59kk5_70ec6221-d9c4-449b-b469-8725e711cae5/kube-auth-proxy/0.log" Jul 06 22:50:04.799368 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:50:04.799340 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-mbpp9_0fd612d1-61ff-411a-a137-dded5a45a55e/istio-proxy/0.log" Jul 06 22:50:04.835930 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:50:04.835886 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-mbpp9_0fd612d1-61ff-411a-a137-dded5a45a55e/istio-proxy/1.log" Jul 06 22:50:04.863617 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:50:04.863570 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-9548w_85da56b6-db6b-4aa3-9478-0954a40c4105/payload-pre-processing/0.log" Jul 06 22:50:04.915008 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:50:04.914905 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-5mvrk_68adbb52-4e6d-4fc0-bda7-271551f24007/payload-processing/0.log" Jul 06 22:50:04.931416 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:50:04.931388 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-78989f99df-sxq46_6fdb1134-ed1c-4505-a741-6e99dba1e2c8/router/0.log" Jul 06 22:50:06.166156 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:50:06.166127 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4vcm7z_e046a164-38cc-40ae-a412-18ad8af5b2e3/istio-proxy/0.log" Jul 06 22:50:06.269098 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:50:06.268987 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-658774566b-59kk5_70ec6221-d9c4-449b-b469-8725e711cae5/kube-auth-proxy/0.log" Jul 06 22:50:06.313450 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:50:06.313412 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-mbpp9_0fd612d1-61ff-411a-a137-dded5a45a55e/istio-proxy/0.log" Jul 06 22:50:06.349545 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:50:06.349514 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-mbpp9_0fd612d1-61ff-411a-a137-dded5a45a55e/istio-proxy/1.log" Jul 06 22:50:06.375978 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:50:06.375945 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-9548w_85da56b6-db6b-4aa3-9478-0954a40c4105/payload-pre-processing/0.log" Jul 06 22:50:06.423201 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:50:06.423170 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-5mvrk_68adbb52-4e6d-4fc0-bda7-271551f24007/payload-processing/0.log" Jul 06 22:50:06.448646 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:50:06.448617 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-78989f99df-sxq46_6fdb1134-ed1c-4505-a741-6e99dba1e2c8/router/0.log" Jul 06 22:50:10.176415 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:50:10.176377 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-846bf668f7-hfs2q_d3642934-8f79-4aed-bdfe-5a6777a655aa/kube-storage-version-migrator-operator/1.log" Jul 06 22:50:10.177288 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:50:10.177269 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-846bf668f7-hfs2q_d3642934-8f79-4aed-bdfe-5a6777a655aa/kube-storage-version-migrator-operator/0.log" Jul 06 22:50:11.142378 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:50:11.142307 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5nrhc_f1de085f-40f2-4f95-a5bb-78aba60de69e/kube-multus-additional-cni-plugins/0.log" Jul 06 22:50:11.167977 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:50:11.167947 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5nrhc_f1de085f-40f2-4f95-a5bb-78aba60de69e/egress-router-binary-copy/0.log" Jul 06 22:50:11.190862 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:50:11.190827 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5nrhc_f1de085f-40f2-4f95-a5bb-78aba60de69e/cni-plugins/0.log" Jul 06 22:50:11.214049 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:50:11.214008 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5nrhc_f1de085f-40f2-4f95-a5bb-78aba60de69e/bond-cni-plugin/0.log" Jul 06 22:50:11.237517 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:50:11.237483 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5nrhc_f1de085f-40f2-4f95-a5bb-78aba60de69e/routeoverride-cni/0.log" Jul 06 22:50:11.259668 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:50:11.259640 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5nrhc_f1de085f-40f2-4f95-a5bb-78aba60de69e/whereabouts-cni-bincopy/0.log" Jul 06 22:50:11.280528 ip-10-0-137-101 kubenswrapper[2581]: I0706 22:50:11.280500 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5nrhc_f1de085f-40f2-4f95-a5bb-78aba60de69e/whereabouts-cni/0.log"