Jul 06 19:09:33.149863 ip-10-0-138-35 systemd[1]: Starting Kubernetes Kubelet... Jul 06 19:09:33.543884 ip-10-0-138-35 kubenswrapper[2566]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 06 19:09:33.543884 ip-10-0-138-35 kubenswrapper[2566]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 06 19:09:33.543884 ip-10-0-138-35 kubenswrapper[2566]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 06 19:09:33.543884 ip-10-0-138-35 kubenswrapper[2566]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 06 19:09:33.543884 ip-10-0-138-35 kubenswrapper[2566]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 06 19:09:33.544755 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.544662 2566 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 06 19:09:33.549956 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.549929 2566 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 19:09:33.549956 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.549952 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 19:09:33.549956 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.549956 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 19:09:33.549956 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.549959 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 19:09:33.549956 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.549962 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 19:09:33.549956 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.549966 2566 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 19:09:33.549956 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.549969 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 19:09:33.550232 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.549972 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 19:09:33.550232 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.549975 2566 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 19:09:33.550232 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.549978 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 19:09:33.550232 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.549981 2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 19:09:33.550232 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.549983 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 19:09:33.550232 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.549986 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 19:09:33.550232 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.549988 2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 19:09:33.550232 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.549991 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 19:09:33.550232 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.549993 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 19:09:33.550232 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.549996 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 19:09:33.550232 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.549998 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 19:09:33.550232 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.550001 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 19:09:33.550232 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.550004 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 19:09:33.550232 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.550007 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 19:09:33.550232 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.550009 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 19:09:33.550232 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.550012 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 19:09:33.550232 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.550014 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 19:09:33.550232 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.550017 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 19:09:33.550232 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.550019 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 19:09:33.550232 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.550022 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 19:09:33.550730 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.550024 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 19:09:33.550730 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.550027 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 19:09:33.550730 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.550030 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 19:09:33.550730 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.550032 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 19:09:33.550730 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.550035 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 19:09:33.550730 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.550044 2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 19:09:33.550730 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.550047 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 19:09:33.550730 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.550049 2566 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 19:09:33.550730 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.550051 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 19:09:33.550730 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.550054 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 19:09:33.550730 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.550057 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 19:09:33.550730 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.550059 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 19:09:33.550730 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.550062 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 19:09:33.550730 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.550065 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 19:09:33.550730 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.550067 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 19:09:33.550730 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.550070 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 19:09:33.550730 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.550072 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 19:09:33.550730 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.550075 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 19:09:33.550730 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.550078 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 19:09:33.550730 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.550080 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 19:09:33.551247 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.550082 2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 19:09:33.551247 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.550085 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 19:09:33.551247 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.550087 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 19:09:33.551247 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.550092 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 19:09:33.551247 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.550095 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 19:09:33.551247 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.550097 2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 19:09:33.551247 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.550100 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 19:09:33.551247 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.550102 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 19:09:33.551247 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.550106 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 19:09:33.551247 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.550110 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 19:09:33.551247 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.550113 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 19:09:33.551247 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.550117 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 19:09:33.551247 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.550120 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 19:09:33.551247 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.550125 2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 19:09:33.551247 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.550128 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 19:09:33.551247 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.550133 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 19:09:33.551247 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.550135 2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 19:09:33.551247 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.550138 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 19:09:33.551247 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.550141 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 19:09:33.551721 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.550144 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 19:09:33.551721 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.550148 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 19:09:33.551721 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.550151 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 19:09:33.551721 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.550153 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 19:09:33.551721 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.550156 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 19:09:33.551721 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.550160 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 19:09:33.551721 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.550163 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 19:09:33.551721 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.550165 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 19:09:33.551721 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.550168 2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 19:09:33.551721 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.550170 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 19:09:33.551721 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.550173 2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 19:09:33.551721 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.550176 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 19:09:33.551721 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.550178 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 19:09:33.551721 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.550181 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 19:09:33.551721 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.550184 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 19:09:33.551721 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.550187 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 19:09:33.551721 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.550189 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 19:09:33.551721 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.550193 2566 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 19:09:33.551721 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.550196 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 19:09:33.552208 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.550198 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 19:09:33.552208 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.550201 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 19:09:33.552208 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.550203 2566 feature_gate.go:328] unrecognized feature gate: Example Jul 06 19:09:33.552208 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.550206 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 19:09:33.552208 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.550208 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 19:09:33.552208 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.550211 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 19:09:33.552208 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.550213 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 19:09:33.552208 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.550216 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 19:09:33.552208 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.550218 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 19:09:33.552208 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.550221 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 19:09:33.552208 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.550223 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 19:09:33.552208 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.550226 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 19:09:33.552208 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.550228 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 19:09:33.552208 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.550230 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 19:09:33.552208 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.550946 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 19:09:33.552208 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.550953 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 19:09:33.552208 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.550957 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 19:09:33.552208 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.550959 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 19:09:33.552208 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.550963 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 19:09:33.552208 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.550966 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 19:09:33.552208 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.550968 2566 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 19:09:33.552730 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.550971 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 19:09:33.552730 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.550974 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 19:09:33.552730 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.550976 2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 19:09:33.552730 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.550979 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 19:09:33.552730 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.550981 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 19:09:33.552730 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.550984 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 19:09:33.552730 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.550986 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 19:09:33.552730 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.550989 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 19:09:33.552730 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.550991 2566 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 19:09:33.552730 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.550994 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 19:09:33.552730 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.550997 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 19:09:33.552730 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551000 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 19:09:33.552730 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551003 2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 19:09:33.552730 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551005 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 19:09:33.552730 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551007 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 19:09:33.552730 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551010 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 19:09:33.552730 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551012 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 19:09:33.552730 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551015 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 19:09:33.552730 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551017 2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 19:09:33.552730 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551020 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 19:09:33.553236 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551022 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 19:09:33.553236 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551025 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 19:09:33.553236 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551028 2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 19:09:33.553236 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551031 2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 19:09:33.553236 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551033 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 19:09:33.553236 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551036 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 19:09:33.553236 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551039 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 19:09:33.553236 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551041 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 19:09:33.553236 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551044 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 19:09:33.553236 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551047 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 19:09:33.553236 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551049 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 19:09:33.553236 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551052 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 19:09:33.553236 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551054 2566 feature_gate.go:328] unrecognized feature gate: Example Jul 06 19:09:33.553236 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551057 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 19:09:33.553236 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551059 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 19:09:33.553236 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551062 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 19:09:33.553236 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551064 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 19:09:33.553236 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551066 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 19:09:33.553236 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551069 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 19:09:33.553236 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551071 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 19:09:33.553749 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551076 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 19:09:33.553749 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551080 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 19:09:33.553749 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551083 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 19:09:33.553749 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551087 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 19:09:33.553749 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551090 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 19:09:33.553749 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551092 2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 19:09:33.553749 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551095 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 19:09:33.553749 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551098 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 19:09:33.553749 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551100 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 19:09:33.553749 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551103 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 19:09:33.553749 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551105 2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 19:09:33.553749 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551108 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 19:09:33.553749 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551110 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 19:09:33.553749 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551113 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 19:09:33.553749 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551116 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 19:09:33.553749 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551120 2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 19:09:33.553749 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551124 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 19:09:33.553749 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551127 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 19:09:33.553749 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551130 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 19:09:33.554205 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551133 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 19:09:33.554205 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551136 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 19:09:33.554205 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551139 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 19:09:33.554205 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551142 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 19:09:33.554205 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551144 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 19:09:33.554205 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551147 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 19:09:33.554205 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551149 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 19:09:33.554205 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551152 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 19:09:33.554205 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551156 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 19:09:33.554205 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551158 2566 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 19:09:33.554205 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551161 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 19:09:33.554205 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551163 2566 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 19:09:33.554205 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551166 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 19:09:33.554205 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551168 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 19:09:33.554205 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551171 2566 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 19:09:33.554205 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551173 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 19:09:33.554205 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551175 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 19:09:33.554205 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551179 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 19:09:33.554205 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551181 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 19:09:33.554205 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551184 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 19:09:33.554695 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551186 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 19:09:33.554695 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551189 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 19:09:33.554695 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551191 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 19:09:33.554695 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551194 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 19:09:33.554695 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551196 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 19:09:33.554695 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551198 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 19:09:33.554695 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551204 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 19:09:33.554695 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551206 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 19:09:33.554695 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551208 2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 19:09:33.554695 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551211 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 19:09:33.554695 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551215 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 19:09:33.554695 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551218 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 19:09:33.554695 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551221 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 19:09:33.554695 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551618 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 19:09:33.554695 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551624 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 19:09:33.554695 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551626 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 19:09:33.554695 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551629 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 19:09:33.554695 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551632 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 19:09:33.554695 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551634 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 19:09:33.555163 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551637 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 19:09:33.555163 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551640 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 19:09:33.555163 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551643 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 19:09:33.555163 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551645 2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 19:09:33.555163 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551648 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 19:09:33.555163 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551650 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 19:09:33.555163 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551653 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 19:09:33.555163 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551655 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 19:09:33.555163 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551658 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 19:09:33.555163 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551660 2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 19:09:33.555163 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551663 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 19:09:33.555163 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551666 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 19:09:33.555163 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551669 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 19:09:33.555163 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551671 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 19:09:33.555163 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551674 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 19:09:33.555163 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551676 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 19:09:33.555163 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551679 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 19:09:33.555163 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551681 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 19:09:33.555163 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551684 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 19:09:33.555163 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551689 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 19:09:33.555721 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551691 2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 19:09:33.555721 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551694 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 19:09:33.555721 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551696 2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 19:09:33.555721 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551699 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 19:09:33.555721 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551701 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 19:09:33.555721 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551703 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 19:09:33.555721 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551706 2566 feature_gate.go:328] unrecognized feature gate: Example Jul 06 19:09:33.555721 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551709 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 19:09:33.555721 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551713 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 19:09:33.555721 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551716 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 19:09:33.555721 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551718 2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 19:09:33.555721 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551721 2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 19:09:33.555721 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551723 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 19:09:33.555721 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551726 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 19:09:33.555721 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551728 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 19:09:33.555721 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551731 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 19:09:33.555721 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551733 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 19:09:33.555721 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551736 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 19:09:33.555721 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551738 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 19:09:33.555721 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551741 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 19:09:33.556220 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551744 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 19:09:33.556220 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551747 2566 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 19:09:33.556220 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551749 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 19:09:33.556220 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551752 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 19:09:33.556220 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551754 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 19:09:33.556220 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551757 2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 19:09:33.556220 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551761 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 19:09:33.556220 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551764 2566 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 19:09:33.556220 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551766 2566 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 19:09:33.556220 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551769 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 19:09:33.556220 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551772 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 19:09:33.556220 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551775 2566 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 19:09:33.556220 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551777 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 19:09:33.556220 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551780 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 19:09:33.556220 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551783 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 19:09:33.556220 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551785 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 19:09:33.556220 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551788 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 19:09:33.556220 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551790 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 19:09:33.556220 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551792 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 19:09:33.556220 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551795 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 19:09:33.556758 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551797 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 19:09:33.556758 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551799 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 19:09:33.556758 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551802 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 19:09:33.556758 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551804 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 19:09:33.556758 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551807 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 19:09:33.556758 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551811 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 19:09:33.556758 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551813 2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 19:09:33.556758 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551816 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 19:09:33.556758 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551819 2566 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 19:09:33.556758 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551821 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 19:09:33.556758 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551824 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 19:09:33.556758 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551826 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 19:09:33.556758 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551828 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 19:09:33.556758 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551831 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 19:09:33.556758 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551833 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 19:09:33.556758 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551836 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 19:09:33.556758 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551842 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 19:09:33.556758 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551845 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 19:09:33.556758 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551848 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 19:09:33.557218 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551850 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 19:09:33.557218 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551852 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 19:09:33.557218 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551855 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 19:09:33.557218 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551858 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 19:09:33.557218 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551860 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 19:09:33.557218 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551863 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 19:09:33.557218 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551867 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 19:09:33.557218 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551869 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 19:09:33.557218 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551872 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 19:09:33.557218 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551874 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 19:09:33.557218 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551877 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 19:09:33.557218 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551880 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 19:09:33.557218 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551882 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 19:09:33.557218 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.551886 2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 19:09:33.557218 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.551964 2566 flags.go:64] FLAG: --address="0.0.0.0" Jul 06 19:09:33.557218 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.551977 2566 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 06 19:09:33.557218 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.551984 2566 flags.go:64] FLAG: --anonymous-auth="true" Jul 06 19:09:33.557218 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.551989 2566 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 06 19:09:33.557218 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.551993 2566 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 06 19:09:33.557218 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.551996 2566 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 06 19:09:33.557218 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552000 2566 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 06 19:09:33.557745 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552005 2566 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 06 19:09:33.557745 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552008 2566 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 06 19:09:33.557745 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552011 2566 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 06 19:09:33.557745 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552015 2566 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 06 19:09:33.557745 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552018 2566 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 06 19:09:33.557745 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552021 2566 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 06 19:09:33.557745 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552023 2566 flags.go:64] FLAG: --cgroup-root="" Jul 06 19:09:33.557745 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552026 2566 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 06 19:09:33.557745 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552029 2566 flags.go:64] FLAG: --client-ca-file="" Jul 06 19:09:33.557745 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552033 2566 flags.go:64] FLAG: --cloud-provider="external" Jul 06 19:09:33.557745 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552036 2566 flags.go:64] FLAG: --cluster-dns="[]" Jul 06 19:09:33.557745 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552040 2566 flags.go:64] FLAG: --cluster-domain="" Jul 06 19:09:33.557745 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552043 2566 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 06 19:09:33.557745 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552046 2566 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 06 19:09:33.557745 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552049 2566 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 06 19:09:33.557745 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552052 2566 flags.go:64] FLAG: --container-log-max-files="5" Jul 06 19:09:33.557745 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552056 2566 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 06 19:09:33.557745 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552060 2566 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 06 19:09:33.557745 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552063 2566 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 06 19:09:33.557745 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552066 2566 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 06 19:09:33.557745 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552069 2566 flags.go:64] FLAG: --contention-profiling="false" Jul 06 19:09:33.557745 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552072 2566 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 06 19:09:33.557745 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552075 2566 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 06 19:09:33.557745 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552078 2566 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 06 19:09:33.558365 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552081 2566 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 06 19:09:33.558365 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552085 2566 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 06 19:09:33.558365 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552089 2566 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 06 19:09:33.558365 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552091 2566 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 06 19:09:33.558365 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552094 2566 flags.go:64] FLAG: --enable-load-reader="false" Jul 06 19:09:33.558365 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552097 2566 flags.go:64] FLAG: --enable-server="true" Jul 06 19:09:33.558365 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552100 2566 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 06 19:09:33.558365 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552104 2566 flags.go:64] FLAG: --event-burst="100" Jul 06 19:09:33.558365 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552107 2566 flags.go:64] FLAG: --event-qps="50" Jul 06 19:09:33.558365 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552110 2566 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 06 19:09:33.558365 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552113 2566 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 06 19:09:33.558365 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552116 2566 flags.go:64] FLAG: --eviction-hard="" Jul 06 19:09:33.558365 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552119 2566 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 06 19:09:33.558365 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552122 2566 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 06 19:09:33.558365 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552125 2566 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 06 19:09:33.558365 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552127 2566 flags.go:64] FLAG: --eviction-soft="" Jul 06 19:09:33.558365 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552130 2566 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 06 19:09:33.558365 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552133 2566 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 06 19:09:33.558365 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552137 2566 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 06 19:09:33.558365 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552140 2566 flags.go:64] FLAG: --experimental-mounter-path="" Jul 06 19:09:33.558365 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552142 2566 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 06 19:09:33.558365 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552145 2566 flags.go:64] FLAG: --fail-swap-on="true" Jul 06 19:09:33.558365 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552148 2566 flags.go:64] FLAG: --feature-gates="" Jul 06 19:09:33.558365 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552151 2566 flags.go:64] FLAG: --file-check-frequency="20s" Jul 06 19:09:33.558365 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552154 2566 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 06 19:09:33.559044 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552159 2566 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 06 19:09:33.559044 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552161 2566 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 06 19:09:33.559044 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552164 2566 flags.go:64] FLAG: --healthz-port="10248" Jul 06 19:09:33.559044 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552167 2566 flags.go:64] FLAG: --help="false" Jul 06 19:09:33.559044 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552170 2566 flags.go:64] FLAG: --hostname-override="ip-10-0-138-35.ec2.internal" Jul 06 19:09:33.559044 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552173 2566 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 06 19:09:33.559044 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552176 2566 flags.go:64] FLAG: --http-check-frequency="20s" Jul 06 19:09:33.559044 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552179 2566 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 06 19:09:33.559044 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552182 2566 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 06 19:09:33.559044 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552186 2566 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 06 19:09:33.559044 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552188 2566 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 06 19:09:33.559044 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552191 2566 flags.go:64] FLAG: --image-service-endpoint="" Jul 06 19:09:33.559044 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552194 2566 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 06 19:09:33.559044 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552197 2566 flags.go:64] FLAG: --kube-api-burst="100" Jul 06 19:09:33.559044 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552199 2566 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 06 19:09:33.559044 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552202 2566 flags.go:64] FLAG: --kube-api-qps="50" Jul 06 19:09:33.559044 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552205 2566 flags.go:64] FLAG: --kube-reserved="" Jul 06 19:09:33.559044 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552208 2566 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 06 19:09:33.559044 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552211 2566 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 06 19:09:33.559044 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552214 2566 flags.go:64] FLAG: --kubelet-cgroups="" Jul 06 19:09:33.559044 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552217 2566 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 06 19:09:33.559044 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552220 2566 flags.go:64] FLAG: --lock-file="" Jul 06 19:09:33.559044 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552223 2566 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 06 19:09:33.559044 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552226 2566 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 06 19:09:33.559658 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552228 2566 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 06 19:09:33.559658 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552234 2566 flags.go:64] FLAG: --log-json-split-stream="false" Jul 06 19:09:33.559658 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552237 2566 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 06 19:09:33.559658 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552239 2566 flags.go:64] FLAG: --log-text-split-stream="false" Jul 06 19:09:33.559658 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552242 2566 flags.go:64] FLAG: --logging-format="text" Jul 06 19:09:33.559658 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552246 2566 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 06 19:09:33.559658 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552249 2566 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 06 19:09:33.559658 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552252 2566 flags.go:64] FLAG: --manifest-url="" Jul 06 19:09:33.559658 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552255 2566 flags.go:64] FLAG: --manifest-url-header="" Jul 06 19:09:33.559658 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552259 2566 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 06 19:09:33.559658 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552262 2566 flags.go:64] FLAG: --max-open-files="1000000" Jul 06 19:09:33.559658 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552266 2566 flags.go:64] FLAG: --max-pods="110" Jul 06 19:09:33.559658 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552269 2566 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 06 19:09:33.559658 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552272 2566 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 06 19:09:33.559658 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552275 2566 flags.go:64] FLAG: --memory-manager-policy="None" Jul 06 19:09:33.559658 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552277 2566 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 06 19:09:33.559658 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552280 2566 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 06 19:09:33.559658 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552283 2566 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 06 19:09:33.559658 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552285 2566 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 06 19:09:33.559658 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552292 2566 flags.go:64] FLAG: --node-status-max-images="50" Jul 06 19:09:33.559658 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552295 2566 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 06 19:09:33.559658 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552298 2566 flags.go:64] FLAG: --oom-score-adj="-999" Jul 06 19:09:33.559658 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552301 2566 flags.go:64] FLAG: --pod-cidr="" Jul 06 19:09:33.560214 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552304 2566 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 06 19:09:33.560214 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552308 2566 flags.go:64] FLAG: --pod-manifest-path="" Jul 06 19:09:33.560214 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552311 2566 flags.go:64] FLAG: --pod-max-pids="-1" Jul 06 19:09:33.560214 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552314 2566 flags.go:64] FLAG: --pods-per-core="0" Jul 06 19:09:33.560214 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552317 2566 flags.go:64] FLAG: --port="10250" Jul 06 19:09:33.560214 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552319 2566 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 06 19:09:33.560214 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552322 2566 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0f22f77c6356a54ba" Jul 06 19:09:33.560214 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552325 2566 flags.go:64] FLAG: --qos-reserved="" Jul 06 19:09:33.560214 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552328 2566 flags.go:64] FLAG: --read-only-port="10255" Jul 06 19:09:33.560214 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552331 2566 flags.go:64] FLAG: --register-node="true" Jul 06 19:09:33.560214 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552334 2566 flags.go:64] FLAG: --register-with-taints="" Jul 06 19:09:33.560214 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552337 2566 flags.go:64] FLAG: --registry-burst="10" Jul 06 19:09:33.560214 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552340 2566 flags.go:64] FLAG: --registry-qps="5" Jul 06 19:09:33.560214 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552343 2566 flags.go:64] FLAG: --reserved-cpus="" Jul 06 19:09:33.560214 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552347 2566 flags.go:64] FLAG: --reserved-memory="" Jul 06 19:09:33.560214 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552351 2566 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 06 19:09:33.560214 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552353 2566 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 06 19:09:33.560214 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552356 2566 flags.go:64] FLAG: --rotate-certificates="false" Jul 06 19:09:33.560214 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552359 2566 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 06 19:09:33.560214 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552362 2566 flags.go:64] FLAG: --runonce="false" Jul 06 19:09:33.560214 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552365 2566 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 06 19:09:33.560214 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552368 2566 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 06 19:09:33.560214 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552370 2566 flags.go:64] FLAG: --seccomp-default="false" Jul 06 19:09:33.560214 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552373 2566 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 06 19:09:33.560214 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552376 2566 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 06 19:09:33.560862 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552379 2566 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 06 19:09:33.560862 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552382 2566 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 06 19:09:33.560862 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552385 2566 flags.go:64] FLAG: --storage-driver-password="root" Jul 06 19:09:33.560862 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552388 2566 flags.go:64] FLAG: --storage-driver-secure="false" Jul 06 19:09:33.560862 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552391 2566 flags.go:64] FLAG: --storage-driver-table="stats" Jul 06 19:09:33.560862 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552394 2566 flags.go:64] FLAG: --storage-driver-user="root" Jul 06 19:09:33.560862 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552396 2566 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 06 19:09:33.560862 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552399 2566 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 06 19:09:33.560862 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552402 2566 flags.go:64] FLAG: --system-cgroups="" Jul 06 19:09:33.560862 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552405 2566 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 06 19:09:33.560862 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552409 2566 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 06 19:09:33.560862 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552412 2566 flags.go:64] FLAG: --tls-cert-file="" Jul 06 19:09:33.560862 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552415 2566 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 06 19:09:33.560862 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552419 2566 flags.go:64] FLAG: --tls-min-version="" Jul 06 19:09:33.560862 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552421 2566 flags.go:64] FLAG: --tls-private-key-file="" Jul 06 19:09:33.560862 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552424 2566 flags.go:64] FLAG: --topology-manager-policy="none" Jul 06 19:09:33.560862 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552427 2566 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 06 19:09:33.560862 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552429 2566 flags.go:64] FLAG: --topology-manager-scope="container" Jul 06 19:09:33.560862 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552432 2566 flags.go:64] FLAG: --v="2" Jul 06 19:09:33.560862 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552436 2566 flags.go:64] FLAG: --version="false" Jul 06 19:09:33.560862 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552440 2566 flags.go:64] FLAG: --vmodule="" Jul 06 19:09:33.560862 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552445 2566 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 06 19:09:33.560862 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552448 2566 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 06 19:09:33.560862 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.552576 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 19:09:33.560862 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.552581 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 19:09:33.561474 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.552584 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 19:09:33.561474 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.552587 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 19:09:33.561474 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.552590 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 19:09:33.561474 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.552593 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 19:09:33.561474 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.552596 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 19:09:33.561474 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.552598 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 19:09:33.561474 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.552601 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 19:09:33.561474 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.552603 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 19:09:33.561474 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.552606 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 19:09:33.561474 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.552608 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 19:09:33.561474 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.552611 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 19:09:33.561474 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.552613 2566 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 19:09:33.561474 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.552616 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 19:09:33.561474 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.552618 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 19:09:33.561474 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.552621 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 19:09:33.561474 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.552623 2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 19:09:33.561474 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.552626 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 19:09:33.561474 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.552628 2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 19:09:33.561474 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.552630 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 19:09:33.561474 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.552632 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 19:09:33.562060 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.552635 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 19:09:33.562060 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.552638 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 19:09:33.562060 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.552640 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 19:09:33.562060 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.552643 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 19:09:33.562060 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.552645 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 19:09:33.562060 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.552648 2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 19:09:33.562060 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.552650 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 19:09:33.562060 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.552652 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 19:09:33.562060 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.552657 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 19:09:33.562060 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.552660 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 19:09:33.562060 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.552662 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 19:09:33.562060 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.552664 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 19:09:33.562060 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.552667 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 19:09:33.562060 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.552669 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 19:09:33.562060 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.552672 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 19:09:33.562060 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.552674 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 19:09:33.562060 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.552676 2566 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 19:09:33.562060 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.552679 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 19:09:33.562060 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.552681 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 19:09:33.562060 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.552683 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 19:09:33.562574 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.552686 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 19:09:33.562574 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.552688 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 19:09:33.562574 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.552691 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 19:09:33.562574 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.552693 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 19:09:33.562574 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.552695 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 19:09:33.562574 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.552698 2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 19:09:33.562574 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.552700 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 19:09:33.562574 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.552703 2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 19:09:33.562574 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.552705 2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 19:09:33.562574 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.552707 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 19:09:33.562574 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.552710 2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 19:09:33.562574 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.552712 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 19:09:33.562574 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.552714 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 19:09:33.562574 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.552717 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 19:09:33.562574 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.552719 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 19:09:33.562574 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.552721 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 19:09:33.562574 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.552724 2566 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 19:09:33.562574 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.552727 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 19:09:33.562574 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.552729 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 19:09:33.562574 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.552733 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 19:09:33.563064 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.552737 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 19:09:33.563064 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.552740 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 19:09:33.563064 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.552742 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 19:09:33.563064 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.552745 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 19:09:33.563064 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.552749 2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 19:09:33.563064 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.552752 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 19:09:33.563064 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.552755 2566 feature_gate.go:328] unrecognized feature gate: Example Jul 06 19:09:33.563064 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.552758 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 19:09:33.563064 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.552760 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 19:09:33.563064 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.552763 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 19:09:33.563064 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.552766 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 19:09:33.563064 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.552768 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 19:09:33.563064 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.552771 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 19:09:33.563064 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.552773 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 19:09:33.563064 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.552776 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 19:09:33.563064 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.552778 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 19:09:33.563064 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.552781 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 19:09:33.563064 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.552783 2566 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 19:09:33.563064 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.552786 2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 19:09:33.563064 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.552788 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 19:09:33.563665 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.552791 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 19:09:33.563665 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.552793 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 19:09:33.563665 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.552795 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 19:09:33.563665 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.552798 2566 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 19:09:33.563665 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.552800 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 19:09:33.563665 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.552804 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 19:09:33.563665 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.552807 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 19:09:33.563665 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.552810 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 19:09:33.563665 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.552813 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 19:09:33.563665 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.552815 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 19:09:33.563665 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.552817 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 19:09:33.563665 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.552822 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 19:09:33.563665 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.552827 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 19:09:33.563665 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.552830 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 19:09:33.563665 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.552833 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 19:09:33.563665 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.552836 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 19:09:33.563665 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.552838 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 19:09:33.564119 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.552844 2566 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 06 19:09:33.564119 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.559963 2566 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 06 19:09:33.564119 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.559981 2566 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 06 19:09:33.564119 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.560917 2566 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 06 19:09:33.564119 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.560924 2566 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 06 19:09:33.564119 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.560996 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 19:09:33.564119 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561001 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 19:09:33.564119 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561005 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 19:09:33.564119 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561008 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 19:09:33.564119 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561011 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 19:09:33.564119 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561014 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 19:09:33.564119 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561017 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 19:09:33.564119 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561019 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 19:09:33.564119 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561022 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 19:09:33.564119 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561025 2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 19:09:33.564543 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561028 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 19:09:33.564543 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561032 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 19:09:33.564543 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561036 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 19:09:33.564543 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561038 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 19:09:33.564543 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561041 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 19:09:33.564543 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561044 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 19:09:33.564543 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561046 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 19:09:33.564543 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561049 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 19:09:33.564543 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561052 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 19:09:33.564543 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561054 2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 19:09:33.564543 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561057 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 19:09:33.564543 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561059 2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 19:09:33.564543 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561062 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 19:09:33.564543 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561064 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 19:09:33.564543 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561067 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 19:09:33.564543 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561070 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 19:09:33.564543 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561073 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 19:09:33.564543 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561075 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 19:09:33.564543 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561078 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 19:09:33.565015 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561081 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 19:09:33.565015 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561083 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 19:09:33.565015 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561086 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 19:09:33.565015 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561089 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 19:09:33.565015 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561091 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 19:09:33.565015 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561094 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 19:09:33.565015 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561096 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 19:09:33.565015 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561099 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 19:09:33.565015 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561102 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 19:09:33.565015 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561104 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 19:09:33.565015 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561106 2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 19:09:33.565015 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561109 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 19:09:33.565015 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561112 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 19:09:33.565015 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561116 2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 19:09:33.565015 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561119 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 19:09:33.565015 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561122 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 19:09:33.565015 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561124 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 19:09:33.565015 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561127 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 19:09:33.565015 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561130 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 19:09:33.565726 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561133 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 19:09:33.565726 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561135 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 19:09:33.565726 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561137 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 19:09:33.565726 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561140 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 19:09:33.565726 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561142 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 19:09:33.565726 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561145 2566 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 19:09:33.565726 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561147 2566 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 19:09:33.565726 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561149 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 19:09:33.565726 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561152 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 19:09:33.565726 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561155 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 19:09:33.565726 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561157 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 19:09:33.565726 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561160 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 19:09:33.565726 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561162 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 19:09:33.565726 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561165 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 19:09:33.565726 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561169 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 19:09:33.565726 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561171 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 19:09:33.565726 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561173 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 19:09:33.565726 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561176 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 19:09:33.565726 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561178 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 19:09:33.565726 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561181 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 19:09:33.565726 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561184 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 19:09:33.566238 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561186 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 19:09:33.566238 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561189 2566 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 19:09:33.566238 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561193 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 19:09:33.566238 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561196 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 19:09:33.566238 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561199 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 19:09:33.566238 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561201 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 19:09:33.566238 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561204 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 19:09:33.566238 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561206 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 19:09:33.566238 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561209 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 19:09:33.566238 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561212 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 19:09:33.566238 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561214 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 19:09:33.566238 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561217 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 19:09:33.566238 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561219 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 19:09:33.566238 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561222 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 19:09:33.566238 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561224 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 19:09:33.566238 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561227 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 19:09:33.566238 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561229 2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 19:09:33.566238 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561232 2566 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 19:09:33.566238 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561234 2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 19:09:33.566762 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561237 2566 feature_gate.go:328] unrecognized feature gate: Example Jul 06 19:09:33.566762 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561239 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 19:09:33.566762 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561242 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 19:09:33.566762 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561244 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 19:09:33.566762 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561247 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 19:09:33.566762 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561250 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 19:09:33.566762 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561253 2566 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 19:09:33.566762 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561255 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 19:09:33.566762 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561257 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 19:09:33.566762 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561260 2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 19:09:33.566762 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561262 2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 19:09:33.566762 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.561267 2566 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 06 19:09:33.566762 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561398 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 06 19:09:33.566762 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561403 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 06 19:09:33.566762 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561406 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 06 19:09:33.567139 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561409 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 06 19:09:33.567139 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561412 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 06 19:09:33.567139 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561415 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 06 19:09:33.567139 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561418 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 06 19:09:33.567139 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561421 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 06 19:09:33.567139 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561423 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 06 19:09:33.567139 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561426 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 06 19:09:33.567139 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561429 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 06 19:09:33.567139 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561431 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 06 19:09:33.567139 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561434 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 06 19:09:33.567139 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561437 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 06 19:09:33.567139 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561440 2566 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 06 19:09:33.567139 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561443 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 06 19:09:33.567139 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561445 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 06 19:09:33.567139 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561448 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 06 19:09:33.567139 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561450 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 06 19:09:33.567139 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561453 2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 06 19:09:33.567139 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561456 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 06 19:09:33.567139 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561458 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 06 19:09:33.567139 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561460 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 06 19:09:33.567732 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561463 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 06 19:09:33.567732 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561466 2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 06 19:09:33.567732 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561469 2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 06 19:09:33.567732 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561472 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 06 19:09:33.567732 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561474 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 06 19:09:33.567732 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561477 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 06 19:09:33.567732 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561480 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 06 19:09:33.567732 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561482 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 06 19:09:33.567732 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561486 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 06 19:09:33.567732 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561490 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 06 19:09:33.567732 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561493 2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 06 19:09:33.567732 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561495 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 06 19:09:33.567732 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561498 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 06 19:09:33.567732 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561518 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 06 19:09:33.567732 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561522 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 06 19:09:33.567732 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561525 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 06 19:09:33.567732 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561527 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 06 19:09:33.567732 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561530 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 06 19:09:33.567732 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561532 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 06 19:09:33.568341 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561535 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 06 19:09:33.568341 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561537 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 06 19:09:33.568341 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561540 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 06 19:09:33.568341 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561542 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 06 19:09:33.568341 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561545 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 06 19:09:33.568341 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561547 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 06 19:09:33.568341 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561550 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 06 19:09:33.568341 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561552 2566 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 06 19:09:33.568341 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561555 2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 06 19:09:33.568341 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561557 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 06 19:09:33.568341 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561560 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 06 19:09:33.568341 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561562 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 06 19:09:33.568341 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561564 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 06 19:09:33.568341 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561567 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 06 19:09:33.568341 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561571 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 06 19:09:33.568341 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561575 2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 06 19:09:33.568341 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561578 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 06 19:09:33.568341 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561580 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 06 19:09:33.568341 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561584 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 06 19:09:33.569147 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561587 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 06 19:09:33.569147 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561589 2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 06 19:09:33.569147 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561592 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 06 19:09:33.569147 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561594 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 06 19:09:33.569147 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561597 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 06 19:09:33.569147 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561600 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 06 19:09:33.569147 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561602 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 06 19:09:33.569147 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561605 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 06 19:09:33.569147 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561607 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 06 19:09:33.569147 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561611 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 06 19:09:33.569147 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561613 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 06 19:09:33.569147 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561615 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 06 19:09:33.569147 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561618 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 06 19:09:33.569147 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561621 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 06 19:09:33.569147 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561623 2566 feature_gate.go:328] unrecognized feature gate: Example2 Jul 06 19:09:33.569147 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561626 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 06 19:09:33.569147 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561628 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 06 19:09:33.569147 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561630 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 06 19:09:33.569147 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561633 2566 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 06 19:09:33.569147 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561635 2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 06 19:09:33.569717 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561638 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 06 19:09:33.569717 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561641 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 06 19:09:33.569717 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561643 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 06 19:09:33.569717 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561646 2566 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 06 19:09:33.569717 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561648 2566 feature_gate.go:328] unrecognized feature gate: Example Jul 06 19:09:33.569717 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561650 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 06 19:09:33.569717 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561653 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 06 19:09:33.569717 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561655 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 06 19:09:33.569717 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561665 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 06 19:09:33.569717 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561668 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 06 19:09:33.569717 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561670 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 06 19:09:33.569717 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561673 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 06 19:09:33.569717 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561675 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 06 19:09:33.569717 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561677 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 06 19:09:33.569717 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561680 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 06 19:09:33.569717 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561702 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 06 19:09:33.569717 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561705 2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 06 19:09:33.569717 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:33.561708 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 06 19:09:33.570162 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.561714 2566 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 06 19:09:33.570162 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.561834 2566 server.go:962] "Client rotation is on, will bootstrap in background" Jul 06 19:09:33.570162 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.563887 2566 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 06 19:09:33.570162 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.565295 2566 server.go:1019] "Starting client certificate rotation" Jul 06 19:09:33.570162 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.565426 2566 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 06 19:09:33.570162 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.565472 2566 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 06 19:09:33.570162 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.567789 2566 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 06 19:09:33.571191 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.569536 2566 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 06 19:09:33.578755 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.578707 2566 log.go:25] "Validated CRI v1 runtime API" Jul 06 19:09:33.580710 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.580694 2566 log.go:25] "Validated CRI v1 image API" Jul 06 19:09:33.581598 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.581578 2566 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 06 19:09:33.587954 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.587929 2566 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 d834c8ea-e4cc-48d9-9d5e-2097ca63f8e8:/dev/nvme0n1p4 ea69a825-6aae-42e3-9990-c0ff6735abd0:/dev/nvme0n1p3] Jul 06 19:09:33.588044 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.587954 2566 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 06 19:09:33.593446 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.593318 2566 manager.go:216] Machine: {Timestamp:2026-07-06 19:09:33.591872364 +0000 UTC m=+0.326793757 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3103954 MemoryCapacity:32812171264 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec28ff144ff2c9a013649dd505e8c9f2 SystemUUID:ec28ff14-4ff2-c9a0-1364-9dd505e8c9f2 BootID:3ea95f76-476c-42ca-9cbb-058d4009e1b2 Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406085632 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:c1:ff:4c:b6:fb Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:c1:ff:4c:b6:fb Speed:0 Mtu:9001} {Name:ovs-system MacAddress:e6:4b:3e:d0:5b:aa Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812171264 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 06 19:09:33.593446 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.593434 2566 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 06 19:09:33.593588 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.593569 2566 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 06 19:09:33.596593 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.596566 2566 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 06 19:09:33.596752 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.596596 2566 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-138-35.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 06 19:09:33.597722 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.597706 2566 topology_manager.go:138] "Creating topology manager with none policy" Jul 06 19:09:33.597722 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.597722 2566 container_manager_linux.go:309] "Creating device plugin manager" Jul 06 19:09:33.597821 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.597729 2566 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 06 19:09:33.598335 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.598322 2566 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 06 19:09:33.599161 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.599139 2566 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 06 19:09:33.600760 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.600745 2566 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 06 19:09:33.603907 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.603891 2566 state_mem.go:36] "Initialized new in-memory state store" Jul 06 19:09:33.604052 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.604042 2566 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 06 19:09:33.604082 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.604076 2566 kubelet.go:486] "Attempting to sync node with API server" Jul 06 19:09:33.604110 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.604087 2566 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 06 19:09:33.604110 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.604102 2566 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 06 19:09:33.604110 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.604111 2566 kubelet.go:398] "Adding apiserver pod source" Jul 06 19:09:33.604203 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.604120 2566 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 06 19:09:33.604563 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.604551 2566 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 06 19:09:33.604608 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.604573 2566 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 06 19:09:33.607522 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.607488 2566 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 06 19:09:33.609363 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.609339 2566 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 06 19:09:33.609450 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.609383 2566 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 06 19:09:33.610821 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.610807 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 06 19:09:33.610903 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.610828 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 06 19:09:33.610903 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.610840 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 06 19:09:33.610903 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.610848 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 06 19:09:33.610903 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.610861 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 06 19:09:33.610903 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.610870 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 06 19:09:33.610903 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.610881 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 06 19:09:33.610903 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.610889 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 06 19:09:33.610903 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.610898 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 06 19:09:33.610903 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.610907 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 06 19:09:33.611186 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.610919 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 06 19:09:33.611186 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.610929 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 06 19:09:33.612215 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.612203 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 06 19:09:33.612267 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.612218 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 06 19:09:33.615731 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.615713 2566 server.go:1268] "Started kubelet" Jul 06 19:09:33.615825 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.615798 2566 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 06 19:09:33.615872 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.615839 2566 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 06 19:09:33.618917 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.615821 2566 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 06 19:09:33.619911 ip-10-0-138-35 systemd[1]: Started Kubernetes Kubelet. Jul 06 19:09:33.620192 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:09:33.620091 2566 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-138-35.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 06 19:09:33.620192 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:09:33.620091 2566 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 06 19:09:33.620320 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.620300 2566 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-138-35.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 06 19:09:33.621484 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.620904 2566 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 06 19:09:33.621484 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.621480 2566 server.go:310] "Adding debug handlers to kubelet server" Jul 06 19:09:33.624572 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:09:33.624551 2566 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 06 19:09:33.626415 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:09:33.625552 2566 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-35.ec2.internal.18bfc8fead3bade0 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-35.ec2.internal,UID:ip-10-0-138-35.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-138-35.ec2.internal,},FirstTimestamp:2026-07-06 19:09:33.615623648 +0000 UTC m=+0.350545040,LastTimestamp:2026-07-06 19:09:33.615623648 +0000 UTC m=+0.350545040,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-35.ec2.internal,}" Jul 06 19:09:33.631203 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.631187 2566 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 06 19:09:33.631248 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.631189 2566 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 06 19:09:33.631362 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:09:33.631336 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-35.ec2.internal\" not found" Jul 06 19:09:33.631964 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.631938 2566 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 06 19:09:33.632192 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.632174 2566 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 06 19:09:33.632192 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.631943 2566 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 06 19:09:33.632315 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.632110 2566 factory.go:55] Registering systemd factory Jul 06 19:09:33.632315 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.632213 2566 factory.go:223] Registration of the systemd container factory successfully Jul 06 19:09:33.632397 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.632316 2566 reconstruct.go:97] "Volume reconstruction finished" Jul 06 19:09:33.632397 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.632323 2566 reconciler.go:29] "Reconciler: start to sync state" Jul 06 19:09:33.632478 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.632422 2566 factory.go:153] Registering CRI-O factory Jul 06 19:09:33.632478 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.632434 2566 factory.go:223] Registration of the crio container factory successfully Jul 06 19:09:33.632590 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.632491 2566 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 06 19:09:33.632590 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.632547 2566 factory.go:103] Registering Raw factory Jul 06 19:09:33.632590 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.632563 2566 manager.go:1196] Started watching for new ooms in manager Jul 06 19:09:33.632941 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.632927 2566 manager.go:349] Starting recovery of all containers Jul 06 19:09:33.633067 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:09:33.633044 2566 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-138-35.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 06 19:09:33.633343 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:09:33.633319 2566 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 06 19:09:33.640701 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.640479 2566 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 06 19:09:33.643717 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.643692 2566 manager.go:354] Recovery completed Jul 06 19:09:33.645456 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:09:33.645425 2566 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": readdirent /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory Jul 06 19:09:33.647793 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.647759 2566 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-qxbzm" Jul 06 19:09:33.649109 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.649096 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 19:09:33.651410 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.651394 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-35.ec2.internal" event="NodeHasSufficientMemory" Jul 06 19:09:33.651483 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.651428 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-35.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 19:09:33.651483 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.651439 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-35.ec2.internal" event="NodeHasSufficientPID" Jul 06 19:09:33.652531 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.652498 2566 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 06 19:09:33.652531 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.652530 2566 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 06 19:09:33.652693 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.652548 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 19:09:33.652693 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.652549 2566 state_mem.go:36] "Initialized new in-memory state store" Jul 06 19:09:33.653250 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.653234 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-35.ec2.internal" event="NodeHasSufficientMemory" Jul 06 19:09:33.653333 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.653263 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-35.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 19:09:33.653333 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.653288 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-35.ec2.internal" event="NodeHasSufficientPID" Jul 06 19:09:33.653927 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.653914 2566 policy_none.go:49] "None policy: Start" Jul 06 19:09:33.653966 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.653933 2566 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 06 19:09:33.653966 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.653943 2566 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 06 19:09:33.654866 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.654854 2566 policy_none.go:47] "Start" Jul 06 19:09:33.655402 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.655388 2566 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-qxbzm" Jul 06 19:09:33.694123 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.694101 2566 manager.go:337] "Starting Device Plugin manager" Jul 06 19:09:33.694253 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:09:33.694145 2566 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 06 19:09:33.694253 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.694158 2566 server.go:87] "Starting device plugin registration server" Jul 06 19:09:33.694456 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.694444 2566 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 06 19:09:33.694493 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.694459 2566 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 06 19:09:33.694780 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.694591 2566 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 06 19:09:33.694780 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.694707 2566 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 06 19:09:33.694780 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.694716 2566 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 06 19:09:33.695280 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:09:33.695250 2566 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 06 19:09:33.695349 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:09:33.695310 2566 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-138-35.ec2.internal\" not found" Jul 06 19:09:33.795124 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.795027 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 19:09:33.796307 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.796290 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-35.ec2.internal" event="NodeHasSufficientMemory" Jul 06 19:09:33.796377 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.796324 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-35.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 19:09:33.796377 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.796334 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-35.ec2.internal" event="NodeHasSufficientPID" Jul 06 19:09:33.796377 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.796367 2566 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-138-35.ec2.internal" Jul 06 19:09:33.802250 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.802225 2566 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 06 19:09:33.802364 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.802270 2566 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 06 19:09:33.802364 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.802296 2566 kubelet.go:2448] "Starting kubelet main sync loop" Jul 06 19:09:33.802364 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:09:33.802333 2566 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 06 19:09:33.804209 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.804192 2566 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 19:09:33.806149 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.806132 2566 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-138-35.ec2.internal" Jul 06 19:09:33.806214 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:09:33.806162 2566 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-138-35.ec2.internal\": node \"ip-10-0-138-35.ec2.internal\" not found" Jul 06 19:09:33.847423 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:09:33.847385 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-35.ec2.internal\" not found" Jul 06 19:09:33.903684 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.903084 2566 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-35.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-138-35.ec2.internal"] Jul 06 19:09:33.903781 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.903730 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 19:09:33.906012 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.905996 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-35.ec2.internal" event="NodeHasSufficientMemory" Jul 06 19:09:33.906095 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.906027 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-35.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 19:09:33.906095 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.906048 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-35.ec2.internal" event="NodeHasSufficientPID" Jul 06 19:09:33.907619 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.907605 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 19:09:33.907763 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.907746 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-35.ec2.internal" Jul 06 19:09:33.907810 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.907788 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 19:09:33.911793 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.911778 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-35.ec2.internal" event="NodeHasSufficientMemory" Jul 06 19:09:33.911873 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.911807 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-35.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 19:09:33.911873 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.911818 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-35.ec2.internal" event="NodeHasSufficientPID" Jul 06 19:09:33.911937 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.911817 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-35.ec2.internal" event="NodeHasSufficientMemory" Jul 06 19:09:33.911937 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.911923 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-35.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 19:09:33.912003 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.911939 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-35.ec2.internal" event="NodeHasSufficientPID" Jul 06 19:09:33.912948 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.912933 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-35.ec2.internal" Jul 06 19:09:33.913014 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.912961 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 06 19:09:33.914768 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.914748 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-35.ec2.internal" event="NodeHasSufficientMemory" Jul 06 19:09:33.914839 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.914776 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-35.ec2.internal" event="NodeHasNoDiskPressure" Jul 06 19:09:33.914839 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.914787 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-35.ec2.internal" event="NodeHasSufficientPID" Jul 06 19:09:33.933870 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.933845 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/04637dd40c4987c56de8cdb4373521aa-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-35.ec2.internal\" (UID: \"04637dd40c4987c56de8cdb4373521aa\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-35.ec2.internal" Jul 06 19:09:33.933987 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.933874 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/04637dd40c4987c56de8cdb4373521aa-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-35.ec2.internal\" (UID: \"04637dd40c4987c56de8cdb4373521aa\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-35.ec2.internal" Jul 06 19:09:33.933987 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:33.933902 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9fd512c517b497732795138b0a0b8677-config\") pod \"kube-apiserver-proxy-ip-10-0-138-35.ec2.internal\" (UID: \"9fd512c517b497732795138b0a0b8677\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-35.ec2.internal" Jul 06 19:09:33.944323 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:09:33.944304 2566 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-35.ec2.internal\" not found" node="ip-10-0-138-35.ec2.internal" Jul 06 19:09:33.947579 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:09:33.947564 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-35.ec2.internal\" not found" Jul 06 19:09:33.948828 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:09:33.948814 2566 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-35.ec2.internal\" not found" node="ip-10-0-138-35.ec2.internal" Jul 06 19:09:34.034895 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:34.034857 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9fd512c517b497732795138b0a0b8677-config\") pod \"kube-apiserver-proxy-ip-10-0-138-35.ec2.internal\" (UID: \"9fd512c517b497732795138b0a0b8677\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-35.ec2.internal" Jul 06 19:09:34.034895 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:34.034899 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/04637dd40c4987c56de8cdb4373521aa-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-35.ec2.internal\" (UID: \"04637dd40c4987c56de8cdb4373521aa\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-35.ec2.internal" Jul 06 19:09:34.035087 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:34.034921 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/04637dd40c4987c56de8cdb4373521aa-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-35.ec2.internal\" (UID: \"04637dd40c4987c56de8cdb4373521aa\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-35.ec2.internal" Jul 06 19:09:34.035087 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:34.034970 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9fd512c517b497732795138b0a0b8677-config\") pod \"kube-apiserver-proxy-ip-10-0-138-35.ec2.internal\" (UID: \"9fd512c517b497732795138b0a0b8677\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-35.ec2.internal" Jul 06 19:09:34.035087 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:34.034981 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/04637dd40c4987c56de8cdb4373521aa-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-35.ec2.internal\" (UID: \"04637dd40c4987c56de8cdb4373521aa\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-35.ec2.internal" Jul 06 19:09:34.035087 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:34.034997 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/04637dd40c4987c56de8cdb4373521aa-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-35.ec2.internal\" (UID: \"04637dd40c4987c56de8cdb4373521aa\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-35.ec2.internal" Jul 06 19:09:34.048037 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:09:34.047976 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-35.ec2.internal\" not found" Jul 06 19:09:34.148900 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:09:34.148870 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-35.ec2.internal\" not found" Jul 06 19:09:35.674087 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:35.674012 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-35.ec2.internal" Jul 06 19:09:35.676582 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:09:35.676561 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-35.ec2.internal\" not found" Jul 06 19:09:35.679330 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:35.679313 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-35.ec2.internal" Jul 06 19:09:35.777197 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:09:35.777096 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-35.ec2.internal\" not found" Jul 06 19:09:35.877729 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:09:35.877678 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-35.ec2.internal\" not found" Jul 06 19:09:35.978221 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:09:35.978181 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-35.ec2.internal\" not found" Jul 06 19:09:35.992459 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:35.992419 2566 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 06 19:09:35.992632 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:35.992614 2566 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 06 19:09:36.078487 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:09:36.078395 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-35.ec2.internal\" not found" Jul 06 19:09:36.087587 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:36.087546 2566 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-05 19:04:35 +0000 UTC" deadline="2027-12-15 15:44:51.133312597 +0000 UTC" Jul 06 19:09:36.087587 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:36.087583 2566 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12644h35m15.045732664s" Jul 06 19:09:36.179242 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:09:36.179196 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-35.ec2.internal\" not found" Jul 06 19:09:36.279840 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:09:36.279810 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-35.ec2.internal\" not found" Jul 06 19:09:36.339681 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:36.339600 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9fd512c517b497732795138b0a0b8677.slice/crio-8d208087256911384d8fb2020088d37f221eb423a23bbf9d49d8b767c50ae9b3 WatchSource:0}: Error finding container 8d208087256911384d8fb2020088d37f221eb423a23bbf9d49d8b767c50ae9b3: Status 404 returned error can't find the container with id 8d208087256911384d8fb2020088d37f221eb423a23bbf9d49d8b767c50ae9b3 Jul 06 19:09:36.339820 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:36.339806 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04637dd40c4987c56de8cdb4373521aa.slice/crio-5ccbd25b87a735a682097b1439784eacb1fe6d8ee76acfbd9a1f935370c9bb72 WatchSource:0}: Error finding container 5ccbd25b87a735a682097b1439784eacb1fe6d8ee76acfbd9a1f935370c9bb72: Status 404 returned error can't find the container with id 5ccbd25b87a735a682097b1439784eacb1fe6d8ee76acfbd9a1f935370c9bb72 Jul 06 19:09:36.344360 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:36.344338 2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 19:09:36.380795 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:09:36.380741 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-35.ec2.internal\" not found" Jul 06 19:09:36.383423 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:36.383407 2566 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 19:09:36.423337 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:36.423295 2566 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 19:09:36.481566 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:09:36.481524 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-35.ec2.internal\" not found" Jul 06 19:09:36.582462 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:09:36.582422 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-35.ec2.internal\" not found" Jul 06 19:09:36.603716 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:36.603647 2566 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 19:09:36.659364 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:36.659332 2566 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-35.ec2.internal" Jul 06 19:09:36.670988 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:36.670960 2566 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 06 19:09:36.673490 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:36.673180 2566 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-35.ec2.internal" Jul 06 19:09:36.680733 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:36.680707 2566 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 06 19:09:36.797873 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:36.797840 2566 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 06 19:09:37.033730 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.033638 2566 apiserver.go:52] "Watching apiserver" Jul 06 19:09:37.039478 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.039443 2566 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 06 19:09:37.041638 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.041607 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/node-ca-qbwhx","openshift-multus/multus-s84ln","openshift-multus/network-metrics-daemon-8mnph","openshift-network-diagnostics/network-check-target-f9z4j","openshift-ovn-kubernetes/ovnkube-node-4g4j2","kube-system/kube-apiserver-proxy-ip-10-0-138-35.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vjjbn","openshift-cluster-node-tuning-operator/tuned-f9jxc","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-35.ec2.internal","openshift-multus/multus-additional-cni-plugins-65k2z","openshift-network-operator/iptables-alerter-mznhh","kube-system/global-pull-secret-syncer-svgl2","kube-system/konnectivity-agent-fjdvf","openshift-dns/node-resolver-8btbx"] Jul 06 19:09:37.043145 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.043113 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vjjbn" Jul 06 19:09:37.044259 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.044237 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-fjdvf" Jul 06 19:09:37.045718 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.045428 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8mnph" Jul 06 19:09:37.045718 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.045462 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 06 19:09:37.045718 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.045437 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 06 19:09:37.045718 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:09:37.045539 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8mnph" podUID="6b76700e-1870-40a7-a1f2-ab3c1332058c" Jul 06 19:09:37.045718 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.045439 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-4t98n\"" Jul 06 19:09:37.046169 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.046148 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-hk2jv\"" Jul 06 19:09:37.046257 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.046249 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 06 19:09:37.046317 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.046253 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 06 19:09:37.046404 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.046384 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 06 19:09:37.046722 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.046702 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f9z4j" Jul 06 19:09:37.046819 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:09:37.046765 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f9z4j" podUID="2f2852c7-f9ef-445c-aa57-e5c697cb2c4b" Jul 06 19:09:37.048009 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.047988 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-4g4j2" Jul 06 19:09:37.049907 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.049885 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-8btbx" Jul 06 19:09:37.050423 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.050404 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 06 19:09:37.050554 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.050543 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 06 19:09:37.050628 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.050543 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 06 19:09:37.050828 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.050811 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 06 19:09:37.050863 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.050838 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 06 19:09:37.051107 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.051093 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-vl4lx\"" Jul 06 19:09:37.051166 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.051154 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 06 19:09:37.052128 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.052109 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 06 19:09:37.052623 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.052599 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 06 19:09:37.052708 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.052690 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-g9k42\"" Jul 06 19:09:37.053219 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.053201 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-f9jxc" Jul 06 19:09:37.054773 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.054391 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-qbwhx" Jul 06 19:09:37.056217 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.055632 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 06 19:09:37.056217 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.055753 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-8pksp\"" Jul 06 19:09:37.056217 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.055782 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-s84ln" Jul 06 19:09:37.056217 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.055917 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 06 19:09:37.056576 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.056456 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 06 19:09:37.057867 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.057786 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 06 19:09:37.057867 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.057821 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-tw488\"" Jul 06 19:09:37.058260 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.058092 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 06 19:09:37.058649 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.058632 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 06 19:09:37.059137 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.059043 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-65k2z" Jul 06 19:09:37.059413 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.059393 2566 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 06 19:09:37.059858 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.059833 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 06 19:09:37.060529 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.060137 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-fpcgn\"" Jul 06 19:09:37.060529 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.060199 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 06 19:09:37.060529 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.060144 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 06 19:09:37.061731 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.060919 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-mznhh" Jul 06 19:09:37.063262 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.063236 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-svgl2" Jul 06 19:09:37.063389 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.063366 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-cb7hv\"" Jul 06 19:09:37.063457 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:09:37.063408 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-svgl2" podUID="e8c2f347-1744-49ff-b46e-f036d27a5e46" Jul 06 19:09:37.063617 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.063597 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 06 19:09:37.063682 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.063627 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 06 19:09:37.064697 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.064675 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 06 19:09:37.064849 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.064822 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 06 19:09:37.064967 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.064951 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 06 19:09:37.065382 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.065364 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-dpwdb\"" Jul 06 19:09:37.071836 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.071789 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c75820ce-6423-4c3f-8e13-0ab0e5b00914-hosts-file\") pod \"node-resolver-8btbx\" (UID: \"c75820ce-6423-4c3f-8e13-0ab0e5b00914\") " pod="openshift-dns/node-resolver-8btbx" Jul 06 19:09:37.071836 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.071828 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/51b731db-0d7f-4a6d-bd3b-30d452687c30-cni-binary-copy\") pod \"multus-s84ln\" (UID: \"51b731db-0d7f-4a6d-bd3b-30d452687c30\") " pod="openshift-multus/multus-s84ln" Jul 06 19:09:37.072021 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.071855 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/51b731db-0d7f-4a6d-bd3b-30d452687c30-multus-daemon-config\") pod \"multus-s84ln\" (UID: \"51b731db-0d7f-4a6d-bd3b-30d452687c30\") " pod="openshift-multus/multus-s84ln" Jul 06 19:09:37.072021 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.071879 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h4nf5\" (UniqueName: \"kubernetes.io/projected/f1803bd2-22ae-4026-b66d-320e1c122a10-kube-api-access-h4nf5\") pod \"iptables-alerter-mznhh\" (UID: \"f1803bd2-22ae-4026-b66d-320e1c122a10\") " pod="openshift-network-operator/iptables-alerter-mznhh" Jul 06 19:09:37.072021 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.071945 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6b947af4-1d82-43fd-9137-73d083aed449-device-dir\") pod \"aws-ebs-csi-driver-node-vjjbn\" (UID: \"6b947af4-1d82-43fd-9137-73d083aed449\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vjjbn" Jul 06 19:09:37.072021 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.071978 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/fa566ec3-6b97-4393-9744-5a9f7e8f0614-agent-certs\") pod \"konnectivity-agent-fjdvf\" (UID: \"fa566ec3-6b97-4393-9744-5a9f7e8f0614\") " pod="kube-system/konnectivity-agent-fjdvf" Jul 06 19:09:37.072021 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.072004 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/12b34f02-96ce-44b0-84bc-2a2aa73a9e78-tmp\") pod \"tuned-f9jxc\" (UID: \"12b34f02-96ce-44b0-84bc-2a2aa73a9e78\") " pod="openshift-cluster-node-tuning-operator/tuned-f9jxc" Jul 06 19:09:37.072232 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.072043 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/02f3dffe-4bee-4c6d-9a45-7d1672d59adc-env-overrides\") pod \"ovnkube-node-4g4j2\" (UID: \"02f3dffe-4bee-4c6d-9a45-7d1672d59adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-4g4j2" Jul 06 19:09:37.072232 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.072104 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/51b731db-0d7f-4a6d-bd3b-30d452687c30-host-var-lib-cni-bin\") pod \"multus-s84ln\" (UID: \"51b731db-0d7f-4a6d-bd3b-30d452687c30\") " pod="openshift-multus/multus-s84ln" Jul 06 19:09:37.072232 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.072178 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xmm8n\" (UniqueName: \"kubernetes.io/projected/51b731db-0d7f-4a6d-bd3b-30d452687c30-kube-api-access-xmm8n\") pod \"multus-s84ln\" (UID: \"51b731db-0d7f-4a6d-bd3b-30d452687c30\") " pod="openshift-multus/multus-s84ln" Jul 06 19:09:37.072232 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.072195 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e9a3a524-5e1b-48db-ba03-750883088470-tuning-conf-dir\") pod \"multus-additional-cni-plugins-65k2z\" (UID: \"e9a3a524-5e1b-48db-ba03-750883088470\") " pod="openshift-multus/multus-additional-cni-plugins-65k2z" Jul 06 19:09:37.072232 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.072227 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/02f3dffe-4bee-4c6d-9a45-7d1672d59adc-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4g4j2\" (UID: \"02f3dffe-4bee-4c6d-9a45-7d1672d59adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-4g4j2" Jul 06 19:09:37.072386 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.072249 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6b947af4-1d82-43fd-9137-73d083aed449-socket-dir\") pod \"aws-ebs-csi-driver-node-vjjbn\" (UID: \"6b947af4-1d82-43fd-9137-73d083aed449\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vjjbn" Jul 06 19:09:37.072386 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.072265 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8wvhb\" (UniqueName: \"kubernetes.io/projected/6b76700e-1870-40a7-a1f2-ab3c1332058c-kube-api-access-8wvhb\") pod \"network-metrics-daemon-8mnph\" (UID: \"6b76700e-1870-40a7-a1f2-ab3c1332058c\") " pod="openshift-multus/network-metrics-daemon-8mnph" Jul 06 19:09:37.072386 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.072280 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/12b34f02-96ce-44b0-84bc-2a2aa73a9e78-etc-sysconfig\") pod \"tuned-f9jxc\" (UID: \"12b34f02-96ce-44b0-84bc-2a2aa73a9e78\") " pod="openshift-cluster-node-tuning-operator/tuned-f9jxc" Jul 06 19:09:37.072386 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.072295 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/12b34f02-96ce-44b0-84bc-2a2aa73a9e78-etc-sysctl-conf\") pod \"tuned-f9jxc\" (UID: \"12b34f02-96ce-44b0-84bc-2a2aa73a9e78\") " pod="openshift-cluster-node-tuning-operator/tuned-f9jxc" Jul 06 19:09:37.072386 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.072313 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/12b34f02-96ce-44b0-84bc-2a2aa73a9e78-lib-modules\") pod \"tuned-f9jxc\" (UID: \"12b34f02-96ce-44b0-84bc-2a2aa73a9e78\") " pod="openshift-cluster-node-tuning-operator/tuned-f9jxc" Jul 06 19:09:37.072386 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.072334 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/12b34f02-96ce-44b0-84bc-2a2aa73a9e78-host\") pod \"tuned-f9jxc\" (UID: \"12b34f02-96ce-44b0-84bc-2a2aa73a9e78\") " pod="openshift-cluster-node-tuning-operator/tuned-f9jxc" Jul 06 19:09:37.072386 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.072358 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/02f3dffe-4bee-4c6d-9a45-7d1672d59adc-ovnkube-script-lib\") pod \"ovnkube-node-4g4j2\" (UID: \"02f3dffe-4bee-4c6d-9a45-7d1672d59adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-4g4j2" Jul 06 19:09:37.072386 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.072374 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6b947af4-1d82-43fd-9137-73d083aed449-kubelet-dir\") pod \"aws-ebs-csi-driver-node-vjjbn\" (UID: \"6b947af4-1d82-43fd-9137-73d083aed449\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vjjbn" Jul 06 19:09:37.072617 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.072408 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/12b34f02-96ce-44b0-84bc-2a2aa73a9e78-etc-sysctl-d\") pod \"tuned-f9jxc\" (UID: \"12b34f02-96ce-44b0-84bc-2a2aa73a9e78\") " pod="openshift-cluster-node-tuning-operator/tuned-f9jxc" Jul 06 19:09:37.072617 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.072455 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/12b34f02-96ce-44b0-84bc-2a2aa73a9e78-etc-tuned\") pod \"tuned-f9jxc\" (UID: \"12b34f02-96ce-44b0-84bc-2a2aa73a9e78\") " pod="openshift-cluster-node-tuning-operator/tuned-f9jxc" Jul 06 19:09:37.072617 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.072481 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/02f3dffe-4bee-4c6d-9a45-7d1672d59adc-host-slash\") pod \"ovnkube-node-4g4j2\" (UID: \"02f3dffe-4bee-4c6d-9a45-7d1672d59adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-4g4j2" Jul 06 19:09:37.072617 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.072505 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/51b731db-0d7f-4a6d-bd3b-30d452687c30-cnibin\") pod \"multus-s84ln\" (UID: \"51b731db-0d7f-4a6d-bd3b-30d452687c30\") " pod="openshift-multus/multus-s84ln" Jul 06 19:09:37.072617 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.072530 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/51b731db-0d7f-4a6d-bd3b-30d452687c30-host-var-lib-cni-multus\") pod \"multus-s84ln\" (UID: \"51b731db-0d7f-4a6d-bd3b-30d452687c30\") " pod="openshift-multus/multus-s84ln" Jul 06 19:09:37.072617 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.072554 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/12b34f02-96ce-44b0-84bc-2a2aa73a9e78-var-lib-kubelet\") pod \"tuned-f9jxc\" (UID: \"12b34f02-96ce-44b0-84bc-2a2aa73a9e78\") " pod="openshift-cluster-node-tuning-operator/tuned-f9jxc" Jul 06 19:09:37.072617 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.072581 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f1803bd2-22ae-4026-b66d-320e1c122a10-iptables-alerter-script\") pod \"iptables-alerter-mznhh\" (UID: \"f1803bd2-22ae-4026-b66d-320e1c122a10\") " pod="openshift-network-operator/iptables-alerter-mznhh" Jul 06 19:09:37.072617 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.072607 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6b76700e-1870-40a7-a1f2-ab3c1332058c-metrics-certs\") pod \"network-metrics-daemon-8mnph\" (UID: \"6b76700e-1870-40a7-a1f2-ab3c1332058c\") " pod="openshift-multus/network-metrics-daemon-8mnph" Jul 06 19:09:37.072936 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.072630 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/02f3dffe-4bee-4c6d-9a45-7d1672d59adc-host-kubelet\") pod \"ovnkube-node-4g4j2\" (UID: \"02f3dffe-4bee-4c6d-9a45-7d1672d59adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-4g4j2" Jul 06 19:09:37.072936 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.072652 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/51b731db-0d7f-4a6d-bd3b-30d452687c30-multus-cni-dir\") pod \"multus-s84ln\" (UID: \"51b731db-0d7f-4a6d-bd3b-30d452687c30\") " pod="openshift-multus/multus-s84ln" Jul 06 19:09:37.072936 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.072696 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/51b731db-0d7f-4a6d-bd3b-30d452687c30-host-run-multus-certs\") pod \"multus-s84ln\" (UID: \"51b731db-0d7f-4a6d-bd3b-30d452687c30\") " pod="openshift-multus/multus-s84ln" Jul 06 19:09:37.072936 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.072743 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e9a3a524-5e1b-48db-ba03-750883088470-system-cni-dir\") pod \"multus-additional-cni-plugins-65k2z\" (UID: \"e9a3a524-5e1b-48db-ba03-750883088470\") " pod="openshift-multus/multus-additional-cni-plugins-65k2z" Jul 06 19:09:37.072936 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.072775 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ds78w\" (UniqueName: \"kubernetes.io/projected/e9a3a524-5e1b-48db-ba03-750883088470-kube-api-access-ds78w\") pod \"multus-additional-cni-plugins-65k2z\" (UID: \"e9a3a524-5e1b-48db-ba03-750883088470\") " pod="openshift-multus/multus-additional-cni-plugins-65k2z" Jul 06 19:09:37.072936 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.072813 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/12b34f02-96ce-44b0-84bc-2a2aa73a9e78-etc-modprobe-d\") pod \"tuned-f9jxc\" (UID: \"12b34f02-96ce-44b0-84bc-2a2aa73a9e78\") " pod="openshift-cluster-node-tuning-operator/tuned-f9jxc" Jul 06 19:09:37.072936 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.072829 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/02f3dffe-4bee-4c6d-9a45-7d1672d59adc-run-ovn\") pod \"ovnkube-node-4g4j2\" (UID: \"02f3dffe-4bee-4c6d-9a45-7d1672d59adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-4g4j2" Jul 06 19:09:37.072936 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.072843 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/02f3dffe-4bee-4c6d-9a45-7d1672d59adc-ovn-node-metrics-cert\") pod \"ovnkube-node-4g4j2\" (UID: \"02f3dffe-4bee-4c6d-9a45-7d1672d59adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-4g4j2" Jul 06 19:09:37.072936 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.072857 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/51b731db-0d7f-4a6d-bd3b-30d452687c30-host-run-k8s-cni-cncf-io\") pod \"multus-s84ln\" (UID: \"51b731db-0d7f-4a6d-bd3b-30d452687c30\") " pod="openshift-multus/multus-s84ln" Jul 06 19:09:37.072936 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.072871 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/51b731db-0d7f-4a6d-bd3b-30d452687c30-hostroot\") pod \"multus-s84ln\" (UID: \"51b731db-0d7f-4a6d-bd3b-30d452687c30\") " pod="openshift-multus/multus-s84ln" Jul 06 19:09:37.072936 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.072885 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e9a3a524-5e1b-48db-ba03-750883088470-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-65k2z\" (UID: \"e9a3a524-5e1b-48db-ba03-750883088470\") " pod="openshift-multus/multus-additional-cni-plugins-65k2z" Jul 06 19:09:37.072936 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.072917 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/12b34f02-96ce-44b0-84bc-2a2aa73a9e78-sys\") pod \"tuned-f9jxc\" (UID: \"12b34f02-96ce-44b0-84bc-2a2aa73a9e78\") " pod="openshift-cluster-node-tuning-operator/tuned-f9jxc" Jul 06 19:09:37.073451 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.072951 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-865s6\" (UniqueName: \"kubernetes.io/projected/c75820ce-6423-4c3f-8e13-0ab0e5b00914-kube-api-access-865s6\") pod \"node-resolver-8btbx\" (UID: \"c75820ce-6423-4c3f-8e13-0ab0e5b00914\") " pod="openshift-dns/node-resolver-8btbx" Jul 06 19:09:37.073451 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.072974 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e9a3a524-5e1b-48db-ba03-750883088470-cnibin\") pod \"multus-additional-cni-plugins-65k2z\" (UID: \"e9a3a524-5e1b-48db-ba03-750883088470\") " pod="openshift-multus/multus-additional-cni-plugins-65k2z" Jul 06 19:09:37.073451 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.073006 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6b947af4-1d82-43fd-9137-73d083aed449-etc-selinux\") pod \"aws-ebs-csi-driver-node-vjjbn\" (UID: \"6b947af4-1d82-43fd-9137-73d083aed449\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vjjbn" Jul 06 19:09:37.073451 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.073029 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/12b34f02-96ce-44b0-84bc-2a2aa73a9e78-etc-systemd\") pod \"tuned-f9jxc\" (UID: \"12b34f02-96ce-44b0-84bc-2a2aa73a9e78\") " pod="openshift-cluster-node-tuning-operator/tuned-f9jxc" Jul 06 19:09:37.073451 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.073053 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/51b731db-0d7f-4a6d-bd3b-30d452687c30-system-cni-dir\") pod \"multus-s84ln\" (UID: \"51b731db-0d7f-4a6d-bd3b-30d452687c30\") " pod="openshift-multus/multus-s84ln" Jul 06 19:09:37.073451 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.073097 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/51b731db-0d7f-4a6d-bd3b-30d452687c30-host-run-netns\") pod \"multus-s84ln\" (UID: \"51b731db-0d7f-4a6d-bd3b-30d452687c30\") " pod="openshift-multus/multus-s84ln" Jul 06 19:09:37.073451 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.073110 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/12b34f02-96ce-44b0-84bc-2a2aa73a9e78-run\") pod \"tuned-f9jxc\" (UID: \"12b34f02-96ce-44b0-84bc-2a2aa73a9e78\") " pod="openshift-cluster-node-tuning-operator/tuned-f9jxc" Jul 06 19:09:37.073451 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.073122 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/02f3dffe-4bee-4c6d-9a45-7d1672d59adc-host-run-netns\") pod \"ovnkube-node-4g4j2\" (UID: \"02f3dffe-4bee-4c6d-9a45-7d1672d59adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-4g4j2" Jul 06 19:09:37.073451 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.073142 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/02f3dffe-4bee-4c6d-9a45-7d1672d59adc-host-run-ovn-kubernetes\") pod \"ovnkube-node-4g4j2\" (UID: \"02f3dffe-4bee-4c6d-9a45-7d1672d59adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-4g4j2" Jul 06 19:09:37.073451 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.073179 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/51b731db-0d7f-4a6d-bd3b-30d452687c30-multus-conf-dir\") pod \"multus-s84ln\" (UID: \"51b731db-0d7f-4a6d-bd3b-30d452687c30\") " pod="openshift-multus/multus-s84ln" Jul 06 19:09:37.073451 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.073198 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e9a3a524-5e1b-48db-ba03-750883088470-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-65k2z\" (UID: \"e9a3a524-5e1b-48db-ba03-750883088470\") " pod="openshift-multus/multus-additional-cni-plugins-65k2z" Jul 06 19:09:37.073451 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.073213 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/02f3dffe-4bee-4c6d-9a45-7d1672d59adc-ovnkube-config\") pod \"ovnkube-node-4g4j2\" (UID: \"02f3dffe-4bee-4c6d-9a45-7d1672d59adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-4g4j2" Jul 06 19:09:37.073451 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.073234 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6b947af4-1d82-43fd-9137-73d083aed449-registration-dir\") pod \"aws-ebs-csi-driver-node-vjjbn\" (UID: \"6b947af4-1d82-43fd-9137-73d083aed449\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vjjbn" Jul 06 19:09:37.073451 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.073256 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-88wjg\" (UniqueName: \"kubernetes.io/projected/6b947af4-1d82-43fd-9137-73d083aed449-kube-api-access-88wjg\") pod \"aws-ebs-csi-driver-node-vjjbn\" (UID: \"6b947af4-1d82-43fd-9137-73d083aed449\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vjjbn" Jul 06 19:09:37.073451 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.073271 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e37e30a5-e12f-4c79-a1ff-28c9460d44b3-host\") pod \"node-ca-qbwhx\" (UID: \"e37e30a5-e12f-4c79-a1ff-28c9460d44b3\") " pod="openshift-image-registry/node-ca-qbwhx" Jul 06 19:09:37.073451 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.073287 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/02f3dffe-4bee-4c6d-9a45-7d1672d59adc-log-socket\") pod \"ovnkube-node-4g4j2\" (UID: \"02f3dffe-4bee-4c6d-9a45-7d1672d59adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-4g4j2" Jul 06 19:09:37.074057 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.073327 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tkmtq\" (UniqueName: \"kubernetes.io/projected/02f3dffe-4bee-4c6d-9a45-7d1672d59adc-kube-api-access-tkmtq\") pod \"ovnkube-node-4g4j2\" (UID: \"02f3dffe-4bee-4c6d-9a45-7d1672d59adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-4g4j2" Jul 06 19:09:37.074057 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.073366 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rg6fx\" (UniqueName: \"kubernetes.io/projected/12b34f02-96ce-44b0-84bc-2a2aa73a9e78-kube-api-access-rg6fx\") pod \"tuned-f9jxc\" (UID: \"12b34f02-96ce-44b0-84bc-2a2aa73a9e78\") " pod="openshift-cluster-node-tuning-operator/tuned-f9jxc" Jul 06 19:09:37.074057 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.073394 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/02f3dffe-4bee-4c6d-9a45-7d1672d59adc-host-cni-bin\") pod \"ovnkube-node-4g4j2\" (UID: \"02f3dffe-4bee-4c6d-9a45-7d1672d59adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-4g4j2" Jul 06 19:09:37.074057 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.073417 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/02f3dffe-4bee-4c6d-9a45-7d1672d59adc-run-openvswitch\") pod \"ovnkube-node-4g4j2\" (UID: \"02f3dffe-4bee-4c6d-9a45-7d1672d59adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-4g4j2" Jul 06 19:09:37.074057 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.073440 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/02f3dffe-4bee-4c6d-9a45-7d1672d59adc-node-log\") pod \"ovnkube-node-4g4j2\" (UID: \"02f3dffe-4bee-4c6d-9a45-7d1672d59adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-4g4j2" Jul 06 19:09:37.074057 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.073460 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/02f3dffe-4bee-4c6d-9a45-7d1672d59adc-host-cni-netd\") pod \"ovnkube-node-4g4j2\" (UID: \"02f3dffe-4bee-4c6d-9a45-7d1672d59adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-4g4j2" Jul 06 19:09:37.074057 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.073484 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6b947af4-1d82-43fd-9137-73d083aed449-sys-fs\") pod \"aws-ebs-csi-driver-node-vjjbn\" (UID: \"6b947af4-1d82-43fd-9137-73d083aed449\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vjjbn" Jul 06 19:09:37.074057 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.073507 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/fa566ec3-6b97-4393-9744-5a9f7e8f0614-konnectivity-ca\") pod \"konnectivity-agent-fjdvf\" (UID: \"fa566ec3-6b97-4393-9744-5a9f7e8f0614\") " pod="kube-system/konnectivity-agent-fjdvf" Jul 06 19:09:37.074057 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.073534 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8sdmq\" (UniqueName: \"kubernetes.io/projected/2f2852c7-f9ef-445c-aa57-e5c697cb2c4b-kube-api-access-8sdmq\") pod \"network-check-target-f9z4j\" (UID: \"2f2852c7-f9ef-445c-aa57-e5c697cb2c4b\") " pod="openshift-network-diagnostics/network-check-target-f9z4j" Jul 06 19:09:37.074057 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.073556 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/02f3dffe-4bee-4c6d-9a45-7d1672d59adc-systemd-units\") pod \"ovnkube-node-4g4j2\" (UID: \"02f3dffe-4bee-4c6d-9a45-7d1672d59adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-4g4j2" Jul 06 19:09:37.074057 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.073577 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f1803bd2-22ae-4026-b66d-320e1c122a10-host-slash\") pod \"iptables-alerter-mznhh\" (UID: \"f1803bd2-22ae-4026-b66d-320e1c122a10\") " pod="openshift-network-operator/iptables-alerter-mznhh" Jul 06 19:09:37.074057 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.073604 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/02f3dffe-4bee-4c6d-9a45-7d1672d59adc-var-lib-openvswitch\") pod \"ovnkube-node-4g4j2\" (UID: \"02f3dffe-4bee-4c6d-9a45-7d1672d59adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-4g4j2" Jul 06 19:09:37.074057 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.073629 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/02f3dffe-4bee-4c6d-9a45-7d1672d59adc-etc-openvswitch\") pod \"ovnkube-node-4g4j2\" (UID: \"02f3dffe-4bee-4c6d-9a45-7d1672d59adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-4g4j2" Jul 06 19:09:37.074057 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.073651 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/51b731db-0d7f-4a6d-bd3b-30d452687c30-multus-socket-dir-parent\") pod \"multus-s84ln\" (UID: \"51b731db-0d7f-4a6d-bd3b-30d452687c30\") " pod="openshift-multus/multus-s84ln" Jul 06 19:09:37.074057 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.073681 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/51b731db-0d7f-4a6d-bd3b-30d452687c30-host-var-lib-kubelet\") pod \"multus-s84ln\" (UID: \"51b731db-0d7f-4a6d-bd3b-30d452687c30\") " pod="openshift-multus/multus-s84ln" Jul 06 19:09:37.074057 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.073716 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e9a3a524-5e1b-48db-ba03-750883088470-cni-binary-copy\") pod \"multus-additional-cni-plugins-65k2z\" (UID: \"e9a3a524-5e1b-48db-ba03-750883088470\") " pod="openshift-multus/multus-additional-cni-plugins-65k2z" Jul 06 19:09:37.074740 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.073739 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c75820ce-6423-4c3f-8e13-0ab0e5b00914-tmp-dir\") pod \"node-resolver-8btbx\" (UID: \"c75820ce-6423-4c3f-8e13-0ab0e5b00914\") " pod="openshift-dns/node-resolver-8btbx" Jul 06 19:09:37.074740 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.073802 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/51b731db-0d7f-4a6d-bd3b-30d452687c30-os-release\") pod \"multus-s84ln\" (UID: \"51b731db-0d7f-4a6d-bd3b-30d452687c30\") " pod="openshift-multus/multus-s84ln" Jul 06 19:09:37.074740 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.073838 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/51b731db-0d7f-4a6d-bd3b-30d452687c30-etc-kubernetes\") pod \"multus-s84ln\" (UID: \"51b731db-0d7f-4a6d-bd3b-30d452687c30\") " pod="openshift-multus/multus-s84ln" Jul 06 19:09:37.074740 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.073868 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e9a3a524-5e1b-48db-ba03-750883088470-os-release\") pod \"multus-additional-cni-plugins-65k2z\" (UID: \"e9a3a524-5e1b-48db-ba03-750883088470\") " pod="openshift-multus/multus-additional-cni-plugins-65k2z" Jul 06 19:09:37.074740 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.073892 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/12b34f02-96ce-44b0-84bc-2a2aa73a9e78-etc-kubernetes\") pod \"tuned-f9jxc\" (UID: \"12b34f02-96ce-44b0-84bc-2a2aa73a9e78\") " pod="openshift-cluster-node-tuning-operator/tuned-f9jxc" Jul 06 19:09:37.074740 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.073915 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e37e30a5-e12f-4c79-a1ff-28c9460d44b3-serviceca\") pod \"node-ca-qbwhx\" (UID: \"e37e30a5-e12f-4c79-a1ff-28c9460d44b3\") " pod="openshift-image-registry/node-ca-qbwhx" Jul 06 19:09:37.074740 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.073933 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d87n7\" (UniqueName: \"kubernetes.io/projected/e37e30a5-e12f-4c79-a1ff-28c9460d44b3-kube-api-access-d87n7\") pod \"node-ca-qbwhx\" (UID: \"e37e30a5-e12f-4c79-a1ff-28c9460d44b3\") " pod="openshift-image-registry/node-ca-qbwhx" Jul 06 19:09:37.074740 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.073950 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/02f3dffe-4bee-4c6d-9a45-7d1672d59adc-run-systemd\") pod \"ovnkube-node-4g4j2\" (UID: \"02f3dffe-4bee-4c6d-9a45-7d1672d59adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-4g4j2" Jul 06 19:09:37.089492 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.089454 2566 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 06 19:09:37.119328 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.119297 2566 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-2r8jb" Jul 06 19:09:37.128732 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.128695 2566 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-2r8jb" Jul 06 19:09:37.160824 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.160793 2566 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 06 19:09:37.175043 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.175007 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/12b34f02-96ce-44b0-84bc-2a2aa73a9e78-run\") pod \"tuned-f9jxc\" (UID: \"12b34f02-96ce-44b0-84bc-2a2aa73a9e78\") " pod="openshift-cluster-node-tuning-operator/tuned-f9jxc" Jul 06 19:09:37.175251 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.175051 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/02f3dffe-4bee-4c6d-9a45-7d1672d59adc-host-run-netns\") pod \"ovnkube-node-4g4j2\" (UID: \"02f3dffe-4bee-4c6d-9a45-7d1672d59adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-4g4j2" Jul 06 19:09:37.175251 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.175143 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/02f3dffe-4bee-4c6d-9a45-7d1672d59adc-host-run-ovn-kubernetes\") pod \"ovnkube-node-4g4j2\" (UID: \"02f3dffe-4bee-4c6d-9a45-7d1672d59adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-4g4j2" Jul 06 19:09:37.175251 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.175157 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/12b34f02-96ce-44b0-84bc-2a2aa73a9e78-run\") pod \"tuned-f9jxc\" (UID: \"12b34f02-96ce-44b0-84bc-2a2aa73a9e78\") " pod="openshift-cluster-node-tuning-operator/tuned-f9jxc" Jul 06 19:09:37.175251 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.175187 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/51b731db-0d7f-4a6d-bd3b-30d452687c30-multus-conf-dir\") pod \"multus-s84ln\" (UID: \"51b731db-0d7f-4a6d-bd3b-30d452687c30\") " pod="openshift-multus/multus-s84ln" Jul 06 19:09:37.175251 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.175149 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/02f3dffe-4bee-4c6d-9a45-7d1672d59adc-host-run-netns\") pod \"ovnkube-node-4g4j2\" (UID: \"02f3dffe-4bee-4c6d-9a45-7d1672d59adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-4g4j2" Jul 06 19:09:37.175251 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.175207 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/02f3dffe-4bee-4c6d-9a45-7d1672d59adc-host-run-ovn-kubernetes\") pod \"ovnkube-node-4g4j2\" (UID: \"02f3dffe-4bee-4c6d-9a45-7d1672d59adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-4g4j2" Jul 06 19:09:37.175251 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.175226 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e9a3a524-5e1b-48db-ba03-750883088470-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-65k2z\" (UID: \"e9a3a524-5e1b-48db-ba03-750883088470\") " pod="openshift-multus/multus-additional-cni-plugins-65k2z" Jul 06 19:09:37.175251 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.175236 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/51b731db-0d7f-4a6d-bd3b-30d452687c30-multus-conf-dir\") pod \"multus-s84ln\" (UID: \"51b731db-0d7f-4a6d-bd3b-30d452687c30\") " pod="openshift-multus/multus-s84ln" Jul 06 19:09:37.175617 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.175256 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/02f3dffe-4bee-4c6d-9a45-7d1672d59adc-ovnkube-config\") pod \"ovnkube-node-4g4j2\" (UID: \"02f3dffe-4bee-4c6d-9a45-7d1672d59adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-4g4j2" Jul 06 19:09:37.175617 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.175281 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6b947af4-1d82-43fd-9137-73d083aed449-registration-dir\") pod \"aws-ebs-csi-driver-node-vjjbn\" (UID: \"6b947af4-1d82-43fd-9137-73d083aed449\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vjjbn" Jul 06 19:09:37.175617 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.175306 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-88wjg\" (UniqueName: \"kubernetes.io/projected/6b947af4-1d82-43fd-9137-73d083aed449-kube-api-access-88wjg\") pod \"aws-ebs-csi-driver-node-vjjbn\" (UID: \"6b947af4-1d82-43fd-9137-73d083aed449\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vjjbn" Jul 06 19:09:37.175617 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.175328 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e37e30a5-e12f-4c79-a1ff-28c9460d44b3-host\") pod \"node-ca-qbwhx\" (UID: \"e37e30a5-e12f-4c79-a1ff-28c9460d44b3\") " pod="openshift-image-registry/node-ca-qbwhx" Jul 06 19:09:37.175617 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.175352 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/02f3dffe-4bee-4c6d-9a45-7d1672d59adc-log-socket\") pod \"ovnkube-node-4g4j2\" (UID: \"02f3dffe-4bee-4c6d-9a45-7d1672d59adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-4g4j2" Jul 06 19:09:37.175617 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.175375 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6b947af4-1d82-43fd-9137-73d083aed449-registration-dir\") pod \"aws-ebs-csi-driver-node-vjjbn\" (UID: \"6b947af4-1d82-43fd-9137-73d083aed449\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vjjbn" Jul 06 19:09:37.175617 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.175420 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e37e30a5-e12f-4c79-a1ff-28c9460d44b3-host\") pod \"node-ca-qbwhx\" (UID: \"e37e30a5-e12f-4c79-a1ff-28c9460d44b3\") " pod="openshift-image-registry/node-ca-qbwhx" Jul 06 19:09:37.175617 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.175455 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/02f3dffe-4bee-4c6d-9a45-7d1672d59adc-log-socket\") pod \"ovnkube-node-4g4j2\" (UID: \"02f3dffe-4bee-4c6d-9a45-7d1672d59adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-4g4j2" Jul 06 19:09:37.175617 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.175375 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tkmtq\" (UniqueName: \"kubernetes.io/projected/02f3dffe-4bee-4c6d-9a45-7d1672d59adc-kube-api-access-tkmtq\") pod \"ovnkube-node-4g4j2\" (UID: \"02f3dffe-4bee-4c6d-9a45-7d1672d59adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-4g4j2" Jul 06 19:09:37.175617 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.175486 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rg6fx\" (UniqueName: \"kubernetes.io/projected/12b34f02-96ce-44b0-84bc-2a2aa73a9e78-kube-api-access-rg6fx\") pod \"tuned-f9jxc\" (UID: \"12b34f02-96ce-44b0-84bc-2a2aa73a9e78\") " pod="openshift-cluster-node-tuning-operator/tuned-f9jxc" Jul 06 19:09:37.175617 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.175507 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/02f3dffe-4bee-4c6d-9a45-7d1672d59adc-host-cni-bin\") pod \"ovnkube-node-4g4j2\" (UID: \"02f3dffe-4bee-4c6d-9a45-7d1672d59adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-4g4j2" Jul 06 19:09:37.175617 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.175531 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/02f3dffe-4bee-4c6d-9a45-7d1672d59adc-run-openvswitch\") pod \"ovnkube-node-4g4j2\" (UID: \"02f3dffe-4bee-4c6d-9a45-7d1672d59adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-4g4j2" Jul 06 19:09:37.175617 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.175556 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/02f3dffe-4bee-4c6d-9a45-7d1672d59adc-node-log\") pod \"ovnkube-node-4g4j2\" (UID: \"02f3dffe-4bee-4c6d-9a45-7d1672d59adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-4g4j2" Jul 06 19:09:37.175617 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.175584 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/02f3dffe-4bee-4c6d-9a45-7d1672d59adc-host-cni-netd\") pod \"ovnkube-node-4g4j2\" (UID: \"02f3dffe-4bee-4c6d-9a45-7d1672d59adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-4g4j2" Jul 06 19:09:37.175617 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.175609 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6b947af4-1d82-43fd-9137-73d083aed449-sys-fs\") pod \"aws-ebs-csi-driver-node-vjjbn\" (UID: \"6b947af4-1d82-43fd-9137-73d083aed449\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vjjbn" Jul 06 19:09:37.176397 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.175632 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/fa566ec3-6b97-4393-9744-5a9f7e8f0614-konnectivity-ca\") pod \"konnectivity-agent-fjdvf\" (UID: \"fa566ec3-6b97-4393-9744-5a9f7e8f0614\") " pod="kube-system/konnectivity-agent-fjdvf" Jul 06 19:09:37.176397 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.175659 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8sdmq\" (UniqueName: \"kubernetes.io/projected/2f2852c7-f9ef-445c-aa57-e5c697cb2c4b-kube-api-access-8sdmq\") pod \"network-check-target-f9z4j\" (UID: \"2f2852c7-f9ef-445c-aa57-e5c697cb2c4b\") " pod="openshift-network-diagnostics/network-check-target-f9z4j" Jul 06 19:09:37.176397 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.175685 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/02f3dffe-4bee-4c6d-9a45-7d1672d59adc-systemd-units\") pod \"ovnkube-node-4g4j2\" (UID: \"02f3dffe-4bee-4c6d-9a45-7d1672d59adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-4g4j2" Jul 06 19:09:37.176397 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.175707 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f1803bd2-22ae-4026-b66d-320e1c122a10-host-slash\") pod \"iptables-alerter-mznhh\" (UID: \"f1803bd2-22ae-4026-b66d-320e1c122a10\") " pod="openshift-network-operator/iptables-alerter-mznhh" Jul 06 19:09:37.176397 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.175737 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/02f3dffe-4bee-4c6d-9a45-7d1672d59adc-var-lib-openvswitch\") pod \"ovnkube-node-4g4j2\" (UID: \"02f3dffe-4bee-4c6d-9a45-7d1672d59adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-4g4j2" Jul 06 19:09:37.176397 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.175770 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/02f3dffe-4bee-4c6d-9a45-7d1672d59adc-etc-openvswitch\") pod \"ovnkube-node-4g4j2\" (UID: \"02f3dffe-4bee-4c6d-9a45-7d1672d59adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-4g4j2" Jul 06 19:09:37.176397 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.175795 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/51b731db-0d7f-4a6d-bd3b-30d452687c30-multus-socket-dir-parent\") pod \"multus-s84ln\" (UID: \"51b731db-0d7f-4a6d-bd3b-30d452687c30\") " pod="openshift-multus/multus-s84ln" Jul 06 19:09:37.176397 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.175824 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/51b731db-0d7f-4a6d-bd3b-30d452687c30-host-var-lib-kubelet\") pod \"multus-s84ln\" (UID: \"51b731db-0d7f-4a6d-bd3b-30d452687c30\") " pod="openshift-multus/multus-s84ln" Jul 06 19:09:37.176397 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.175848 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e9a3a524-5e1b-48db-ba03-750883088470-cni-binary-copy\") pod \"multus-additional-cni-plugins-65k2z\" (UID: \"e9a3a524-5e1b-48db-ba03-750883088470\") " pod="openshift-multus/multus-additional-cni-plugins-65k2z" Jul 06 19:09:37.176397 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.175871 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c75820ce-6423-4c3f-8e13-0ab0e5b00914-tmp-dir\") pod \"node-resolver-8btbx\" (UID: \"c75820ce-6423-4c3f-8e13-0ab0e5b00914\") " pod="openshift-dns/node-resolver-8btbx" Jul 06 19:09:37.176397 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.175894 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/51b731db-0d7f-4a6d-bd3b-30d452687c30-os-release\") pod \"multus-s84ln\" (UID: \"51b731db-0d7f-4a6d-bd3b-30d452687c30\") " pod="openshift-multus/multus-s84ln" Jul 06 19:09:37.176397 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.175916 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/51b731db-0d7f-4a6d-bd3b-30d452687c30-etc-kubernetes\") pod \"multus-s84ln\" (UID: \"51b731db-0d7f-4a6d-bd3b-30d452687c30\") " pod="openshift-multus/multus-s84ln" Jul 06 19:09:37.176397 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.175943 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/02f3dffe-4bee-4c6d-9a45-7d1672d59adc-systemd-units\") pod \"ovnkube-node-4g4j2\" (UID: \"02f3dffe-4bee-4c6d-9a45-7d1672d59adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-4g4j2" Jul 06 19:09:37.176397 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.175946 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e9a3a524-5e1b-48db-ba03-750883088470-os-release\") pod \"multus-additional-cni-plugins-65k2z\" (UID: \"e9a3a524-5e1b-48db-ba03-750883088470\") " pod="openshift-multus/multus-additional-cni-plugins-65k2z" Jul 06 19:09:37.176397 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.175984 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/02f3dffe-4bee-4c6d-9a45-7d1672d59adc-host-cni-bin\") pod \"ovnkube-node-4g4j2\" (UID: \"02f3dffe-4bee-4c6d-9a45-7d1672d59adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-4g4j2" Jul 06 19:09:37.176397 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.176009 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/02f3dffe-4bee-4c6d-9a45-7d1672d59adc-run-openvswitch\") pod \"ovnkube-node-4g4j2\" (UID: \"02f3dffe-4bee-4c6d-9a45-7d1672d59adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-4g4j2" Jul 06 19:09:37.176397 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.176019 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/e9a3a524-5e1b-48db-ba03-750883088470-os-release\") pod \"multus-additional-cni-plugins-65k2z\" (UID: \"e9a3a524-5e1b-48db-ba03-750883088470\") " pod="openshift-multus/multus-additional-cni-plugins-65k2z" Jul 06 19:09:37.177228 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.176033 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/02f3dffe-4bee-4c6d-9a45-7d1672d59adc-node-log\") pod \"ovnkube-node-4g4j2\" (UID: \"02f3dffe-4bee-4c6d-9a45-7d1672d59adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-4g4j2" Jul 06 19:09:37.177228 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.176019 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/02f3dffe-4bee-4c6d-9a45-7d1672d59adc-ovnkube-config\") pod \"ovnkube-node-4g4j2\" (UID: \"02f3dffe-4bee-4c6d-9a45-7d1672d59adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-4g4j2" Jul 06 19:09:37.177228 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.176057 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/51b731db-0d7f-4a6d-bd3b-30d452687c30-host-var-lib-kubelet\") pod \"multus-s84ln\" (UID: \"51b731db-0d7f-4a6d-bd3b-30d452687c30\") " pod="openshift-multus/multus-s84ln" Jul 06 19:09:37.177228 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.176106 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/02f3dffe-4bee-4c6d-9a45-7d1672d59adc-host-cni-netd\") pod \"ovnkube-node-4g4j2\" (UID: \"02f3dffe-4bee-4c6d-9a45-7d1672d59adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-4g4j2" Jul 06 19:09:37.177228 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.176103 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e9a3a524-5e1b-48db-ba03-750883088470-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-65k2z\" (UID: \"e9a3a524-5e1b-48db-ba03-750883088470\") " pod="openshift-multus/multus-additional-cni-plugins-65k2z" Jul 06 19:09:37.177228 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.176128 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f1803bd2-22ae-4026-b66d-320e1c122a10-host-slash\") pod \"iptables-alerter-mznhh\" (UID: \"f1803bd2-22ae-4026-b66d-320e1c122a10\") " pod="openshift-network-operator/iptables-alerter-mznhh" Jul 06 19:09:37.177228 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.176164 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e8c2f347-1744-49ff-b46e-f036d27a5e46-kubelet-config\") pod \"global-pull-secret-syncer-svgl2\" (UID: \"e8c2f347-1744-49ff-b46e-f036d27a5e46\") " pod="kube-system/global-pull-secret-syncer-svgl2" Jul 06 19:09:37.177228 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.176184 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/51b731db-0d7f-4a6d-bd3b-30d452687c30-etc-kubernetes\") pod \"multus-s84ln\" (UID: \"51b731db-0d7f-4a6d-bd3b-30d452687c30\") " pod="openshift-multus/multus-s84ln" Jul 06 19:09:37.177228 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.176201 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6b947af4-1d82-43fd-9137-73d083aed449-sys-fs\") pod \"aws-ebs-csi-driver-node-vjjbn\" (UID: \"6b947af4-1d82-43fd-9137-73d083aed449\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vjjbn" Jul 06 19:09:37.177228 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.176230 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/02f3dffe-4bee-4c6d-9a45-7d1672d59adc-etc-openvswitch\") pod \"ovnkube-node-4g4j2\" (UID: \"02f3dffe-4bee-4c6d-9a45-7d1672d59adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-4g4j2" Jul 06 19:09:37.177228 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.176289 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/51b731db-0d7f-4a6d-bd3b-30d452687c30-multus-socket-dir-parent\") pod \"multus-s84ln\" (UID: \"51b731db-0d7f-4a6d-bd3b-30d452687c30\") " pod="openshift-multus/multus-s84ln" Jul 06 19:09:37.177228 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.176325 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/12b34f02-96ce-44b0-84bc-2a2aa73a9e78-etc-kubernetes\") pod \"tuned-f9jxc\" (UID: \"12b34f02-96ce-44b0-84bc-2a2aa73a9e78\") " pod="openshift-cluster-node-tuning-operator/tuned-f9jxc" Jul 06 19:09:37.177228 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.176351 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e37e30a5-e12f-4c79-a1ff-28c9460d44b3-serviceca\") pod \"node-ca-qbwhx\" (UID: \"e37e30a5-e12f-4c79-a1ff-28c9460d44b3\") " pod="openshift-image-registry/node-ca-qbwhx" Jul 06 19:09:37.177228 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.176377 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d87n7\" (UniqueName: \"kubernetes.io/projected/e37e30a5-e12f-4c79-a1ff-28c9460d44b3-kube-api-access-d87n7\") pod \"node-ca-qbwhx\" (UID: \"e37e30a5-e12f-4c79-a1ff-28c9460d44b3\") " pod="openshift-image-registry/node-ca-qbwhx" Jul 06 19:09:37.177228 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.176404 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/02f3dffe-4bee-4c6d-9a45-7d1672d59adc-run-systemd\") pod \"ovnkube-node-4g4j2\" (UID: \"02f3dffe-4bee-4c6d-9a45-7d1672d59adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-4g4j2" Jul 06 19:09:37.177228 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.176463 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c75820ce-6423-4c3f-8e13-0ab0e5b00914-hosts-file\") pod \"node-resolver-8btbx\" (UID: \"c75820ce-6423-4c3f-8e13-0ab0e5b00914\") " pod="openshift-dns/node-resolver-8btbx" Jul 06 19:09:37.177228 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.176485 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/02f3dffe-4bee-4c6d-9a45-7d1672d59adc-var-lib-openvswitch\") pod \"ovnkube-node-4g4j2\" (UID: \"02f3dffe-4bee-4c6d-9a45-7d1672d59adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-4g4j2" Jul 06 19:09:37.178049 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.176491 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e8c2f347-1744-49ff-b46e-f036d27a5e46-dbus\") pod \"global-pull-secret-syncer-svgl2\" (UID: \"e8c2f347-1744-49ff-b46e-f036d27a5e46\") " pod="kube-system/global-pull-secret-syncer-svgl2" Jul 06 19:09:37.178049 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.176516 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/51b731db-0d7f-4a6d-bd3b-30d452687c30-cni-binary-copy\") pod \"multus-s84ln\" (UID: \"51b731db-0d7f-4a6d-bd3b-30d452687c30\") " pod="openshift-multus/multus-s84ln" Jul 06 19:09:37.178049 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.176521 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e9a3a524-5e1b-48db-ba03-750883088470-cni-binary-copy\") pod \"multus-additional-cni-plugins-65k2z\" (UID: \"e9a3a524-5e1b-48db-ba03-750883088470\") " pod="openshift-multus/multus-additional-cni-plugins-65k2z" Jul 06 19:09:37.178049 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.176540 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/51b731db-0d7f-4a6d-bd3b-30d452687c30-multus-daemon-config\") pod \"multus-s84ln\" (UID: \"51b731db-0d7f-4a6d-bd3b-30d452687c30\") " pod="openshift-multus/multus-s84ln" Jul 06 19:09:37.178049 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.176564 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h4nf5\" (UniqueName: \"kubernetes.io/projected/f1803bd2-22ae-4026-b66d-320e1c122a10-kube-api-access-h4nf5\") pod \"iptables-alerter-mznhh\" (UID: \"f1803bd2-22ae-4026-b66d-320e1c122a10\") " pod="openshift-network-operator/iptables-alerter-mznhh" Jul 06 19:09:37.178049 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.176635 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6b947af4-1d82-43fd-9137-73d083aed449-device-dir\") pod \"aws-ebs-csi-driver-node-vjjbn\" (UID: \"6b947af4-1d82-43fd-9137-73d083aed449\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vjjbn" Jul 06 19:09:37.178049 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.176659 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/fa566ec3-6b97-4393-9744-5a9f7e8f0614-agent-certs\") pod \"konnectivity-agent-fjdvf\" (UID: \"fa566ec3-6b97-4393-9744-5a9f7e8f0614\") " pod="kube-system/konnectivity-agent-fjdvf" Jul 06 19:09:37.178049 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.176683 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/12b34f02-96ce-44b0-84bc-2a2aa73a9e78-tmp\") pod \"tuned-f9jxc\" (UID: \"12b34f02-96ce-44b0-84bc-2a2aa73a9e78\") " pod="openshift-cluster-node-tuning-operator/tuned-f9jxc" Jul 06 19:09:37.178049 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.176687 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c75820ce-6423-4c3f-8e13-0ab0e5b00914-tmp-dir\") pod \"node-resolver-8btbx\" (UID: \"c75820ce-6423-4c3f-8e13-0ab0e5b00914\") " pod="openshift-dns/node-resolver-8btbx" Jul 06 19:09:37.178049 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.176708 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/fa566ec3-6b97-4393-9744-5a9f7e8f0614-konnectivity-ca\") pod \"konnectivity-agent-fjdvf\" (UID: \"fa566ec3-6b97-4393-9744-5a9f7e8f0614\") " pod="kube-system/konnectivity-agent-fjdvf" Jul 06 19:09:37.178049 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.176708 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/02f3dffe-4bee-4c6d-9a45-7d1672d59adc-env-overrides\") pod \"ovnkube-node-4g4j2\" (UID: \"02f3dffe-4bee-4c6d-9a45-7d1672d59adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-4g4j2" Jul 06 19:09:37.178049 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.176755 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/51b731db-0d7f-4a6d-bd3b-30d452687c30-host-var-lib-cni-bin\") pod \"multus-s84ln\" (UID: \"51b731db-0d7f-4a6d-bd3b-30d452687c30\") " pod="openshift-multus/multus-s84ln" Jul 06 19:09:37.178049 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.176779 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xmm8n\" (UniqueName: \"kubernetes.io/projected/51b731db-0d7f-4a6d-bd3b-30d452687c30-kube-api-access-xmm8n\") pod \"multus-s84ln\" (UID: \"51b731db-0d7f-4a6d-bd3b-30d452687c30\") " pod="openshift-multus/multus-s84ln" Jul 06 19:09:37.178049 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.176789 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/51b731db-0d7f-4a6d-bd3b-30d452687c30-os-release\") pod \"multus-s84ln\" (UID: \"51b731db-0d7f-4a6d-bd3b-30d452687c30\") " pod="openshift-multus/multus-s84ln" Jul 06 19:09:37.178049 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.176805 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e9a3a524-5e1b-48db-ba03-750883088470-tuning-conf-dir\") pod \"multus-additional-cni-plugins-65k2z\" (UID: \"e9a3a524-5e1b-48db-ba03-750883088470\") " pod="openshift-multus/multus-additional-cni-plugins-65k2z" Jul 06 19:09:37.178049 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.176900 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/02f3dffe-4bee-4c6d-9a45-7d1672d59adc-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4g4j2\" (UID: \"02f3dffe-4bee-4c6d-9a45-7d1672d59adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-4g4j2" Jul 06 19:09:37.178049 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.176947 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6b947af4-1d82-43fd-9137-73d083aed449-socket-dir\") pod \"aws-ebs-csi-driver-node-vjjbn\" (UID: \"6b947af4-1d82-43fd-9137-73d083aed449\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vjjbn" Jul 06 19:09:37.178837 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.176961 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/51b731db-0d7f-4a6d-bd3b-30d452687c30-host-var-lib-cni-bin\") pod \"multus-s84ln\" (UID: \"51b731db-0d7f-4a6d-bd3b-30d452687c30\") " pod="openshift-multus/multus-s84ln" Jul 06 19:09:37.178837 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.176975 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8wvhb\" (UniqueName: \"kubernetes.io/projected/6b76700e-1870-40a7-a1f2-ab3c1332058c-kube-api-access-8wvhb\") pod \"network-metrics-daemon-8mnph\" (UID: \"6b76700e-1870-40a7-a1f2-ab3c1332058c\") " pod="openshift-multus/network-metrics-daemon-8mnph" Jul 06 19:09:37.178837 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.177001 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/12b34f02-96ce-44b0-84bc-2a2aa73a9e78-etc-sysconfig\") pod \"tuned-f9jxc\" (UID: \"12b34f02-96ce-44b0-84bc-2a2aa73a9e78\") " pod="openshift-cluster-node-tuning-operator/tuned-f9jxc" Jul 06 19:09:37.178837 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.177012 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6b947af4-1d82-43fd-9137-73d083aed449-device-dir\") pod \"aws-ebs-csi-driver-node-vjjbn\" (UID: \"6b947af4-1d82-43fd-9137-73d083aed449\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vjjbn" Jul 06 19:09:37.178837 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.177038 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/c75820ce-6423-4c3f-8e13-0ab0e5b00914-hosts-file\") pod \"node-resolver-8btbx\" (UID: \"c75820ce-6423-4c3f-8e13-0ab0e5b00914\") " pod="openshift-dns/node-resolver-8btbx" Jul 06 19:09:37.178837 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.177024 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/12b34f02-96ce-44b0-84bc-2a2aa73a9e78-etc-sysctl-conf\") pod \"tuned-f9jxc\" (UID: \"12b34f02-96ce-44b0-84bc-2a2aa73a9e78\") " pod="openshift-cluster-node-tuning-operator/tuned-f9jxc" Jul 06 19:09:37.178837 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.177086 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/12b34f02-96ce-44b0-84bc-2a2aa73a9e78-lib-modules\") pod \"tuned-f9jxc\" (UID: \"12b34f02-96ce-44b0-84bc-2a2aa73a9e78\") " pod="openshift-cluster-node-tuning-operator/tuned-f9jxc" Jul 06 19:09:37.178837 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.177116 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/12b34f02-96ce-44b0-84bc-2a2aa73a9e78-host\") pod \"tuned-f9jxc\" (UID: \"12b34f02-96ce-44b0-84bc-2a2aa73a9e78\") " pod="openshift-cluster-node-tuning-operator/tuned-f9jxc" Jul 06 19:09:37.178837 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.177144 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e8c2f347-1744-49ff-b46e-f036d27a5e46-original-pull-secret\") pod \"global-pull-secret-syncer-svgl2\" (UID: \"e8c2f347-1744-49ff-b46e-f036d27a5e46\") " pod="kube-system/global-pull-secret-syncer-svgl2" Jul 06 19:09:37.178837 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.177160 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e9a3a524-5e1b-48db-ba03-750883088470-tuning-conf-dir\") pod \"multus-additional-cni-plugins-65k2z\" (UID: \"e9a3a524-5e1b-48db-ba03-750883088470\") " pod="openshift-multus/multus-additional-cni-plugins-65k2z" Jul 06 19:09:37.178837 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.177167 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/12b34f02-96ce-44b0-84bc-2a2aa73a9e78-etc-sysctl-conf\") pod \"tuned-f9jxc\" (UID: \"12b34f02-96ce-44b0-84bc-2a2aa73a9e78\") " pod="openshift-cluster-node-tuning-operator/tuned-f9jxc" Jul 06 19:09:37.178837 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.177174 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/02f3dffe-4bee-4c6d-9a45-7d1672d59adc-ovnkube-script-lib\") pod \"ovnkube-node-4g4j2\" (UID: \"02f3dffe-4bee-4c6d-9a45-7d1672d59adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-4g4j2" Jul 06 19:09:37.178837 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.177198 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6b947af4-1d82-43fd-9137-73d083aed449-kubelet-dir\") pod \"aws-ebs-csi-driver-node-vjjbn\" (UID: \"6b947af4-1d82-43fd-9137-73d083aed449\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vjjbn" Jul 06 19:09:37.178837 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.177223 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/12b34f02-96ce-44b0-84bc-2a2aa73a9e78-etc-sysctl-d\") pod \"tuned-f9jxc\" (UID: \"12b34f02-96ce-44b0-84bc-2a2aa73a9e78\") " pod="openshift-cluster-node-tuning-operator/tuned-f9jxc" Jul 06 19:09:37.178837 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.177248 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/12b34f02-96ce-44b0-84bc-2a2aa73a9e78-etc-tuned\") pod \"tuned-f9jxc\" (UID: \"12b34f02-96ce-44b0-84bc-2a2aa73a9e78\") " pod="openshift-cluster-node-tuning-operator/tuned-f9jxc" Jul 06 19:09:37.178837 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.177271 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/02f3dffe-4bee-4c6d-9a45-7d1672d59adc-host-slash\") pod \"ovnkube-node-4g4j2\" (UID: \"02f3dffe-4bee-4c6d-9a45-7d1672d59adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-4g4j2" Jul 06 19:09:37.178837 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.177297 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/51b731db-0d7f-4a6d-bd3b-30d452687c30-cnibin\") pod \"multus-s84ln\" (UID: \"51b731db-0d7f-4a6d-bd3b-30d452687c30\") " pod="openshift-multus/multus-s84ln" Jul 06 19:09:37.179614 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.177322 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/51b731db-0d7f-4a6d-bd3b-30d452687c30-host-var-lib-cni-multus\") pod \"multus-s84ln\" (UID: \"51b731db-0d7f-4a6d-bd3b-30d452687c30\") " pod="openshift-multus/multus-s84ln" Jul 06 19:09:37.179614 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.176919 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e37e30a5-e12f-4c79-a1ff-28c9460d44b3-serviceca\") pod \"node-ca-qbwhx\" (UID: \"e37e30a5-e12f-4c79-a1ff-28c9460d44b3\") " pod="openshift-image-registry/node-ca-qbwhx" Jul 06 19:09:37.179614 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.177341 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/51b731db-0d7f-4a6d-bd3b-30d452687c30-cni-binary-copy\") pod \"multus-s84ln\" (UID: \"51b731db-0d7f-4a6d-bd3b-30d452687c30\") " pod="openshift-multus/multus-s84ln" Jul 06 19:09:37.179614 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.177399 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/02f3dffe-4bee-4c6d-9a45-7d1672d59adc-run-systemd\") pod \"ovnkube-node-4g4j2\" (UID: \"02f3dffe-4bee-4c6d-9a45-7d1672d59adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-4g4j2" Jul 06 19:09:37.179614 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.177461 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/12b34f02-96ce-44b0-84bc-2a2aa73a9e78-etc-kubernetes\") pod \"tuned-f9jxc\" (UID: \"12b34f02-96ce-44b0-84bc-2a2aa73a9e78\") " pod="openshift-cluster-node-tuning-operator/tuned-f9jxc" Jul 06 19:09:37.179614 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.177456 2566 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 06 19:09:37.179614 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.177509 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/12b34f02-96ce-44b0-84bc-2a2aa73a9e78-var-lib-kubelet\") pod \"tuned-f9jxc\" (UID: \"12b34f02-96ce-44b0-84bc-2a2aa73a9e78\") " pod="openshift-cluster-node-tuning-operator/tuned-f9jxc" Jul 06 19:09:37.179614 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.177536 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/02f3dffe-4bee-4c6d-9a45-7d1672d59adc-ovnkube-script-lib\") pod \"ovnkube-node-4g4j2\" (UID: \"02f3dffe-4bee-4c6d-9a45-7d1672d59adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-4g4j2" Jul 06 19:09:37.179614 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.177542 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f1803bd2-22ae-4026-b66d-320e1c122a10-iptables-alerter-script\") pod \"iptables-alerter-mznhh\" (UID: \"f1803bd2-22ae-4026-b66d-320e1c122a10\") " pod="openshift-network-operator/iptables-alerter-mznhh" Jul 06 19:09:37.179614 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.177572 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6b76700e-1870-40a7-a1f2-ab3c1332058c-metrics-certs\") pod \"network-metrics-daemon-8mnph\" (UID: \"6b76700e-1870-40a7-a1f2-ab3c1332058c\") " pod="openshift-multus/network-metrics-daemon-8mnph" Jul 06 19:09:37.179614 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.177598 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/02f3dffe-4bee-4c6d-9a45-7d1672d59adc-host-kubelet\") pod \"ovnkube-node-4g4j2\" (UID: \"02f3dffe-4bee-4c6d-9a45-7d1672d59adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-4g4j2" Jul 06 19:09:37.179614 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.177627 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/51b731db-0d7f-4a6d-bd3b-30d452687c30-multus-cni-dir\") pod \"multus-s84ln\" (UID: \"51b731db-0d7f-4a6d-bd3b-30d452687c30\") " pod="openshift-multus/multus-s84ln" Jul 06 19:09:37.179614 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.177651 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/51b731db-0d7f-4a6d-bd3b-30d452687c30-host-run-multus-certs\") pod \"multus-s84ln\" (UID: \"51b731db-0d7f-4a6d-bd3b-30d452687c30\") " pod="openshift-multus/multus-s84ln" Jul 06 19:09:37.179614 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.177675 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e9a3a524-5e1b-48db-ba03-750883088470-system-cni-dir\") pod \"multus-additional-cni-plugins-65k2z\" (UID: \"e9a3a524-5e1b-48db-ba03-750883088470\") " pod="openshift-multus/multus-additional-cni-plugins-65k2z" Jul 06 19:09:37.179614 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.177702 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ds78w\" (UniqueName: \"kubernetes.io/projected/e9a3a524-5e1b-48db-ba03-750883088470-kube-api-access-ds78w\") pod \"multus-additional-cni-plugins-65k2z\" (UID: \"e9a3a524-5e1b-48db-ba03-750883088470\") " pod="openshift-multus/multus-additional-cni-plugins-65k2z" Jul 06 19:09:37.179614 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.177727 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/12b34f02-96ce-44b0-84bc-2a2aa73a9e78-etc-modprobe-d\") pod \"tuned-f9jxc\" (UID: \"12b34f02-96ce-44b0-84bc-2a2aa73a9e78\") " pod="openshift-cluster-node-tuning-operator/tuned-f9jxc" Jul 06 19:09:37.179614 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.177768 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/02f3dffe-4bee-4c6d-9a45-7d1672d59adc-env-overrides\") pod \"ovnkube-node-4g4j2\" (UID: \"02f3dffe-4bee-4c6d-9a45-7d1672d59adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-4g4j2" Jul 06 19:09:37.179614 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.177512 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6b947af4-1d82-43fd-9137-73d083aed449-socket-dir\") pod \"aws-ebs-csi-driver-node-vjjbn\" (UID: \"6b947af4-1d82-43fd-9137-73d083aed449\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vjjbn" Jul 06 19:09:37.180432 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.177853 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/02f3dffe-4bee-4c6d-9a45-7d1672d59adc-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4g4j2\" (UID: \"02f3dffe-4bee-4c6d-9a45-7d1672d59adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-4g4j2" Jul 06 19:09:37.180432 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.177878 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/51b731db-0d7f-4a6d-bd3b-30d452687c30-multus-daemon-config\") pod \"multus-s84ln\" (UID: \"51b731db-0d7f-4a6d-bd3b-30d452687c30\") " pod="openshift-multus/multus-s84ln" Jul 06 19:09:37.180432 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.177905 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/02f3dffe-4bee-4c6d-9a45-7d1672d59adc-run-ovn\") pod \"ovnkube-node-4g4j2\" (UID: \"02f3dffe-4bee-4c6d-9a45-7d1672d59adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-4g4j2" Jul 06 19:09:37.180432 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.177931 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e9a3a524-5e1b-48db-ba03-750883088470-system-cni-dir\") pod \"multus-additional-cni-plugins-65k2z\" (UID: \"e9a3a524-5e1b-48db-ba03-750883088470\") " pod="openshift-multus/multus-additional-cni-plugins-65k2z" Jul 06 19:09:37.180432 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.177973 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/12b34f02-96ce-44b0-84bc-2a2aa73a9e78-etc-modprobe-d\") pod \"tuned-f9jxc\" (UID: \"12b34f02-96ce-44b0-84bc-2a2aa73a9e78\") " pod="openshift-cluster-node-tuning-operator/tuned-f9jxc" Jul 06 19:09:37.180432 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.177993 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/51b731db-0d7f-4a6d-bd3b-30d452687c30-multus-cni-dir\") pod \"multus-s84ln\" (UID: \"51b731db-0d7f-4a6d-bd3b-30d452687c30\") " pod="openshift-multus/multus-s84ln" Jul 06 19:09:37.180432 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.178028 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/51b731db-0d7f-4a6d-bd3b-30d452687c30-host-run-multus-certs\") pod \"multus-s84ln\" (UID: \"51b731db-0d7f-4a6d-bd3b-30d452687c30\") " pod="openshift-multus/multus-s84ln" Jul 06 19:09:37.180432 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.178044 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/12b34f02-96ce-44b0-84bc-2a2aa73a9e78-var-lib-kubelet\") pod \"tuned-f9jxc\" (UID: \"12b34f02-96ce-44b0-84bc-2a2aa73a9e78\") " pod="openshift-cluster-node-tuning-operator/tuned-f9jxc" Jul 06 19:09:37.180432 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.178117 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/51b731db-0d7f-4a6d-bd3b-30d452687c30-cnibin\") pod \"multus-s84ln\" (UID: \"51b731db-0d7f-4a6d-bd3b-30d452687c30\") " pod="openshift-multus/multus-s84ln" Jul 06 19:09:37.180432 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:09:37.178400 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 19:09:37.180432 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.178441 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/51b731db-0d7f-4a6d-bd3b-30d452687c30-host-var-lib-cni-multus\") pod \"multus-s84ln\" (UID: \"51b731db-0d7f-4a6d-bd3b-30d452687c30\") " pod="openshift-multus/multus-s84ln" Jul 06 19:09:37.180432 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.178402 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/02f3dffe-4bee-4c6d-9a45-7d1672d59adc-ovn-node-metrics-cert\") pod \"ovnkube-node-4g4j2\" (UID: \"02f3dffe-4bee-4c6d-9a45-7d1672d59adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-4g4j2" Jul 06 19:09:37.180432 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.177981 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/12b34f02-96ce-44b0-84bc-2a2aa73a9e78-lib-modules\") pod \"tuned-f9jxc\" (UID: \"12b34f02-96ce-44b0-84bc-2a2aa73a9e78\") " pod="openshift-cluster-node-tuning-operator/tuned-f9jxc" Jul 06 19:09:37.180432 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.178488 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6b947af4-1d82-43fd-9137-73d083aed449-kubelet-dir\") pod \"aws-ebs-csi-driver-node-vjjbn\" (UID: \"6b947af4-1d82-43fd-9137-73d083aed449\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vjjbn" Jul 06 19:09:37.180432 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.178550 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/02f3dffe-4bee-4c6d-9a45-7d1672d59adc-host-slash\") pod \"ovnkube-node-4g4j2\" (UID: \"02f3dffe-4bee-4c6d-9a45-7d1672d59adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-4g4j2" Jul 06 19:09:37.180432 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.178550 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/12b34f02-96ce-44b0-84bc-2a2aa73a9e78-etc-sysconfig\") pod \"tuned-f9jxc\" (UID: \"12b34f02-96ce-44b0-84bc-2a2aa73a9e78\") " pod="openshift-cluster-node-tuning-operator/tuned-f9jxc" Jul 06 19:09:37.180432 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.178589 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/02f3dffe-4bee-4c6d-9a45-7d1672d59adc-host-kubelet\") pod \"ovnkube-node-4g4j2\" (UID: \"02f3dffe-4bee-4c6d-9a45-7d1672d59adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-4g4j2" Jul 06 19:09:37.181240 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:09:37.178594 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6b76700e-1870-40a7-a1f2-ab3c1332058c-metrics-certs podName:6b76700e-1870-40a7-a1f2-ab3c1332058c nodeName:}" failed. No retries permitted until 2026-07-06 19:09:37.678550346 +0000 UTC m=+2.985909283 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6b76700e-1870-40a7-a1f2-ab3c1332058c-metrics-certs") pod "network-metrics-daemon-8mnph" (UID: "6b76700e-1870-40a7-a1f2-ab3c1332058c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 19:09:37.181240 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.178628 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/51b731db-0d7f-4a6d-bd3b-30d452687c30-host-run-k8s-cni-cncf-io\") pod \"multus-s84ln\" (UID: \"51b731db-0d7f-4a6d-bd3b-30d452687c30\") " pod="openshift-multus/multus-s84ln" Jul 06 19:09:37.181240 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.178937 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/12b34f02-96ce-44b0-84bc-2a2aa73a9e78-host\") pod \"tuned-f9jxc\" (UID: \"12b34f02-96ce-44b0-84bc-2a2aa73a9e78\") " pod="openshift-cluster-node-tuning-operator/tuned-f9jxc" Jul 06 19:09:37.181240 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.179097 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/51b731db-0d7f-4a6d-bd3b-30d452687c30-hostroot\") pod \"multus-s84ln\" (UID: \"51b731db-0d7f-4a6d-bd3b-30d452687c30\") " pod="openshift-multus/multus-s84ln" Jul 06 19:09:37.181240 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.179128 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e9a3a524-5e1b-48db-ba03-750883088470-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-65k2z\" (UID: \"e9a3a524-5e1b-48db-ba03-750883088470\") " pod="openshift-multus/multus-additional-cni-plugins-65k2z" Jul 06 19:09:37.181240 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.179134 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/f1803bd2-22ae-4026-b66d-320e1c122a10-iptables-alerter-script\") pod \"iptables-alerter-mznhh\" (UID: \"f1803bd2-22ae-4026-b66d-320e1c122a10\") " pod="openshift-network-operator/iptables-alerter-mznhh" Jul 06 19:09:37.181240 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.179209 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/51b731db-0d7f-4a6d-bd3b-30d452687c30-host-run-k8s-cni-cncf-io\") pod \"multus-s84ln\" (UID: \"51b731db-0d7f-4a6d-bd3b-30d452687c30\") " pod="openshift-multus/multus-s84ln" Jul 06 19:09:37.181240 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.179208 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/12b34f02-96ce-44b0-84bc-2a2aa73a9e78-sys\") pod \"tuned-f9jxc\" (UID: \"12b34f02-96ce-44b0-84bc-2a2aa73a9e78\") " pod="openshift-cluster-node-tuning-operator/tuned-f9jxc" Jul 06 19:09:37.181240 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.179259 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-865s6\" (UniqueName: \"kubernetes.io/projected/c75820ce-6423-4c3f-8e13-0ab0e5b00914-kube-api-access-865s6\") pod \"node-resolver-8btbx\" (UID: \"c75820ce-6423-4c3f-8e13-0ab0e5b00914\") " pod="openshift-dns/node-resolver-8btbx" Jul 06 19:09:37.181240 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.179273 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/51b731db-0d7f-4a6d-bd3b-30d452687c30-hostroot\") pod \"multus-s84ln\" (UID: \"51b731db-0d7f-4a6d-bd3b-30d452687c30\") " pod="openshift-multus/multus-s84ln" Jul 06 19:09:37.181240 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.179389 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e9a3a524-5e1b-48db-ba03-750883088470-cnibin\") pod \"multus-additional-cni-plugins-65k2z\" (UID: \"e9a3a524-5e1b-48db-ba03-750883088470\") " pod="openshift-multus/multus-additional-cni-plugins-65k2z" Jul 06 19:09:37.181240 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.179390 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/12b34f02-96ce-44b0-84bc-2a2aa73a9e78-sys\") pod \"tuned-f9jxc\" (UID: \"12b34f02-96ce-44b0-84bc-2a2aa73a9e78\") " pod="openshift-cluster-node-tuning-operator/tuned-f9jxc" Jul 06 19:09:37.181240 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.179511 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6b947af4-1d82-43fd-9137-73d083aed449-etc-selinux\") pod \"aws-ebs-csi-driver-node-vjjbn\" (UID: \"6b947af4-1d82-43fd-9137-73d083aed449\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vjjbn" Jul 06 19:09:37.181240 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.179540 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/12b34f02-96ce-44b0-84bc-2a2aa73a9e78-etc-systemd\") pod \"tuned-f9jxc\" (UID: \"12b34f02-96ce-44b0-84bc-2a2aa73a9e78\") " pod="openshift-cluster-node-tuning-operator/tuned-f9jxc" Jul 06 19:09:37.181240 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.179602 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/51b731db-0d7f-4a6d-bd3b-30d452687c30-system-cni-dir\") pod \"multus-s84ln\" (UID: \"51b731db-0d7f-4a6d-bd3b-30d452687c30\") " pod="openshift-multus/multus-s84ln" Jul 06 19:09:37.181240 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.179636 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/51b731db-0d7f-4a6d-bd3b-30d452687c30-host-run-netns\") pod \"multus-s84ln\" (UID: \"51b731db-0d7f-4a6d-bd3b-30d452687c30\") " pod="openshift-multus/multus-s84ln" Jul 06 19:09:37.181240 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.179693 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e9a3a524-5e1b-48db-ba03-750883088470-cnibin\") pod \"multus-additional-cni-plugins-65k2z\" (UID: \"e9a3a524-5e1b-48db-ba03-750883088470\") " pod="openshift-multus/multus-additional-cni-plugins-65k2z" Jul 06 19:09:37.181927 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.179707 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/12b34f02-96ce-44b0-84bc-2a2aa73a9e78-etc-systemd\") pod \"tuned-f9jxc\" (UID: \"12b34f02-96ce-44b0-84bc-2a2aa73a9e78\") " pod="openshift-cluster-node-tuning-operator/tuned-f9jxc" Jul 06 19:09:37.181927 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.179847 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6b947af4-1d82-43fd-9137-73d083aed449-etc-selinux\") pod \"aws-ebs-csi-driver-node-vjjbn\" (UID: \"6b947af4-1d82-43fd-9137-73d083aed449\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vjjbn" Jul 06 19:09:37.181927 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.179854 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/51b731db-0d7f-4a6d-bd3b-30d452687c30-system-cni-dir\") pod \"multus-s84ln\" (UID: \"51b731db-0d7f-4a6d-bd3b-30d452687c30\") " pod="openshift-multus/multus-s84ln" Jul 06 19:09:37.181927 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.179891 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/51b731db-0d7f-4a6d-bd3b-30d452687c30-host-run-netns\") pod \"multus-s84ln\" (UID: \"51b731db-0d7f-4a6d-bd3b-30d452687c30\") " pod="openshift-multus/multus-s84ln" Jul 06 19:09:37.181927 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.179896 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e9a3a524-5e1b-48db-ba03-750883088470-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-65k2z\" (UID: \"e9a3a524-5e1b-48db-ba03-750883088470\") " pod="openshift-multus/multus-additional-cni-plugins-65k2z" Jul 06 19:09:37.181927 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.180023 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/12b34f02-96ce-44b0-84bc-2a2aa73a9e78-etc-sysctl-d\") pod \"tuned-f9jxc\" (UID: \"12b34f02-96ce-44b0-84bc-2a2aa73a9e78\") " pod="openshift-cluster-node-tuning-operator/tuned-f9jxc" Jul 06 19:09:37.181927 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.180105 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/02f3dffe-4bee-4c6d-9a45-7d1672d59adc-run-ovn\") pod \"ovnkube-node-4g4j2\" (UID: \"02f3dffe-4bee-4c6d-9a45-7d1672d59adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-4g4j2" Jul 06 19:09:37.181927 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.181224 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/12b34f02-96ce-44b0-84bc-2a2aa73a9e78-etc-tuned\") pod \"tuned-f9jxc\" (UID: \"12b34f02-96ce-44b0-84bc-2a2aa73a9e78\") " pod="openshift-cluster-node-tuning-operator/tuned-f9jxc" Jul 06 19:09:37.181927 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.181343 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/02f3dffe-4bee-4c6d-9a45-7d1672d59adc-ovn-node-metrics-cert\") pod \"ovnkube-node-4g4j2\" (UID: \"02f3dffe-4bee-4c6d-9a45-7d1672d59adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-4g4j2" Jul 06 19:09:37.181927 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.181367 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/12b34f02-96ce-44b0-84bc-2a2aa73a9e78-tmp\") pod \"tuned-f9jxc\" (UID: \"12b34f02-96ce-44b0-84bc-2a2aa73a9e78\") " pod="openshift-cluster-node-tuning-operator/tuned-f9jxc" Jul 06 19:09:37.181927 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.181752 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/fa566ec3-6b97-4393-9744-5a9f7e8f0614-agent-certs\") pod \"konnectivity-agent-fjdvf\" (UID: \"fa566ec3-6b97-4393-9744-5a9f7e8f0614\") " pod="kube-system/konnectivity-agent-fjdvf" Jul 06 19:09:37.194623 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:09:37.194592 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 19:09:37.194623 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:09:37.194626 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 19:09:37.194902 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:09:37.194641 2566 projected.go:196] Error preparing data for projected volume kube-api-access-8sdmq for pod openshift-network-diagnostics/network-check-target-f9z4j: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 19:09:37.194902 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:09:37.194719 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2f2852c7-f9ef-445c-aa57-e5c697cb2c4b-kube-api-access-8sdmq podName:2f2852c7-f9ef-445c-aa57-e5c697cb2c4b nodeName:}" failed. No retries permitted until 2026-07-06 19:09:37.694697757 +0000 UTC m=+3.002056699 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-8sdmq" (UniqueName: "kubernetes.io/projected/2f2852c7-f9ef-445c-aa57-e5c697cb2c4b-kube-api-access-8sdmq") pod "network-check-target-f9z4j" (UID: "2f2852c7-f9ef-445c-aa57-e5c697cb2c4b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 19:09:37.198318 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.198093 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xmm8n\" (UniqueName: \"kubernetes.io/projected/51b731db-0d7f-4a6d-bd3b-30d452687c30-kube-api-access-xmm8n\") pod \"multus-s84ln\" (UID: \"51b731db-0d7f-4a6d-bd3b-30d452687c30\") " pod="openshift-multus/multus-s84ln" Jul 06 19:09:37.198958 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.198926 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tkmtq\" (UniqueName: \"kubernetes.io/projected/02f3dffe-4bee-4c6d-9a45-7d1672d59adc-kube-api-access-tkmtq\") pod \"ovnkube-node-4g4j2\" (UID: \"02f3dffe-4bee-4c6d-9a45-7d1672d59adc\") " pod="openshift-ovn-kubernetes/ovnkube-node-4g4j2" Jul 06 19:09:37.199238 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.199213 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rg6fx\" (UniqueName: \"kubernetes.io/projected/12b34f02-96ce-44b0-84bc-2a2aa73a9e78-kube-api-access-rg6fx\") pod \"tuned-f9jxc\" (UID: \"12b34f02-96ce-44b0-84bc-2a2aa73a9e78\") " pod="openshift-cluster-node-tuning-operator/tuned-f9jxc" Jul 06 19:09:37.200262 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.200166 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d87n7\" (UniqueName: \"kubernetes.io/projected/e37e30a5-e12f-4c79-a1ff-28c9460d44b3-kube-api-access-d87n7\") pod \"node-ca-qbwhx\" (UID: \"e37e30a5-e12f-4c79-a1ff-28c9460d44b3\") " pod="openshift-image-registry/node-ca-qbwhx" Jul 06 19:09:37.200262 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.200207 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ds78w\" (UniqueName: \"kubernetes.io/projected/e9a3a524-5e1b-48db-ba03-750883088470-kube-api-access-ds78w\") pod \"multus-additional-cni-plugins-65k2z\" (UID: \"e9a3a524-5e1b-48db-ba03-750883088470\") " pod="openshift-multus/multus-additional-cni-plugins-65k2z" Jul 06 19:09:37.200262 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.200217 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-88wjg\" (UniqueName: \"kubernetes.io/projected/6b947af4-1d82-43fd-9137-73d083aed449-kube-api-access-88wjg\") pod \"aws-ebs-csi-driver-node-vjjbn\" (UID: \"6b947af4-1d82-43fd-9137-73d083aed449\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vjjbn" Jul 06 19:09:37.200458 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.200374 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h4nf5\" (UniqueName: \"kubernetes.io/projected/f1803bd2-22ae-4026-b66d-320e1c122a10-kube-api-access-h4nf5\") pod \"iptables-alerter-mznhh\" (UID: \"f1803bd2-22ae-4026-b66d-320e1c122a10\") " pod="openshift-network-operator/iptables-alerter-mznhh" Jul 06 19:09:37.201949 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.201922 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-865s6\" (UniqueName: \"kubernetes.io/projected/c75820ce-6423-4c3f-8e13-0ab0e5b00914-kube-api-access-865s6\") pod \"node-resolver-8btbx\" (UID: \"c75820ce-6423-4c3f-8e13-0ab0e5b00914\") " pod="openshift-dns/node-resolver-8btbx" Jul 06 19:09:37.202056 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.201952 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8wvhb\" (UniqueName: \"kubernetes.io/projected/6b76700e-1870-40a7-a1f2-ab3c1332058c-kube-api-access-8wvhb\") pod \"network-metrics-daemon-8mnph\" (UID: \"6b76700e-1870-40a7-a1f2-ab3c1332058c\") " pod="openshift-multus/network-metrics-daemon-8mnph" Jul 06 19:09:37.236130 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.235956 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-35.ec2.internal" event={"ID":"04637dd40c4987c56de8cdb4373521aa","Type":"ContainerStarted","Data":"5ccbd25b87a735a682097b1439784eacb1fe6d8ee76acfbd9a1f935370c9bb72"} Jul 06 19:09:37.242222 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.242150 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-35.ec2.internal" event={"ID":"9fd512c517b497732795138b0a0b8677","Type":"ContainerStarted","Data":"8d208087256911384d8fb2020088d37f221eb423a23bbf9d49d8b767c50ae9b3"} Jul 06 19:09:37.280200 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.280162 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e8c2f347-1744-49ff-b46e-f036d27a5e46-original-pull-secret\") pod \"global-pull-secret-syncer-svgl2\" (UID: \"e8c2f347-1744-49ff-b46e-f036d27a5e46\") " pod="kube-system/global-pull-secret-syncer-svgl2" Jul 06 19:09:37.280378 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.280240 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e8c2f347-1744-49ff-b46e-f036d27a5e46-kubelet-config\") pod \"global-pull-secret-syncer-svgl2\" (UID: \"e8c2f347-1744-49ff-b46e-f036d27a5e46\") " pod="kube-system/global-pull-secret-syncer-svgl2" Jul 06 19:09:37.280378 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.280263 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e8c2f347-1744-49ff-b46e-f036d27a5e46-dbus\") pod \"global-pull-secret-syncer-svgl2\" (UID: \"e8c2f347-1744-49ff-b46e-f036d27a5e46\") " pod="kube-system/global-pull-secret-syncer-svgl2" Jul 06 19:09:37.280509 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.280447 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e8c2f347-1744-49ff-b46e-f036d27a5e46-dbus\") pod \"global-pull-secret-syncer-svgl2\" (UID: \"e8c2f347-1744-49ff-b46e-f036d27a5e46\") " pod="kube-system/global-pull-secret-syncer-svgl2" Jul 06 19:09:37.280573 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:09:37.280555 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 19:09:37.280629 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:09:37.280618 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e8c2f347-1744-49ff-b46e-f036d27a5e46-original-pull-secret podName:e8c2f347-1744-49ff-b46e-f036d27a5e46 nodeName:}" failed. No retries permitted until 2026-07-06 19:09:37.780603273 +0000 UTC m=+3.087962200 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e8c2f347-1744-49ff-b46e-f036d27a5e46-original-pull-secret") pod "global-pull-secret-syncer-svgl2" (UID: "e8c2f347-1744-49ff-b46e-f036d27a5e46") : object "kube-system"/"original-pull-secret" not registered Jul 06 19:09:37.280711 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.280692 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e8c2f347-1744-49ff-b46e-f036d27a5e46-kubelet-config\") pod \"global-pull-secret-syncer-svgl2\" (UID: \"e8c2f347-1744-49ff-b46e-f036d27a5e46\") " pod="kube-system/global-pull-secret-syncer-svgl2" Jul 06 19:09:37.356104 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.355992 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vjjbn" Jul 06 19:09:37.365434 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.365398 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-fjdvf" Jul 06 19:09:37.370245 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:37.370214 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b947af4_1d82_43fd_9137_73d083aed449.slice/crio-f4319e61999da78945d260564c8161376dfc45f32a3ea6b8e1ece8229e304faa WatchSource:0}: Error finding container f4319e61999da78945d260564c8161376dfc45f32a3ea6b8e1ece8229e304faa: Status 404 returned error can't find the container with id f4319e61999da78945d260564c8161376dfc45f32a3ea6b8e1ece8229e304faa Jul 06 19:09:37.375088 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.374854 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-s84ln" Jul 06 19:09:37.381217 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.380960 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-4g4j2" Jul 06 19:09:37.385653 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:37.385624 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa566ec3_6b97_4393_9744_5a9f7e8f0614.slice/crio-d4982ca576d1947ce07f899eed57b8e645f9941db9b364cbd914c6af5ebbe544 WatchSource:0}: Error finding container d4982ca576d1947ce07f899eed57b8e645f9941db9b364cbd914c6af5ebbe544: Status 404 returned error can't find the container with id d4982ca576d1947ce07f899eed57b8e645f9941db9b364cbd914c6af5ebbe544 Jul 06 19:09:37.386523 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.386478 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-8btbx" Jul 06 19:09:37.392512 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.392481 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-f9jxc" Jul 06 19:09:37.394442 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:37.394407 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02f3dffe_4bee_4c6d_9a45_7d1672d59adc.slice/crio-f3db2ff0d38293fd0f8684d4e2265423ec22fde743cbca7f6b9225c26cdb174b WatchSource:0}: Error finding container f3db2ff0d38293fd0f8684d4e2265423ec22fde743cbca7f6b9225c26cdb174b: Status 404 returned error can't find the container with id f3db2ff0d38293fd0f8684d4e2265423ec22fde743cbca7f6b9225c26cdb174b Jul 06 19:09:37.398642 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.398405 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-qbwhx" Jul 06 19:09:37.402611 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:37.402560 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc75820ce_6423_4c3f_8e13_0ab0e5b00914.slice/crio-9dbb0f3ef30d82a53724c023578352db16b580d25f4a51fccf5801258df52b12 WatchSource:0}: Error finding container 9dbb0f3ef30d82a53724c023578352db16b580d25f4a51fccf5801258df52b12: Status 404 returned error can't find the container with id 9dbb0f3ef30d82a53724c023578352db16b580d25f4a51fccf5801258df52b12 Jul 06 19:09:37.404719 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.404696 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-65k2z" Jul 06 19:09:37.410710 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.410664 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-mznhh" Jul 06 19:09:37.412038 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:37.412012 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode37e30a5_e12f_4c79_a1ff_28c9460d44b3.slice/crio-9f6e26c7d922305e97e3eb5debfc435fcb62a70c3d7a82e0bcd3866af4148365 WatchSource:0}: Error finding container 9f6e26c7d922305e97e3eb5debfc435fcb62a70c3d7a82e0bcd3866af4148365: Status 404 returned error can't find the container with id 9f6e26c7d922305e97e3eb5debfc435fcb62a70c3d7a82e0bcd3866af4148365 Jul 06 19:09:37.422000 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:37.421956 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9a3a524_5e1b_48db_ba03_750883088470.slice/crio-327106db9a34613a4c8f52f821f4527f0bed2f972ed2180f4739864346a57130 WatchSource:0}: Error finding container 327106db9a34613a4c8f52f821f4527f0bed2f972ed2180f4739864346a57130: Status 404 returned error can't find the container with id 327106db9a34613a4c8f52f821f4527f0bed2f972ed2180f4739864346a57130 Jul 06 19:09:37.424772 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:09:37.424744 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1803bd2_22ae_4026_b66d_320e1c122a10.slice/crio-0356e81132b6683123bf785fdb7300ce0633dd2f789b009c22db311cecead585 WatchSource:0}: Error finding container 0356e81132b6683123bf785fdb7300ce0633dd2f789b009c22db311cecead585: Status 404 returned error can't find the container with id 0356e81132b6683123bf785fdb7300ce0633dd2f789b009c22db311cecead585 Jul 06 19:09:37.682856 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.682743 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6b76700e-1870-40a7-a1f2-ab3c1332058c-metrics-certs\") pod \"network-metrics-daemon-8mnph\" (UID: \"6b76700e-1870-40a7-a1f2-ab3c1332058c\") " pod="openshift-multus/network-metrics-daemon-8mnph" Jul 06 19:09:37.683024 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:09:37.682937 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 19:09:37.683024 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:09:37.683008 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6b76700e-1870-40a7-a1f2-ab3c1332058c-metrics-certs podName:6b76700e-1870-40a7-a1f2-ab3c1332058c nodeName:}" failed. No retries permitted until 2026-07-06 19:09:38.682987357 +0000 UTC m=+3.990346284 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6b76700e-1870-40a7-a1f2-ab3c1332058c-metrics-certs") pod "network-metrics-daemon-8mnph" (UID: "6b76700e-1870-40a7-a1f2-ab3c1332058c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 19:09:37.783675 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.783640 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8sdmq\" (UniqueName: \"kubernetes.io/projected/2f2852c7-f9ef-445c-aa57-e5c697cb2c4b-kube-api-access-8sdmq\") pod \"network-check-target-f9z4j\" (UID: \"2f2852c7-f9ef-445c-aa57-e5c697cb2c4b\") " pod="openshift-network-diagnostics/network-check-target-f9z4j" Jul 06 19:09:37.783842 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:37.783708 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e8c2f347-1744-49ff-b46e-f036d27a5e46-original-pull-secret\") pod \"global-pull-secret-syncer-svgl2\" (UID: \"e8c2f347-1744-49ff-b46e-f036d27a5e46\") " pod="kube-system/global-pull-secret-syncer-svgl2" Jul 06 19:09:37.783842 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:09:37.783833 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 19:09:37.783958 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:09:37.783856 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 19:09:37.783958 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:09:37.783870 2566 projected.go:196] Error preparing data for projected volume kube-api-access-8sdmq for pod openshift-network-diagnostics/network-check-target-f9z4j: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 19:09:37.783958 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:09:37.783889 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 19:09:37.783958 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:09:37.783922 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2f2852c7-f9ef-445c-aa57-e5c697cb2c4b-kube-api-access-8sdmq podName:2f2852c7-f9ef-445c-aa57-e5c697cb2c4b nodeName:}" failed. No retries permitted until 2026-07-06 19:09:38.783908595 +0000 UTC m=+4.091267522 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-8sdmq" (UniqueName: "kubernetes.io/projected/2f2852c7-f9ef-445c-aa57-e5c697cb2c4b-kube-api-access-8sdmq") pod "network-check-target-f9z4j" (UID: "2f2852c7-f9ef-445c-aa57-e5c697cb2c4b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 19:09:37.783958 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:09:37.783956 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e8c2f347-1744-49ff-b46e-f036d27a5e46-original-pull-secret podName:e8c2f347-1744-49ff-b46e-f036d27a5e46 nodeName:}" failed. No retries permitted until 2026-07-06 19:09:38.783937396 +0000 UTC m=+4.091296328 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e8c2f347-1744-49ff-b46e-f036d27a5e46-original-pull-secret") pod "global-pull-secret-syncer-svgl2" (UID: "e8c2f347-1744-49ff-b46e-f036d27a5e46") : object "kube-system"/"original-pull-secret" not registered Jul 06 19:09:38.131456 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:38.131387 2566 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-05 19:04:37 +0000 UTC" deadline="2028-03-24 17:51:53.387810896 +0000 UTC" Jul 06 19:09:38.131456 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:38.131453 2566 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15046h42m15.256360933s" Jul 06 19:09:38.231017 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:38.230934 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f9z4j" Jul 06 19:09:38.231196 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:09:38.231085 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f9z4j" podUID="2f2852c7-f9ef-445c-aa57-e5c697cb2c4b" Jul 06 19:09:38.250772 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:38.250728 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-mznhh" event={"ID":"f1803bd2-22ae-4026-b66d-320e1c122a10","Type":"ContainerStarted","Data":"0356e81132b6683123bf785fdb7300ce0633dd2f789b009c22db311cecead585"} Jul 06 19:09:38.255087 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:38.255034 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4g4j2" event={"ID":"02f3dffe-4bee-4c6d-9a45-7d1672d59adc","Type":"ContainerStarted","Data":"f3db2ff0d38293fd0f8684d4e2265423ec22fde743cbca7f6b9225c26cdb174b"} Jul 06 19:09:38.258813 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:38.258771 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-s84ln" event={"ID":"51b731db-0d7f-4a6d-bd3b-30d452687c30","Type":"ContainerStarted","Data":"afa95364ef24e2abaf2ed6080f7d32743b7e8a4dca62468f830e1005a13e05a5"} Jul 06 19:09:38.261178 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:38.261137 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vjjbn" event={"ID":"6b947af4-1d82-43fd-9137-73d083aed449","Type":"ContainerStarted","Data":"f4319e61999da78945d260564c8161376dfc45f32a3ea6b8e1ece8229e304faa"} Jul 06 19:09:38.264164 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:38.264123 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-65k2z" event={"ID":"e9a3a524-5e1b-48db-ba03-750883088470","Type":"ContainerStarted","Data":"327106db9a34613a4c8f52f821f4527f0bed2f972ed2180f4739864346a57130"} Jul 06 19:09:38.266534 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:38.266493 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-qbwhx" event={"ID":"e37e30a5-e12f-4c79-a1ff-28c9460d44b3","Type":"ContainerStarted","Data":"9f6e26c7d922305e97e3eb5debfc435fcb62a70c3d7a82e0bcd3866af4148365"} Jul 06 19:09:38.268518 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:38.268445 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-f9jxc" event={"ID":"12b34f02-96ce-44b0-84bc-2a2aa73a9e78","Type":"ContainerStarted","Data":"f4d8e7c2abe73800337c8b9862a591da4c5a057a3b89ec69c770d355b54ff926"} Jul 06 19:09:38.270017 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:38.269991 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-8btbx" event={"ID":"c75820ce-6423-4c3f-8e13-0ab0e5b00914","Type":"ContainerStarted","Data":"9dbb0f3ef30d82a53724c023578352db16b580d25f4a51fccf5801258df52b12"} Jul 06 19:09:38.271605 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:38.271576 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-fjdvf" event={"ID":"fa566ec3-6b97-4393-9744-5a9f7e8f0614","Type":"ContainerStarted","Data":"d4982ca576d1947ce07f899eed57b8e645f9941db9b364cbd914c6af5ebbe544"} Jul 06 19:09:38.692703 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:38.692040 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6b76700e-1870-40a7-a1f2-ab3c1332058c-metrics-certs\") pod \"network-metrics-daemon-8mnph\" (UID: \"6b76700e-1870-40a7-a1f2-ab3c1332058c\") " pod="openshift-multus/network-metrics-daemon-8mnph" Jul 06 19:09:38.692703 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:09:38.692236 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 19:09:38.692703 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:09:38.692300 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6b76700e-1870-40a7-a1f2-ab3c1332058c-metrics-certs podName:6b76700e-1870-40a7-a1f2-ab3c1332058c nodeName:}" failed. No retries permitted until 2026-07-06 19:09:40.692281842 +0000 UTC m=+5.999640772 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6b76700e-1870-40a7-a1f2-ab3c1332058c-metrics-certs") pod "network-metrics-daemon-8mnph" (UID: "6b76700e-1870-40a7-a1f2-ab3c1332058c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 19:09:38.792775 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:38.792718 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8sdmq\" (UniqueName: \"kubernetes.io/projected/2f2852c7-f9ef-445c-aa57-e5c697cb2c4b-kube-api-access-8sdmq\") pod \"network-check-target-f9z4j\" (UID: \"2f2852c7-f9ef-445c-aa57-e5c697cb2c4b\") " pod="openshift-network-diagnostics/network-check-target-f9z4j" Jul 06 19:09:38.792950 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:38.792814 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e8c2f347-1744-49ff-b46e-f036d27a5e46-original-pull-secret\") pod \"global-pull-secret-syncer-svgl2\" (UID: \"e8c2f347-1744-49ff-b46e-f036d27a5e46\") " pod="kube-system/global-pull-secret-syncer-svgl2" Jul 06 19:09:38.792950 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:09:38.792937 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 19:09:38.793089 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:09:38.793002 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e8c2f347-1744-49ff-b46e-f036d27a5e46-original-pull-secret podName:e8c2f347-1744-49ff-b46e-f036d27a5e46 nodeName:}" failed. No retries permitted until 2026-07-06 19:09:40.792983216 +0000 UTC m=+6.100342146 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e8c2f347-1744-49ff-b46e-f036d27a5e46-original-pull-secret") pod "global-pull-secret-syncer-svgl2" (UID: "e8c2f347-1744-49ff-b46e-f036d27a5e46") : object "kube-system"/"original-pull-secret" not registered Jul 06 19:09:38.793472 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:09:38.793448 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 19:09:38.793564 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:09:38.793482 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 19:09:38.793564 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:09:38.793496 2566 projected.go:196] Error preparing data for projected volume kube-api-access-8sdmq for pod openshift-network-diagnostics/network-check-target-f9z4j: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 19:09:38.793564 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:09:38.793545 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2f2852c7-f9ef-445c-aa57-e5c697cb2c4b-kube-api-access-8sdmq podName:2f2852c7-f9ef-445c-aa57-e5c697cb2c4b nodeName:}" failed. No retries permitted until 2026-07-06 19:09:40.793528105 +0000 UTC m=+6.100887032 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-8sdmq" (UniqueName: "kubernetes.io/projected/2f2852c7-f9ef-445c-aa57-e5c697cb2c4b-kube-api-access-8sdmq") pod "network-check-target-f9z4j" (UID: "2f2852c7-f9ef-445c-aa57-e5c697cb2c4b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 19:09:39.132224 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:39.132127 2566 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-05 19:04:37 +0000 UTC" deadline="2028-04-22 02:39:01.856421668 +0000 UTC" Jul 06 19:09:39.132224 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:39.132170 2566 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15727h29m22.724256622s" Jul 06 19:09:39.232784 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:39.232510 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8mnph" Jul 06 19:09:39.232962 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:09:39.232906 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8mnph" podUID="6b76700e-1870-40a7-a1f2-ab3c1332058c" Jul 06 19:09:39.233342 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:39.233319 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-svgl2" Jul 06 19:09:39.233451 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:09:39.233428 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-svgl2" podUID="e8c2f347-1744-49ff-b46e-f036d27a5e46" Jul 06 19:09:40.231083 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:40.231032 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f9z4j" Jul 06 19:09:40.231561 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:09:40.231189 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f9z4j" podUID="2f2852c7-f9ef-445c-aa57-e5c697cb2c4b" Jul 06 19:09:40.282016 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:40.281920 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-35.ec2.internal" event={"ID":"9fd512c517b497732795138b0a0b8677","Type":"ContainerStarted","Data":"568a612b4b9a48a3fea88ebe1adb6cb933941c642969cbc0906bcd1d8aca5d10"} Jul 06 19:09:40.287381 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:40.287341 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-35.ec2.internal" event={"ID":"04637dd40c4987c56de8cdb4373521aa","Type":"ContainerStarted","Data":"b4dee6577b59f1b2cca20b4bf230b85eaf46da400f04cf647a37bb9ea3dacbf2"} Jul 06 19:09:40.297506 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:40.297443 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-35.ec2.internal" podStartSLOduration=4.297422404 podStartE2EDuration="4.297422404s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 19:09:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:09:40.297297828 +0000 UTC m=+5.604656775" watchObservedRunningTime="2026-07-06 19:09:40.297422404 +0000 UTC m=+5.604781355" Jul 06 19:09:40.714662 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:40.714571 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6b76700e-1870-40a7-a1f2-ab3c1332058c-metrics-certs\") pod \"network-metrics-daemon-8mnph\" (UID: \"6b76700e-1870-40a7-a1f2-ab3c1332058c\") " pod="openshift-multus/network-metrics-daemon-8mnph" Jul 06 19:09:40.714833 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:09:40.714796 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 19:09:40.714894 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:09:40.714862 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6b76700e-1870-40a7-a1f2-ab3c1332058c-metrics-certs podName:6b76700e-1870-40a7-a1f2-ab3c1332058c nodeName:}" failed. No retries permitted until 2026-07-06 19:09:44.714842654 +0000 UTC m=+10.022201587 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6b76700e-1870-40a7-a1f2-ab3c1332058c-metrics-certs") pod "network-metrics-daemon-8mnph" (UID: "6b76700e-1870-40a7-a1f2-ab3c1332058c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 19:09:40.816005 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:40.815964 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8sdmq\" (UniqueName: \"kubernetes.io/projected/2f2852c7-f9ef-445c-aa57-e5c697cb2c4b-kube-api-access-8sdmq\") pod \"network-check-target-f9z4j\" (UID: \"2f2852c7-f9ef-445c-aa57-e5c697cb2c4b\") " pod="openshift-network-diagnostics/network-check-target-f9z4j" Jul 06 19:09:40.816211 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:40.816036 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e8c2f347-1744-49ff-b46e-f036d27a5e46-original-pull-secret\") pod \"global-pull-secret-syncer-svgl2\" (UID: \"e8c2f347-1744-49ff-b46e-f036d27a5e46\") " pod="kube-system/global-pull-secret-syncer-svgl2" Jul 06 19:09:40.816211 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:09:40.816198 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 19:09:40.816348 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:09:40.816267 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e8c2f347-1744-49ff-b46e-f036d27a5e46-original-pull-secret podName:e8c2f347-1744-49ff-b46e-f036d27a5e46 nodeName:}" failed. No retries permitted until 2026-07-06 19:09:44.816245932 +0000 UTC m=+10.123604860 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e8c2f347-1744-49ff-b46e-f036d27a5e46-original-pull-secret") pod "global-pull-secret-syncer-svgl2" (UID: "e8c2f347-1744-49ff-b46e-f036d27a5e46") : object "kube-system"/"original-pull-secret" not registered Jul 06 19:09:40.816723 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:09:40.816704 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 19:09:40.816797 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:09:40.816730 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 19:09:40.816797 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:09:40.816744 2566 projected.go:196] Error preparing data for projected volume kube-api-access-8sdmq for pod openshift-network-diagnostics/network-check-target-f9z4j: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 19:09:40.816912 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:09:40.816820 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2f2852c7-f9ef-445c-aa57-e5c697cb2c4b-kube-api-access-8sdmq podName:2f2852c7-f9ef-445c-aa57-e5c697cb2c4b nodeName:}" failed. No retries permitted until 2026-07-06 19:09:44.816804423 +0000 UTC m=+10.124163357 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-8sdmq" (UniqueName: "kubernetes.io/projected/2f2852c7-f9ef-445c-aa57-e5c697cb2c4b-kube-api-access-8sdmq") pod "network-check-target-f9z4j" (UID: "2f2852c7-f9ef-445c-aa57-e5c697cb2c4b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 19:09:41.233119 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:41.233083 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8mnph" Jul 06 19:09:41.233576 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:09:41.233234 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8mnph" podUID="6b76700e-1870-40a7-a1f2-ab3c1332058c" Jul 06 19:09:41.233642 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:41.233625 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-svgl2" Jul 06 19:09:41.233754 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:09:41.233725 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-svgl2" podUID="e8c2f347-1744-49ff-b46e-f036d27a5e46" Jul 06 19:09:42.230963 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:42.230870 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f9z4j" Jul 06 19:09:42.231162 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:09:42.231006 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f9z4j" podUID="2f2852c7-f9ef-445c-aa57-e5c697cb2c4b" Jul 06 19:09:42.295872 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:42.295807 2566 generic.go:356] "Generic (PLEG): container finished" podID="04637dd40c4987c56de8cdb4373521aa" containerID="b4dee6577b59f1b2cca20b4bf230b85eaf46da400f04cf647a37bb9ea3dacbf2" exitCode=0 Jul 06 19:09:42.295872 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:42.295846 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-35.ec2.internal" event={"ID":"04637dd40c4987c56de8cdb4373521aa","Type":"ContainerDied","Data":"b4dee6577b59f1b2cca20b4bf230b85eaf46da400f04cf647a37bb9ea3dacbf2"} Jul 06 19:09:43.231780 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:43.231034 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8mnph" Jul 06 19:09:43.231780 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:09:43.231206 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8mnph" podUID="6b76700e-1870-40a7-a1f2-ab3c1332058c" Jul 06 19:09:43.231780 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:43.231636 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-svgl2" Jul 06 19:09:43.231780 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:09:43.231737 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-svgl2" podUID="e8c2f347-1744-49ff-b46e-f036d27a5e46" Jul 06 19:09:44.230452 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:44.230418 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f9z4j" Jul 06 19:09:44.230899 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:09:44.230547 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f9z4j" podUID="2f2852c7-f9ef-445c-aa57-e5c697cb2c4b" Jul 06 19:09:44.752050 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:44.752001 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6b76700e-1870-40a7-a1f2-ab3c1332058c-metrics-certs\") pod \"network-metrics-daemon-8mnph\" (UID: \"6b76700e-1870-40a7-a1f2-ab3c1332058c\") " pod="openshift-multus/network-metrics-daemon-8mnph" Jul 06 19:09:44.752231 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:09:44.752200 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 19:09:44.752315 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:09:44.752265 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6b76700e-1870-40a7-a1f2-ab3c1332058c-metrics-certs podName:6b76700e-1870-40a7-a1f2-ab3c1332058c nodeName:}" failed. No retries permitted until 2026-07-06 19:09:52.752247296 +0000 UTC m=+18.059606230 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6b76700e-1870-40a7-a1f2-ab3c1332058c-metrics-certs") pod "network-metrics-daemon-8mnph" (UID: "6b76700e-1870-40a7-a1f2-ab3c1332058c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 19:09:44.856233 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:44.853215 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e8c2f347-1744-49ff-b46e-f036d27a5e46-original-pull-secret\") pod \"global-pull-secret-syncer-svgl2\" (UID: \"e8c2f347-1744-49ff-b46e-f036d27a5e46\") " pod="kube-system/global-pull-secret-syncer-svgl2" Jul 06 19:09:44.856233 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:44.853323 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8sdmq\" (UniqueName: \"kubernetes.io/projected/2f2852c7-f9ef-445c-aa57-e5c697cb2c4b-kube-api-access-8sdmq\") pod \"network-check-target-f9z4j\" (UID: \"2f2852c7-f9ef-445c-aa57-e5c697cb2c4b\") " pod="openshift-network-diagnostics/network-check-target-f9z4j" Jul 06 19:09:44.856233 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:09:44.853423 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 19:09:44.856233 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:09:44.853498 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e8c2f347-1744-49ff-b46e-f036d27a5e46-original-pull-secret podName:e8c2f347-1744-49ff-b46e-f036d27a5e46 nodeName:}" failed. No retries permitted until 2026-07-06 19:09:52.853482943 +0000 UTC m=+18.160841874 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e8c2f347-1744-49ff-b46e-f036d27a5e46-original-pull-secret") pod "global-pull-secret-syncer-svgl2" (UID: "e8c2f347-1744-49ff-b46e-f036d27a5e46") : object "kube-system"/"original-pull-secret" not registered Jul 06 19:09:44.856233 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:09:44.853498 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 19:09:44.856233 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:09:44.853517 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 19:09:44.856233 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:09:44.853529 2566 projected.go:196] Error preparing data for projected volume kube-api-access-8sdmq for pod openshift-network-diagnostics/network-check-target-f9z4j: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 19:09:44.856233 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:09:44.853585 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2f2852c7-f9ef-445c-aa57-e5c697cb2c4b-kube-api-access-8sdmq podName:2f2852c7-f9ef-445c-aa57-e5c697cb2c4b nodeName:}" failed. No retries permitted until 2026-07-06 19:09:52.853569062 +0000 UTC m=+18.160928006 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-8sdmq" (UniqueName: "kubernetes.io/projected/2f2852c7-f9ef-445c-aa57-e5c697cb2c4b-kube-api-access-8sdmq") pod "network-check-target-f9z4j" (UID: "2f2852c7-f9ef-445c-aa57-e5c697cb2c4b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 19:09:45.230882 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:45.230786 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-svgl2" Jul 06 19:09:45.231394 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:09:45.230921 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-svgl2" podUID="e8c2f347-1744-49ff-b46e-f036d27a5e46" Jul 06 19:09:45.231394 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:45.231335 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8mnph" Jul 06 19:09:45.231517 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:09:45.231491 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8mnph" podUID="6b76700e-1870-40a7-a1f2-ab3c1332058c" Jul 06 19:09:46.230730 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:46.230632 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f9z4j" Jul 06 19:09:46.230905 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:09:46.230778 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f9z4j" podUID="2f2852c7-f9ef-445c-aa57-e5c697cb2c4b" Jul 06 19:09:47.234375 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:47.234341 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-svgl2" Jul 06 19:09:47.234901 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:09:47.234482 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-svgl2" podUID="e8c2f347-1744-49ff-b46e-f036d27a5e46" Jul 06 19:09:47.234901 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:47.234808 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8mnph" Jul 06 19:09:47.234901 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:09:47.234897 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8mnph" podUID="6b76700e-1870-40a7-a1f2-ab3c1332058c" Jul 06 19:09:48.230515 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:48.230478 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f9z4j" Jul 06 19:09:48.230693 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:09:48.230590 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f9z4j" podUID="2f2852c7-f9ef-445c-aa57-e5c697cb2c4b" Jul 06 19:09:49.230672 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:49.230637 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-svgl2" Jul 06 19:09:49.231191 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:09:49.230784 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-svgl2" podUID="e8c2f347-1744-49ff-b46e-f036d27a5e46" Jul 06 19:09:49.231191 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:49.230901 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8mnph" Jul 06 19:09:49.231191 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:09:49.230989 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8mnph" podUID="6b76700e-1870-40a7-a1f2-ab3c1332058c" Jul 06 19:09:50.230482 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:50.230440 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f9z4j" Jul 06 19:09:50.230647 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:09:50.230562 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f9z4j" podUID="2f2852c7-f9ef-445c-aa57-e5c697cb2c4b" Jul 06 19:09:51.230211 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:51.230174 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8mnph" Jul 06 19:09:51.230700 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:09:51.230324 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8mnph" podUID="6b76700e-1870-40a7-a1f2-ab3c1332058c" Jul 06 19:09:51.230700 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:51.230594 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-svgl2" Jul 06 19:09:51.230700 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:09:51.230688 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-svgl2" podUID="e8c2f347-1744-49ff-b46e-f036d27a5e46" Jul 06 19:09:52.231129 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:52.231091 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f9z4j" Jul 06 19:09:52.231649 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:09:52.231209 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f9z4j" podUID="2f2852c7-f9ef-445c-aa57-e5c697cb2c4b" Jul 06 19:09:52.816520 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:52.816480 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6b76700e-1870-40a7-a1f2-ab3c1332058c-metrics-certs\") pod \"network-metrics-daemon-8mnph\" (UID: \"6b76700e-1870-40a7-a1f2-ab3c1332058c\") " pod="openshift-multus/network-metrics-daemon-8mnph" Jul 06 19:09:52.816682 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:09:52.816661 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 19:09:52.816733 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:09:52.816725 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6b76700e-1870-40a7-a1f2-ab3c1332058c-metrics-certs podName:6b76700e-1870-40a7-a1f2-ab3c1332058c nodeName:}" failed. No retries permitted until 2026-07-06 19:10:08.816708745 +0000 UTC m=+34.124067671 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6b76700e-1870-40a7-a1f2-ab3c1332058c-metrics-certs") pod "network-metrics-daemon-8mnph" (UID: "6b76700e-1870-40a7-a1f2-ab3c1332058c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 19:09:52.917714 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:52.917677 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8sdmq\" (UniqueName: \"kubernetes.io/projected/2f2852c7-f9ef-445c-aa57-e5c697cb2c4b-kube-api-access-8sdmq\") pod \"network-check-target-f9z4j\" (UID: \"2f2852c7-f9ef-445c-aa57-e5c697cb2c4b\") " pod="openshift-network-diagnostics/network-check-target-f9z4j" Jul 06 19:09:52.917928 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:52.917741 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e8c2f347-1744-49ff-b46e-f036d27a5e46-original-pull-secret\") pod \"global-pull-secret-syncer-svgl2\" (UID: \"e8c2f347-1744-49ff-b46e-f036d27a5e46\") " pod="kube-system/global-pull-secret-syncer-svgl2" Jul 06 19:09:52.917928 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:09:52.917860 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 19:09:52.917928 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:09:52.917871 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 19:09:52.917928 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:09:52.917888 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 19:09:52.917928 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:09:52.917899 2566 projected.go:196] Error preparing data for projected volume kube-api-access-8sdmq for pod openshift-network-diagnostics/network-check-target-f9z4j: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 19:09:52.917928 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:09:52.917930 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e8c2f347-1744-49ff-b46e-f036d27a5e46-original-pull-secret podName:e8c2f347-1744-49ff-b46e-f036d27a5e46 nodeName:}" failed. No retries permitted until 2026-07-06 19:10:08.917912185 +0000 UTC m=+34.225271131 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e8c2f347-1744-49ff-b46e-f036d27a5e46-original-pull-secret") pod "global-pull-secret-syncer-svgl2" (UID: "e8c2f347-1744-49ff-b46e-f036d27a5e46") : object "kube-system"/"original-pull-secret" not registered Jul 06 19:09:52.918167 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:09:52.917950 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2f2852c7-f9ef-445c-aa57-e5c697cb2c4b-kube-api-access-8sdmq podName:2f2852c7-f9ef-445c-aa57-e5c697cb2c4b nodeName:}" failed. No retries permitted until 2026-07-06 19:10:08.917942526 +0000 UTC m=+34.225301455 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-8sdmq" (UniqueName: "kubernetes.io/projected/2f2852c7-f9ef-445c-aa57-e5c697cb2c4b-kube-api-access-8sdmq") pod "network-check-target-f9z4j" (UID: "2f2852c7-f9ef-445c-aa57-e5c697cb2c4b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 19:09:53.230358 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:53.230275 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8mnph" Jul 06 19:09:53.230358 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:53.230317 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-svgl2" Jul 06 19:09:53.230557 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:09:53.230433 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8mnph" podUID="6b76700e-1870-40a7-a1f2-ab3c1332058c" Jul 06 19:09:53.230609 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:09:53.230555 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-svgl2" podUID="e8c2f347-1744-49ff-b46e-f036d27a5e46" Jul 06 19:09:54.230865 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:54.230830 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f9z4j" Jul 06 19:09:54.231337 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:09:54.230950 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f9z4j" podUID="2f2852c7-f9ef-445c-aa57-e5c697cb2c4b" Jul 06 19:09:55.232044 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:55.231415 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8mnph" Jul 06 19:09:55.232044 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:09:55.231538 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8mnph" podUID="6b76700e-1870-40a7-a1f2-ab3c1332058c" Jul 06 19:09:55.232044 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:55.231910 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-svgl2" Jul 06 19:09:55.232044 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:09:55.231994 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-svgl2" podUID="e8c2f347-1744-49ff-b46e-f036d27a5e46" Jul 06 19:09:56.230139 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:56.230101 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f9z4j" Jul 06 19:09:56.230307 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:09:56.230242 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f9z4j" podUID="2f2852c7-f9ef-445c-aa57-e5c697cb2c4b" Jul 06 19:09:57.233747 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:57.232386 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8mnph" Jul 06 19:09:57.233747 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:09:57.232743 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8mnph" podUID="6b76700e-1870-40a7-a1f2-ab3c1332058c" Jul 06 19:09:57.233747 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:57.232814 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-svgl2" Jul 06 19:09:57.233747 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:09:57.232881 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-svgl2" podUID="e8c2f347-1744-49ff-b46e-f036d27a5e46" Jul 06 19:09:57.328823 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:57.327667 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-s84ln" event={"ID":"51b731db-0d7f-4a6d-bd3b-30d452687c30","Type":"ContainerStarted","Data":"e62f2fa5635f2e33b783b874be1d4f3141fab1deca7676e6f644ebcd7ed26379"} Jul 06 19:09:57.332693 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:57.332656 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-35.ec2.internal" event={"ID":"04637dd40c4987c56de8cdb4373521aa","Type":"ContainerStarted","Data":"a31f949c8f722b0703705733983dbce069b564e90a6812a4829dcb95f8d8f6b6"} Jul 06 19:09:57.337231 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:57.336494 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-qbwhx" event={"ID":"e37e30a5-e12f-4c79-a1ff-28c9460d44b3","Type":"ContainerStarted","Data":"7173c6298a59778205df76da8e7e596dc7f53d58208fb5a2509511138a67bce6"} Jul 06 19:09:57.338483 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:57.338449 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-f9jxc" event={"ID":"12b34f02-96ce-44b0-84bc-2a2aa73a9e78","Type":"ContainerStarted","Data":"bd7d3641e21e1e0b852e57937f757c3fa20a0de857143f489f1349cffa86534e"} Jul 06 19:09:57.341685 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:57.341657 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-fjdvf" event={"ID":"fa566ec3-6b97-4393-9744-5a9f7e8f0614","Type":"ContainerStarted","Data":"d01496d3bff82d70d58dc9a8077e442ee4423b8f8c8d56853190eed5380bebed"} Jul 06 19:09:57.360375 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:57.359867 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-qbwhx" podStartSLOduration=2.965950367 podStartE2EDuration="22.359851564s" totalImagesPullingTime="19.393901197s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 19:09:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:09:57.359481363 +0000 UTC m=+22.666840314" watchObservedRunningTime="2026-07-06 19:09:57.359851564 +0000 UTC m=+22.667210513" Jul 06 19:09:57.360375 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:57.359941 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-s84ln" podStartSLOduration=2.649156434 podStartE2EDuration="22.359934516s" totalImagesPullingTime="19.710778082s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 19:09:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:09:57.345258207 +0000 UTC m=+22.652617156" watchObservedRunningTime="2026-07-06 19:09:57.359934516 +0000 UTC m=+22.667293464" Jul 06 19:09:57.375968 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:57.375521 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-35.ec2.internal" podStartSLOduration=21.375493846 podStartE2EDuration="21.375493846s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 19:09:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:09:57.373764709 +0000 UTC m=+22.681123658" watchObservedRunningTime="2026-07-06 19:09:57.375493846 +0000 UTC m=+22.682852787" Jul 06 19:09:57.392290 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:57.392214 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-f9jxc" podStartSLOduration=2.729394536 podStartE2EDuration="22.392192183s" totalImagesPullingTime="19.662797647s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 19:09:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:09:57.392106735 +0000 UTC m=+22.699465685" watchObservedRunningTime="2026-07-06 19:09:57.392192183 +0000 UTC m=+22.699551134" Jul 06 19:09:57.409474 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:57.409369 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-fjdvf" podStartSLOduration=7.311847697 podStartE2EDuration="22.409345477s" totalImagesPullingTime="15.09749778s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:09:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:09:57.409117474 +0000 UTC m=+22.716476423" watchObservedRunningTime="2026-07-06 19:09:57.409345477 +0000 UTC m=+22.716704432" Jul 06 19:09:57.932049 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:57.931787 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-fjdvf" Jul 06 19:09:57.932612 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:57.932590 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-fjdvf" Jul 06 19:09:58.230329 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:58.230295 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f9z4j" Jul 06 19:09:58.230499 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:09:58.230403 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f9z4j" podUID="2f2852c7-f9ef-445c-aa57-e5c697cb2c4b" Jul 06 19:09:58.345948 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:58.345910 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-8btbx" event={"ID":"c75820ce-6423-4c3f-8e13-0ab0e5b00914","Type":"ContainerStarted","Data":"353dff2579108d42de788533a5674ed9e45586187f3782e310d5ed9d4a1fd16a"} Jul 06 19:09:58.349556 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:58.349518 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4g4j2" event={"ID":"02f3dffe-4bee-4c6d-9a45-7d1672d59adc","Type":"ContainerStarted","Data":"fd71f09682268d97d2b46a1a1dd4272fcc6df7820762fc772c0d6fa040433063"} Jul 06 19:09:58.349727 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:58.349605 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4g4j2" event={"ID":"02f3dffe-4bee-4c6d-9a45-7d1672d59adc","Type":"ContainerStarted","Data":"62f2b905397366e6c806efe35e6edf16542daeeac8bbe9c6752ca72c4909eff4"} Jul 06 19:09:58.349727 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:58.349643 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4g4j2" event={"ID":"02f3dffe-4bee-4c6d-9a45-7d1672d59adc","Type":"ContainerStarted","Data":"376071686cfef6cb479ed4c5ca37268f0eed351241d7bb0616adf57fe4a83fee"} Jul 06 19:09:58.349727 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:58.349676 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4g4j2" event={"ID":"02f3dffe-4bee-4c6d-9a45-7d1672d59adc","Type":"ContainerStarted","Data":"bebea7f29902082653e748b6e1443b9488fb78af7e93aec8347cfede179772b7"} Jul 06 19:09:58.349727 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:58.349719 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4g4j2" event={"ID":"02f3dffe-4bee-4c6d-9a45-7d1672d59adc","Type":"ContainerStarted","Data":"f0017a33177a16b44c59cf89c3a9c5fe50ddba7a2450a140705e8c1115e56091"} Jul 06 19:09:58.349936 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:58.349757 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4g4j2" event={"ID":"02f3dffe-4bee-4c6d-9a45-7d1672d59adc","Type":"ContainerStarted","Data":"de3c7d23dfea245a40a677856b028e214acb63d702f3830a6401d1d6401a93b6"} Jul 06 19:09:58.351197 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:58.351174 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vjjbn" event={"ID":"6b947af4-1d82-43fd-9137-73d083aed449","Type":"ContainerStarted","Data":"0a4439f8e66d70304023f56520edbfe49e804e3fba2253bd8be4b6c9b775d42b"} Jul 06 19:09:58.352967 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:58.352939 2566 generic.go:356] "Generic (PLEG): container finished" podID="e9a3a524-5e1b-48db-ba03-750883088470" containerID="8b4fa6cdcd5fcf9afeb7c548076d04bef873edba5fea6fb96b4494bef38b30bb" exitCode=0 Jul 06 19:09:58.353113 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:58.353062 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-65k2z" event={"ID":"e9a3a524-5e1b-48db-ba03-750883088470","Type":"ContainerDied","Data":"8b4fa6cdcd5fcf9afeb7c548076d04bef873edba5fea6fb96b4494bef38b30bb"} Jul 06 19:09:58.353888 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:58.353792 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-fjdvf" Jul 06 19:09:58.354277 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:58.354255 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-fjdvf" Jul 06 19:09:58.360715 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:58.360659 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-8btbx" podStartSLOduration=3.691682941 podStartE2EDuration="23.360634862s" totalImagesPullingTime="19.668951921s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 19:09:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:09:58.359866002 +0000 UTC m=+23.667224954" watchObservedRunningTime="2026-07-06 19:09:58.360634862 +0000 UTC m=+23.667993810" Jul 06 19:09:58.904438 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:58.904399 2566 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 06 19:09:59.134162 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:59.133964 2566 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-06T19:09:58.904429238Z","UUID":"dcdb6183-e145-42c3-8f82-3c60ef2d7bd2","Handler":null,"Name":"","Endpoint":""} Jul 06 19:09:59.135821 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:59.135796 2566 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 06 19:09:59.135972 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:59.135838 2566 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 06 19:09:59.231087 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:59.231036 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8mnph" Jul 06 19:09:59.231305 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:59.231039 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-svgl2" Jul 06 19:09:59.231305 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:09:59.231199 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8mnph" podUID="6b76700e-1870-40a7-a1f2-ab3c1332058c" Jul 06 19:09:59.231305 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:09:59.231224 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-svgl2" podUID="e8c2f347-1744-49ff-b46e-f036d27a5e46" Jul 06 19:09:59.357677 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:59.357642 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-mznhh" event={"ID":"f1803bd2-22ae-4026-b66d-320e1c122a10","Type":"ContainerStarted","Data":"59b4c66d049d08ffe79f21f06d4a8c3ca52b7a481866863bdbd2ae2fd71ec03a"} Jul 06 19:09:59.359864 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:59.359833 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vjjbn" event={"ID":"6b947af4-1d82-43fd-9137-73d083aed449","Type":"ContainerStarted","Data":"accf86e6931b390cb8d475a491c03d046967a5429ef471abbf020ee8195b69b9"} Jul 06 19:09:59.373224 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:09:59.373166 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-mznhh" podStartSLOduration=4.989908194 podStartE2EDuration="24.373149559s" totalImagesPullingTime="19.383241365s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 19:09:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:09:59.37257468 +0000 UTC m=+24.679933629" watchObservedRunningTime="2026-07-06 19:09:59.373149559 +0000 UTC m=+24.680508547" Jul 06 19:10:00.230338 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:00.230298 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f9z4j" Jul 06 19:10:00.230567 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:10:00.230411 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f9z4j" podUID="2f2852c7-f9ef-445c-aa57-e5c697cb2c4b" Jul 06 19:10:00.364846 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:00.364806 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4g4j2" event={"ID":"02f3dffe-4bee-4c6d-9a45-7d1672d59adc","Type":"ContainerStarted","Data":"69023349107b42ef8b883bd536c5483271d7b02fdad5b8273eb47bd79f0f0797"} Jul 06 19:10:01.230378 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:01.230184 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-svgl2" Jul 06 19:10:01.230596 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:01.230250 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8mnph" Jul 06 19:10:01.230596 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:10:01.230451 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-svgl2" podUID="e8c2f347-1744-49ff-b46e-f036d27a5e46" Jul 06 19:10:01.230596 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:10:01.230521 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8mnph" podUID="6b76700e-1870-40a7-a1f2-ab3c1332058c" Jul 06 19:10:01.369403 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:01.369365 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vjjbn" event={"ID":"6b947af4-1d82-43fd-9137-73d083aed449","Type":"ContainerStarted","Data":"948ec5bf3e86a1a6b1d4b4e4802f49eea2be16112155f0cedb848db54c666cc8"} Jul 06 19:10:01.385148 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:01.385038 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vjjbn" podStartSLOduration=3.950904629 podStartE2EDuration="26.385020312s" totalImagesPullingTime="22.434115683s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 19:09:35 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:10:01.385018091 +0000 UTC m=+26.692377041" watchObservedRunningTime="2026-07-06 19:10:01.385020312 +0000 UTC m=+26.692379242" Jul 06 19:10:02.230246 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:02.230212 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f9z4j" Jul 06 19:10:02.230438 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:10:02.230324 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f9z4j" podUID="2f2852c7-f9ef-445c-aa57-e5c697cb2c4b" Jul 06 19:10:02.375919 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:02.375892 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4g4j2" event={"ID":"02f3dffe-4bee-4c6d-9a45-7d1672d59adc","Type":"ContainerStarted","Data":"482c22652fd28fcd89015ec3c9d38fc1f6103d48775a2c27867f57c2f395a3f4"} Jul 06 19:10:02.376712 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:02.376470 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-4g4j2" Jul 06 19:10:02.391887 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:02.391860 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4g4j2" Jul 06 19:10:02.399529 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:02.399487 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-4g4j2" podStartSLOduration=7.51299762 podStartE2EDuration="27.399473537s" totalImagesPullingTime="19.886475917s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 19:09:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:10:02.39856555 +0000 UTC m=+27.705924509" watchObservedRunningTime="2026-07-06 19:10:02.399473537 +0000 UTC m=+27.706832512" Jul 06 19:10:03.230902 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:03.230652 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-svgl2" Jul 06 19:10:03.231043 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:03.230652 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8mnph" Jul 06 19:10:03.231043 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:10:03.230939 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-svgl2" podUID="e8c2f347-1744-49ff-b46e-f036d27a5e46" Jul 06 19:10:03.231150 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:10:03.231051 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8mnph" podUID="6b76700e-1870-40a7-a1f2-ab3c1332058c" Jul 06 19:10:03.378542 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:03.378507 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-4g4j2" Jul 06 19:10:03.378939 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:03.378593 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-4g4j2" Jul 06 19:10:03.396269 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:03.396242 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4g4j2" Jul 06 19:10:04.230327 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:04.230295 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f9z4j" Jul 06 19:10:04.230603 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:10:04.230435 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f9z4j" podUID="2f2852c7-f9ef-445c-aa57-e5c697cb2c4b" Jul 06 19:10:04.241732 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:04.241690 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-svgl2"] Jul 06 19:10:04.241886 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:04.241823 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-svgl2" Jul 06 19:10:04.241959 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:10:04.241923 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-svgl2" podUID="e8c2f347-1744-49ff-b46e-f036d27a5e46" Jul 06 19:10:04.244793 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:04.244755 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-f9z4j"] Jul 06 19:10:04.245348 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:04.245321 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-8mnph"] Jul 06 19:10:04.245437 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:04.245427 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8mnph" Jul 06 19:10:04.245530 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:10:04.245513 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8mnph" podUID="6b76700e-1870-40a7-a1f2-ab3c1332058c" Jul 06 19:10:04.381658 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:04.381269 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f9z4j" Jul 06 19:10:04.381658 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:10:04.381393 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f9z4j" podUID="2f2852c7-f9ef-445c-aa57-e5c697cb2c4b" Jul 06 19:10:06.230155 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:06.230129 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f9z4j" Jul 06 19:10:06.230155 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:06.230151 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-svgl2" Jul 06 19:10:06.230533 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:06.230238 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8mnph" Jul 06 19:10:06.230533 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:10:06.230243 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-svgl2" podUID="e8c2f347-1744-49ff-b46e-f036d27a5e46" Jul 06 19:10:06.230533 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:10:06.230290 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f9z4j" podUID="2f2852c7-f9ef-445c-aa57-e5c697cb2c4b" Jul 06 19:10:06.230533 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:10:06.230376 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8mnph" podUID="6b76700e-1870-40a7-a1f2-ab3c1332058c" Jul 06 19:10:06.388325 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:06.388051 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-65k2z" event={"ID":"e9a3a524-5e1b-48db-ba03-750883088470","Type":"ContainerStarted","Data":"d5a8bc819e6184213b9698b83db577bfd76d3610e79846f55007a788cc84be91"} Jul 06 19:10:07.391958 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:07.391921 2566 generic.go:356] "Generic (PLEG): container finished" podID="e9a3a524-5e1b-48db-ba03-750883088470" containerID="d5a8bc819e6184213b9698b83db577bfd76d3610e79846f55007a788cc84be91" exitCode=0 Jul 06 19:10:07.392437 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:07.391984 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-65k2z" event={"ID":"e9a3a524-5e1b-48db-ba03-750883088470","Type":"ContainerDied","Data":"d5a8bc819e6184213b9698b83db577bfd76d3610e79846f55007a788cc84be91"} Jul 06 19:10:08.230921 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:08.230868 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f9z4j" Jul 06 19:10:08.230921 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:08.230919 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-svgl2" Jul 06 19:10:08.231160 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:08.230929 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8mnph" Jul 06 19:10:08.231160 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:10:08.230997 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f9z4j" podUID="2f2852c7-f9ef-445c-aa57-e5c697cb2c4b" Jul 06 19:10:08.231160 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:10:08.231063 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-svgl2" podUID="e8c2f347-1744-49ff-b46e-f036d27a5e46" Jul 06 19:10:08.231259 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:10:08.231157 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8mnph" podUID="6b76700e-1870-40a7-a1f2-ab3c1332058c" Jul 06 19:10:08.844830 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:08.844738 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6b76700e-1870-40a7-a1f2-ab3c1332058c-metrics-certs\") pod \"network-metrics-daemon-8mnph\" (UID: \"6b76700e-1870-40a7-a1f2-ab3c1332058c\") " pod="openshift-multus/network-metrics-daemon-8mnph" Jul 06 19:10:08.845303 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:10:08.844889 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 19:10:08.845303 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:10:08.844960 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6b76700e-1870-40a7-a1f2-ab3c1332058c-metrics-certs podName:6b76700e-1870-40a7-a1f2-ab3c1332058c nodeName:}" failed. No retries permitted until 2026-07-06 19:10:40.844941321 +0000 UTC m=+66.152300250 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/6b76700e-1870-40a7-a1f2-ab3c1332058c-metrics-certs") pod "network-metrics-daemon-8mnph" (UID: "6b76700e-1870-40a7-a1f2-ab3c1332058c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 06 19:10:08.946041 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:08.946007 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8sdmq\" (UniqueName: \"kubernetes.io/projected/2f2852c7-f9ef-445c-aa57-e5c697cb2c4b-kube-api-access-8sdmq\") pod \"network-check-target-f9z4j\" (UID: \"2f2852c7-f9ef-445c-aa57-e5c697cb2c4b\") " pod="openshift-network-diagnostics/network-check-target-f9z4j" Jul 06 19:10:08.946198 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:08.946055 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e8c2f347-1744-49ff-b46e-f036d27a5e46-original-pull-secret\") pod \"global-pull-secret-syncer-svgl2\" (UID: \"e8c2f347-1744-49ff-b46e-f036d27a5e46\") " pod="kube-system/global-pull-secret-syncer-svgl2" Jul 06 19:10:08.946198 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:10:08.946182 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 06 19:10:08.946341 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:10:08.946204 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 06 19:10:08.946341 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:10:08.946215 2566 projected.go:196] Error preparing data for projected volume kube-api-access-8sdmq for pod openshift-network-diagnostics/network-check-target-f9z4j: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 19:10:08.946341 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:10:08.946204 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 06 19:10:08.946341 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:10:08.946272 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2f2852c7-f9ef-445c-aa57-e5c697cb2c4b-kube-api-access-8sdmq podName:2f2852c7-f9ef-445c-aa57-e5c697cb2c4b nodeName:}" failed. No retries permitted until 2026-07-06 19:10:40.946258307 +0000 UTC m=+66.253617233 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-8sdmq" (UniqueName: "kubernetes.io/projected/2f2852c7-f9ef-445c-aa57-e5c697cb2c4b-kube-api-access-8sdmq") pod "network-check-target-f9z4j" (UID: "2f2852c7-f9ef-445c-aa57-e5c697cb2c4b") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 06 19:10:08.946341 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:10:08.946285 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e8c2f347-1744-49ff-b46e-f036d27a5e46-original-pull-secret podName:e8c2f347-1744-49ff-b46e-f036d27a5e46 nodeName:}" failed. No retries permitted until 2026-07-06 19:10:40.946279731 +0000 UTC m=+66.253638657 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e8c2f347-1744-49ff-b46e-f036d27a5e46-original-pull-secret") pod "global-pull-secret-syncer-svgl2" (UID: "e8c2f347-1744-49ff-b46e-f036d27a5e46") : object "kube-system"/"original-pull-secret" not registered Jul 06 19:10:09.398306 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:09.398274 2566 generic.go:356] "Generic (PLEG): container finished" podID="e9a3a524-5e1b-48db-ba03-750883088470" containerID="a63fc05e85884ee4fad920a5b53e9ba4dfcecc606567d60791d2caa3812eb593" exitCode=0 Jul 06 19:10:09.398472 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:09.398331 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-65k2z" event={"ID":"e9a3a524-5e1b-48db-ba03-750883088470","Type":"ContainerDied","Data":"a63fc05e85884ee4fad920a5b53e9ba4dfcecc606567d60791d2caa3812eb593"} Jul 06 19:10:10.230867 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:10.230823 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8mnph" Jul 06 19:10:10.231346 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:10.230937 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f9z4j" Jul 06 19:10:10.231346 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:10:10.231031 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-f9z4j" podUID="2f2852c7-f9ef-445c-aa57-e5c697cb2c4b" Jul 06 19:10:10.231346 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:10:10.230940 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-8mnph" podUID="6b76700e-1870-40a7-a1f2-ab3c1332058c" Jul 06 19:10:10.231346 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:10.230942 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-svgl2" Jul 06 19:10:10.231346 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:10:10.231133 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-svgl2" podUID="e8c2f347-1744-49ff-b46e-f036d27a5e46" Jul 06 19:10:10.966880 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:10.966852 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-35.ec2.internal" event="NodeReady" Jul 06 19:10:10.967046 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:10.966989 2566 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 06 19:10:11.015368 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:11.015331 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-jg5kg"] Jul 06 19:10:11.029396 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:11.029361 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-lcp9g"] Jul 06 19:10:11.029587 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:11.029418 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-jg5kg" Jul 06 19:10:11.031510 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:11.031478 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 06 19:10:11.031649 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:11.031586 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-2k95g\"" Jul 06 19:10:11.031864 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:11.031848 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 06 19:10:11.041377 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:11.041352 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-lcp9g"] Jul 06 19:10:11.041377 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:11.041383 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-jg5kg"] Jul 06 19:10:11.041538 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:11.041467 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-lcp9g" Jul 06 19:10:11.044211 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:11.044188 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 06 19:10:11.044647 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:11.044306 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 06 19:10:11.044647 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:11.044598 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-5m8hp\"" Jul 06 19:10:11.044647 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:11.044638 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 06 19:10:11.160688 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:11.160647 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0315bc99-2d7d-4988-a643-58c780793906-cert\") pod \"ingress-canary-lcp9g\" (UID: \"0315bc99-2d7d-4988-a643-58c780793906\") " pod="openshift-ingress-canary/ingress-canary-lcp9g" Jul 06 19:10:11.160688 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:11.160686 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/38cfd0e8-69ad-4b63-bd2d-74e5757ca860-metrics-tls\") pod \"dns-default-jg5kg\" (UID: \"38cfd0e8-69ad-4b63-bd2d-74e5757ca860\") " pod="openshift-dns/dns-default-jg5kg" Jul 06 19:10:11.160895 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:11.160706 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/38cfd0e8-69ad-4b63-bd2d-74e5757ca860-tmp-dir\") pod \"dns-default-jg5kg\" (UID: \"38cfd0e8-69ad-4b63-bd2d-74e5757ca860\") " pod="openshift-dns/dns-default-jg5kg" Jul 06 19:10:11.160895 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:11.160779 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f6q4x\" (UniqueName: \"kubernetes.io/projected/38cfd0e8-69ad-4b63-bd2d-74e5757ca860-kube-api-access-f6q4x\") pod \"dns-default-jg5kg\" (UID: \"38cfd0e8-69ad-4b63-bd2d-74e5757ca860\") " pod="openshift-dns/dns-default-jg5kg" Jul 06 19:10:11.160956 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:11.160898 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kxh7x\" (UniqueName: \"kubernetes.io/projected/0315bc99-2d7d-4988-a643-58c780793906-kube-api-access-kxh7x\") pod \"ingress-canary-lcp9g\" (UID: \"0315bc99-2d7d-4988-a643-58c780793906\") " pod="openshift-ingress-canary/ingress-canary-lcp9g" Jul 06 19:10:11.160956 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:11.160924 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/38cfd0e8-69ad-4b63-bd2d-74e5757ca860-config-volume\") pod \"dns-default-jg5kg\" (UID: \"38cfd0e8-69ad-4b63-bd2d-74e5757ca860\") " pod="openshift-dns/dns-default-jg5kg" Jul 06 19:10:11.261955 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:11.261867 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kxh7x\" (UniqueName: \"kubernetes.io/projected/0315bc99-2d7d-4988-a643-58c780793906-kube-api-access-kxh7x\") pod \"ingress-canary-lcp9g\" (UID: \"0315bc99-2d7d-4988-a643-58c780793906\") " pod="openshift-ingress-canary/ingress-canary-lcp9g" Jul 06 19:10:11.261955 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:11.261903 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/38cfd0e8-69ad-4b63-bd2d-74e5757ca860-config-volume\") pod \"dns-default-jg5kg\" (UID: \"38cfd0e8-69ad-4b63-bd2d-74e5757ca860\") " pod="openshift-dns/dns-default-jg5kg" Jul 06 19:10:11.261955 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:11.261944 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0315bc99-2d7d-4988-a643-58c780793906-cert\") pod \"ingress-canary-lcp9g\" (UID: \"0315bc99-2d7d-4988-a643-58c780793906\") " pod="openshift-ingress-canary/ingress-canary-lcp9g" Jul 06 19:10:11.262583 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:11.261967 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/38cfd0e8-69ad-4b63-bd2d-74e5757ca860-metrics-tls\") pod \"dns-default-jg5kg\" (UID: \"38cfd0e8-69ad-4b63-bd2d-74e5757ca860\") " pod="openshift-dns/dns-default-jg5kg" Jul 06 19:10:11.262583 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:11.261987 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/38cfd0e8-69ad-4b63-bd2d-74e5757ca860-tmp-dir\") pod \"dns-default-jg5kg\" (UID: \"38cfd0e8-69ad-4b63-bd2d-74e5757ca860\") " pod="openshift-dns/dns-default-jg5kg" Jul 06 19:10:11.262583 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:11.262008 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f6q4x\" (UniqueName: \"kubernetes.io/projected/38cfd0e8-69ad-4b63-bd2d-74e5757ca860-kube-api-access-f6q4x\") pod \"dns-default-jg5kg\" (UID: \"38cfd0e8-69ad-4b63-bd2d-74e5757ca860\") " pod="openshift-dns/dns-default-jg5kg" Jul 06 19:10:11.262583 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:10:11.262057 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 19:10:11.262583 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:10:11.262150 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/38cfd0e8-69ad-4b63-bd2d-74e5757ca860-metrics-tls podName:38cfd0e8-69ad-4b63-bd2d-74e5757ca860 nodeName:}" failed. No retries permitted until 2026-07-06 19:10:11.762129829 +0000 UTC m=+37.069488755 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/38cfd0e8-69ad-4b63-bd2d-74e5757ca860-metrics-tls") pod "dns-default-jg5kg" (UID: "38cfd0e8-69ad-4b63-bd2d-74e5757ca860") : secret "dns-default-metrics-tls" not found Jul 06 19:10:11.262583 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:11.262423 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/38cfd0e8-69ad-4b63-bd2d-74e5757ca860-tmp-dir\") pod \"dns-default-jg5kg\" (UID: \"38cfd0e8-69ad-4b63-bd2d-74e5757ca860\") " pod="openshift-dns/dns-default-jg5kg" Jul 06 19:10:11.262583 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:11.262499 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/38cfd0e8-69ad-4b63-bd2d-74e5757ca860-config-volume\") pod \"dns-default-jg5kg\" (UID: \"38cfd0e8-69ad-4b63-bd2d-74e5757ca860\") " pod="openshift-dns/dns-default-jg5kg" Jul 06 19:10:11.266564 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:11.266538 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0315bc99-2d7d-4988-a643-58c780793906-cert\") pod \"ingress-canary-lcp9g\" (UID: \"0315bc99-2d7d-4988-a643-58c780793906\") " pod="openshift-ingress-canary/ingress-canary-lcp9g" Jul 06 19:10:11.272620 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:11.272587 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f6q4x\" (UniqueName: \"kubernetes.io/projected/38cfd0e8-69ad-4b63-bd2d-74e5757ca860-kube-api-access-f6q4x\") pod \"dns-default-jg5kg\" (UID: \"38cfd0e8-69ad-4b63-bd2d-74e5757ca860\") " pod="openshift-dns/dns-default-jg5kg" Jul 06 19:10:11.272861 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:11.272839 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kxh7x\" (UniqueName: \"kubernetes.io/projected/0315bc99-2d7d-4988-a643-58c780793906-kube-api-access-kxh7x\") pod \"ingress-canary-lcp9g\" (UID: \"0315bc99-2d7d-4988-a643-58c780793906\") " pod="openshift-ingress-canary/ingress-canary-lcp9g" Jul 06 19:10:11.349928 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:11.349886 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-lcp9g" Jul 06 19:10:11.405413 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:11.405376 2566 generic.go:356] "Generic (PLEG): container finished" podID="e9a3a524-5e1b-48db-ba03-750883088470" containerID="ab1922890e33b560425161a8373df88748818e302a7d84af9b397cc186ac5cc2" exitCode=0 Jul 06 19:10:11.405600 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:11.405457 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-65k2z" event={"ID":"e9a3a524-5e1b-48db-ba03-750883088470","Type":"ContainerDied","Data":"ab1922890e33b560425161a8373df88748818e302a7d84af9b397cc186ac5cc2"} Jul 06 19:10:11.534394 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:11.534141 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-lcp9g"] Jul 06 19:10:11.538140 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:10:11.538113 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0315bc99_2d7d_4988_a643_58c780793906.slice/crio-cc0e1a0ef9e95c9264ad2b3356aa0600610046b495630e8e8dfa9e3a03d48d91 WatchSource:0}: Error finding container cc0e1a0ef9e95c9264ad2b3356aa0600610046b495630e8e8dfa9e3a03d48d91: Status 404 returned error can't find the container with id cc0e1a0ef9e95c9264ad2b3356aa0600610046b495630e8e8dfa9e3a03d48d91 Jul 06 19:10:11.765277 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:11.765232 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/38cfd0e8-69ad-4b63-bd2d-74e5757ca860-metrics-tls\") pod \"dns-default-jg5kg\" (UID: \"38cfd0e8-69ad-4b63-bd2d-74e5757ca860\") " pod="openshift-dns/dns-default-jg5kg" Jul 06 19:10:11.765440 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:10:11.765346 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 19:10:11.765440 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:10:11.765397 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/38cfd0e8-69ad-4b63-bd2d-74e5757ca860-metrics-tls podName:38cfd0e8-69ad-4b63-bd2d-74e5757ca860 nodeName:}" failed. No retries permitted until 2026-07-06 19:10:12.765383852 +0000 UTC m=+38.072742779 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/38cfd0e8-69ad-4b63-bd2d-74e5757ca860-metrics-tls") pod "dns-default-jg5kg" (UID: "38cfd0e8-69ad-4b63-bd2d-74e5757ca860") : secret "dns-default-metrics-tls" not found Jul 06 19:10:12.230390 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:12.230352 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8mnph" Jul 06 19:10:12.230779 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:12.230762 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f9z4j" Jul 06 19:10:12.231263 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:12.231242 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-svgl2" Jul 06 19:10:12.234592 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:12.234567 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 06 19:10:12.235578 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:12.235551 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 06 19:10:12.235766 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:12.235744 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-qqjhw\"" Jul 06 19:10:12.235906 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:12.235887 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 06 19:10:12.236091 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:12.236059 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 06 19:10:12.236241 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:12.236229 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-9tqph\"" Jul 06 19:10:12.409872 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:12.409826 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-lcp9g" event={"ID":"0315bc99-2d7d-4988-a643-58c780793906","Type":"ContainerStarted","Data":"cc0e1a0ef9e95c9264ad2b3356aa0600610046b495630e8e8dfa9e3a03d48d91"} Jul 06 19:10:12.773642 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:12.773604 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/38cfd0e8-69ad-4b63-bd2d-74e5757ca860-metrics-tls\") pod \"dns-default-jg5kg\" (UID: \"38cfd0e8-69ad-4b63-bd2d-74e5757ca860\") " pod="openshift-dns/dns-default-jg5kg" Jul 06 19:10:12.773830 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:10:12.773800 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 19:10:12.773893 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:10:12.773872 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/38cfd0e8-69ad-4b63-bd2d-74e5757ca860-metrics-tls podName:38cfd0e8-69ad-4b63-bd2d-74e5757ca860 nodeName:}" failed. No retries permitted until 2026-07-06 19:10:14.773856669 +0000 UTC m=+40.081215595 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/38cfd0e8-69ad-4b63-bd2d-74e5757ca860-metrics-tls") pod "dns-default-jg5kg" (UID: "38cfd0e8-69ad-4b63-bd2d-74e5757ca860") : secret "dns-default-metrics-tls" not found Jul 06 19:10:14.789021 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:14.788982 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/38cfd0e8-69ad-4b63-bd2d-74e5757ca860-metrics-tls\") pod \"dns-default-jg5kg\" (UID: \"38cfd0e8-69ad-4b63-bd2d-74e5757ca860\") " pod="openshift-dns/dns-default-jg5kg" Jul 06 19:10:14.789476 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:10:14.789117 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 19:10:14.789476 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:10:14.789182 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/38cfd0e8-69ad-4b63-bd2d-74e5757ca860-metrics-tls podName:38cfd0e8-69ad-4b63-bd2d-74e5757ca860 nodeName:}" failed. No retries permitted until 2026-07-06 19:10:18.789167238 +0000 UTC m=+44.096526165 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/38cfd0e8-69ad-4b63-bd2d-74e5757ca860-metrics-tls") pod "dns-default-jg5kg" (UID: "38cfd0e8-69ad-4b63-bd2d-74e5757ca860") : secret "dns-default-metrics-tls" not found Jul 06 19:10:15.446977 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:15.446938 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:10:15.545779 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:15.545737 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:10:15.665370 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:15.665318 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:10:16.869339 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:16.869292 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:10:16.928787 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:16.928752 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:10:17.424280 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:17.424245 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-lcp9g" event={"ID":"0315bc99-2d7d-4988-a643-58c780793906","Type":"ContainerStarted","Data":"2373175e6ef904b6d4bf4c2bfa7a86719ec83d50de7b3d683bab838857b2ca0e"} Jul 06 19:10:17.426932 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:17.426906 2566 generic.go:356] "Generic (PLEG): container finished" podID="e9a3a524-5e1b-48db-ba03-750883088470" containerID="f0e29b76abad6f41eb8fb45d5fcfebe06b8b76862f91386b740eea20235ebf75" exitCode=0 Jul 06 19:10:17.427063 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:17.426968 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-65k2z" event={"ID":"e9a3a524-5e1b-48db-ba03-750883088470","Type":"ContainerDied","Data":"f0e29b76abad6f41eb8fb45d5fcfebe06b8b76862f91386b740eea20235ebf75"} Jul 06 19:10:17.440485 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:17.440422 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-lcp9g" podStartSLOduration=2.317550752 podStartE2EDuration="7.440400709s" totalImagesPullingTime="5.122849957s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:10:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:10:17.439486618 +0000 UTC m=+42.746845565" watchObservedRunningTime="2026-07-06 19:10:17.440400709 +0000 UTC m=+42.747759656" Jul 06 19:10:17.717962 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:17.717877 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:10:17.780092 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:17.780036 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:10:17.840407 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:17.840369 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:10:18.020716 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:18.020641 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:10:18.119219 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:18.119193 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:10:18.174682 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:18.174653 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:10:18.381300 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:18.381269 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-z9c9q"] Jul 06 19:10:18.405136 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:18.405104 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-z9c9q"] Jul 06 19:10:18.405292 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:18.405206 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-z9c9q" Jul 06 19:10:18.407242 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:18.407216 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 06 19:10:18.407365 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:18.407244 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-kwlgh\"" Jul 06 19:10:18.407365 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:18.407225 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 06 19:10:18.408024 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:18.408005 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 06 19:10:18.409033 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:18.408999 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 06 19:10:18.422179 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:18.421551 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/c4564790-651c-4af2-a036-c285bb57f10a-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-z9c9q\" (UID: \"c4564790-651c-4af2-a036-c285bb57f10a\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-z9c9q" Jul 06 19:10:18.422179 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:18.421613 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-28pnl\" (UniqueName: \"kubernetes.io/projected/c4564790-651c-4af2-a036-c285bb57f10a-kube-api-access-28pnl\") pod \"cluster-monitoring-operator-64786d9cbb-z9c9q\" (UID: \"c4564790-651c-4af2-a036-c285bb57f10a\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-z9c9q" Jul 06 19:10:18.422179 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:18.421697 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c4564790-651c-4af2-a036-c285bb57f10a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-z9c9q\" (UID: \"c4564790-651c-4af2-a036-c285bb57f10a\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-z9c9q" Jul 06 19:10:18.431926 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:18.431897 2566 generic.go:356] "Generic (PLEG): container finished" podID="e9a3a524-5e1b-48db-ba03-750883088470" containerID="ee37c46a36e1a173115e3298ff971f138a7ce3594ab82ed84492585726eafd15" exitCode=0 Jul 06 19:10:18.432109 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:18.431986 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-65k2z" event={"ID":"e9a3a524-5e1b-48db-ba03-750883088470","Type":"ContainerDied","Data":"ee37c46a36e1a173115e3298ff971f138a7ce3594ab82ed84492585726eafd15"} Jul 06 19:10:18.522345 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:18.522311 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/c4564790-651c-4af2-a036-c285bb57f10a-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-z9c9q\" (UID: \"c4564790-651c-4af2-a036-c285bb57f10a\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-z9c9q" Jul 06 19:10:18.522528 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:18.522367 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-28pnl\" (UniqueName: \"kubernetes.io/projected/c4564790-651c-4af2-a036-c285bb57f10a-kube-api-access-28pnl\") pod \"cluster-monitoring-operator-64786d9cbb-z9c9q\" (UID: \"c4564790-651c-4af2-a036-c285bb57f10a\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-z9c9q" Jul 06 19:10:18.522765 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:18.522743 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c4564790-651c-4af2-a036-c285bb57f10a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-z9c9q\" (UID: \"c4564790-651c-4af2-a036-c285bb57f10a\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-z9c9q" Jul 06 19:10:18.522885 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:10:18.522871 2566 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 06 19:10:18.522951 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:10:18.522917 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c4564790-651c-4af2-a036-c285bb57f10a-cluster-monitoring-operator-tls podName:c4564790-651c-4af2-a036-c285bb57f10a nodeName:}" failed. No retries permitted until 2026-07-06 19:10:19.022904062 +0000 UTC m=+44.330262988 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/c4564790-651c-4af2-a036-c285bb57f10a-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-z9c9q" (UID: "c4564790-651c-4af2-a036-c285bb57f10a") : secret "cluster-monitoring-operator-tls" not found Jul 06 19:10:18.523115 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:18.523095 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/c4564790-651c-4af2-a036-c285bb57f10a-telemetry-config\") pod \"cluster-monitoring-operator-64786d9cbb-z9c9q\" (UID: \"c4564790-651c-4af2-a036-c285bb57f10a\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-z9c9q" Jul 06 19:10:18.533704 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:18.533668 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-28pnl\" (UniqueName: \"kubernetes.io/projected/c4564790-651c-4af2-a036-c285bb57f10a-kube-api-access-28pnl\") pod \"cluster-monitoring-operator-64786d9cbb-z9c9q\" (UID: \"c4564790-651c-4af2-a036-c285bb57f10a\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-z9c9q" Jul 06 19:10:18.584272 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:18.584239 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-666cddf4fd-nks7n"] Jul 06 19:10:18.595469 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:18.595428 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-666cddf4fd-nks7n" Jul 06 19:10:18.597674 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:18.597650 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 06 19:10:18.597871 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:18.597852 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-5jctw\"" Jul 06 19:10:18.597970 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:18.597890 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 06 19:10:18.598129 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:18.598116 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 06 19:10:18.603851 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:18.603829 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-666cddf4fd-nks7n"] Jul 06 19:10:18.610153 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:18.610128 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 06 19:10:18.623685 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:18.623653 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1ddf025a-43f4-4d0a-826e-06e5137b740f-installation-pull-secrets\") pod \"image-registry-666cddf4fd-nks7n\" (UID: \"1ddf025a-43f4-4d0a-826e-06e5137b740f\") " pod="openshift-image-registry/image-registry-666cddf4fd-nks7n" Jul 06 19:10:18.623841 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:18.623703 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1ddf025a-43f4-4d0a-826e-06e5137b740f-ca-trust-extracted\") pod \"image-registry-666cddf4fd-nks7n\" (UID: \"1ddf025a-43f4-4d0a-826e-06e5137b740f\") " pod="openshift-image-registry/image-registry-666cddf4fd-nks7n" Jul 06 19:10:18.623841 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:18.623723 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1ddf025a-43f4-4d0a-826e-06e5137b740f-trusted-ca\") pod \"image-registry-666cddf4fd-nks7n\" (UID: \"1ddf025a-43f4-4d0a-826e-06e5137b740f\") " pod="openshift-image-registry/image-registry-666cddf4fd-nks7n" Jul 06 19:10:18.623841 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:18.623797 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1ddf025a-43f4-4d0a-826e-06e5137b740f-bound-sa-token\") pod \"image-registry-666cddf4fd-nks7n\" (UID: \"1ddf025a-43f4-4d0a-826e-06e5137b740f\") " pod="openshift-image-registry/image-registry-666cddf4fd-nks7n" Jul 06 19:10:18.623841 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:18.623838 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1ddf025a-43f4-4d0a-826e-06e5137b740f-image-registry-private-configuration\") pod \"image-registry-666cddf4fd-nks7n\" (UID: \"1ddf025a-43f4-4d0a-826e-06e5137b740f\") " pod="openshift-image-registry/image-registry-666cddf4fd-nks7n" Jul 06 19:10:18.623984 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:18.623854 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1ddf025a-43f4-4d0a-826e-06e5137b740f-registry-certificates\") pod \"image-registry-666cddf4fd-nks7n\" (UID: \"1ddf025a-43f4-4d0a-826e-06e5137b740f\") " pod="openshift-image-registry/image-registry-666cddf4fd-nks7n" Jul 06 19:10:18.623984 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:18.623871 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1ddf025a-43f4-4d0a-826e-06e5137b740f-registry-tls\") pod \"image-registry-666cddf4fd-nks7n\" (UID: \"1ddf025a-43f4-4d0a-826e-06e5137b740f\") " pod="openshift-image-registry/image-registry-666cddf4fd-nks7n" Jul 06 19:10:18.623984 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:18.623885 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v5kjx\" (UniqueName: \"kubernetes.io/projected/1ddf025a-43f4-4d0a-826e-06e5137b740f-kube-api-access-v5kjx\") pod \"image-registry-666cddf4fd-nks7n\" (UID: \"1ddf025a-43f4-4d0a-826e-06e5137b740f\") " pod="openshift-image-registry/image-registry-666cddf4fd-nks7n" Jul 06 19:10:18.724232 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:18.724143 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1ddf025a-43f4-4d0a-826e-06e5137b740f-registry-tls\") pod \"image-registry-666cddf4fd-nks7n\" (UID: \"1ddf025a-43f4-4d0a-826e-06e5137b740f\") " pod="openshift-image-registry/image-registry-666cddf4fd-nks7n" Jul 06 19:10:18.724232 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:18.724200 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v5kjx\" (UniqueName: \"kubernetes.io/projected/1ddf025a-43f4-4d0a-826e-06e5137b740f-kube-api-access-v5kjx\") pod \"image-registry-666cddf4fd-nks7n\" (UID: \"1ddf025a-43f4-4d0a-826e-06e5137b740f\") " pod="openshift-image-registry/image-registry-666cddf4fd-nks7n" Jul 06 19:10:18.724232 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:18.724228 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1ddf025a-43f4-4d0a-826e-06e5137b740f-installation-pull-secrets\") pod \"image-registry-666cddf4fd-nks7n\" (UID: \"1ddf025a-43f4-4d0a-826e-06e5137b740f\") " pod="openshift-image-registry/image-registry-666cddf4fd-nks7n" Jul 06 19:10:18.724492 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:18.724264 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1ddf025a-43f4-4d0a-826e-06e5137b740f-ca-trust-extracted\") pod \"image-registry-666cddf4fd-nks7n\" (UID: \"1ddf025a-43f4-4d0a-826e-06e5137b740f\") " pod="openshift-image-registry/image-registry-666cddf4fd-nks7n" Jul 06 19:10:18.724492 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:10:18.724294 2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 19:10:18.724492 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:10:18.724320 2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-666cddf4fd-nks7n: secret "image-registry-tls" not found Jul 06 19:10:18.724492 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:18.724295 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1ddf025a-43f4-4d0a-826e-06e5137b740f-trusted-ca\") pod \"image-registry-666cddf4fd-nks7n\" (UID: \"1ddf025a-43f4-4d0a-826e-06e5137b740f\") " pod="openshift-image-registry/image-registry-666cddf4fd-nks7n" Jul 06 19:10:18.724492 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:10:18.724393 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1ddf025a-43f4-4d0a-826e-06e5137b740f-registry-tls podName:1ddf025a-43f4-4d0a-826e-06e5137b740f nodeName:}" failed. No retries permitted until 2026-07-06 19:10:19.224370657 +0000 UTC m=+44.531729587 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1ddf025a-43f4-4d0a-826e-06e5137b740f-registry-tls") pod "image-registry-666cddf4fd-nks7n" (UID: "1ddf025a-43f4-4d0a-826e-06e5137b740f") : secret "image-registry-tls" not found Jul 06 19:10:18.724738 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:18.724550 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1ddf025a-43f4-4d0a-826e-06e5137b740f-bound-sa-token\") pod \"image-registry-666cddf4fd-nks7n\" (UID: \"1ddf025a-43f4-4d0a-826e-06e5137b740f\") " pod="openshift-image-registry/image-registry-666cddf4fd-nks7n" Jul 06 19:10:18.724738 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:18.724576 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1ddf025a-43f4-4d0a-826e-06e5137b740f-image-registry-private-configuration\") pod \"image-registry-666cddf4fd-nks7n\" (UID: \"1ddf025a-43f4-4d0a-826e-06e5137b740f\") " pod="openshift-image-registry/image-registry-666cddf4fd-nks7n" Jul 06 19:10:18.724738 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:18.724595 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1ddf025a-43f4-4d0a-826e-06e5137b740f-registry-certificates\") pod \"image-registry-666cddf4fd-nks7n\" (UID: \"1ddf025a-43f4-4d0a-826e-06e5137b740f\") " pod="openshift-image-registry/image-registry-666cddf4fd-nks7n" Jul 06 19:10:18.724887 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:18.724739 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1ddf025a-43f4-4d0a-826e-06e5137b740f-ca-trust-extracted\") pod \"image-registry-666cddf4fd-nks7n\" (UID: \"1ddf025a-43f4-4d0a-826e-06e5137b740f\") " pod="openshift-image-registry/image-registry-666cddf4fd-nks7n" Jul 06 19:10:18.725153 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:18.725130 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1ddf025a-43f4-4d0a-826e-06e5137b740f-registry-certificates\") pod \"image-registry-666cddf4fd-nks7n\" (UID: \"1ddf025a-43f4-4d0a-826e-06e5137b740f\") " pod="openshift-image-registry/image-registry-666cddf4fd-nks7n" Jul 06 19:10:18.725382 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:18.725366 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1ddf025a-43f4-4d0a-826e-06e5137b740f-trusted-ca\") pod \"image-registry-666cddf4fd-nks7n\" (UID: \"1ddf025a-43f4-4d0a-826e-06e5137b740f\") " pod="openshift-image-registry/image-registry-666cddf4fd-nks7n" Jul 06 19:10:18.726945 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:18.726923 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1ddf025a-43f4-4d0a-826e-06e5137b740f-installation-pull-secrets\") pod \"image-registry-666cddf4fd-nks7n\" (UID: \"1ddf025a-43f4-4d0a-826e-06e5137b740f\") " pod="openshift-image-registry/image-registry-666cddf4fd-nks7n" Jul 06 19:10:18.727026 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:18.726961 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1ddf025a-43f4-4d0a-826e-06e5137b740f-image-registry-private-configuration\") pod \"image-registry-666cddf4fd-nks7n\" (UID: \"1ddf025a-43f4-4d0a-826e-06e5137b740f\") " pod="openshift-image-registry/image-registry-666cddf4fd-nks7n" Jul 06 19:10:18.730295 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:18.730270 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:10:18.732853 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:18.732831 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1ddf025a-43f4-4d0a-826e-06e5137b740f-bound-sa-token\") pod \"image-registry-666cddf4fd-nks7n\" (UID: \"1ddf025a-43f4-4d0a-826e-06e5137b740f\") " pod="openshift-image-registry/image-registry-666cddf4fd-nks7n" Jul 06 19:10:18.732947 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:18.732920 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v5kjx\" (UniqueName: \"kubernetes.io/projected/1ddf025a-43f4-4d0a-826e-06e5137b740f-kube-api-access-v5kjx\") pod \"image-registry-666cddf4fd-nks7n\" (UID: \"1ddf025a-43f4-4d0a-826e-06e5137b740f\") " pod="openshift-image-registry/image-registry-666cddf4fd-nks7n" Jul 06 19:10:18.825672 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:18.825633 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/38cfd0e8-69ad-4b63-bd2d-74e5757ca860-metrics-tls\") pod \"dns-default-jg5kg\" (UID: \"38cfd0e8-69ad-4b63-bd2d-74e5757ca860\") " pod="openshift-dns/dns-default-jg5kg" Jul 06 19:10:18.825825 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:10:18.825793 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 19:10:18.825868 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:10:18.825861 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/38cfd0e8-69ad-4b63-bd2d-74e5757ca860-metrics-tls podName:38cfd0e8-69ad-4b63-bd2d-74e5757ca860 nodeName:}" failed. No retries permitted until 2026-07-06 19:10:26.825847122 +0000 UTC m=+52.133206049 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/38cfd0e8-69ad-4b63-bd2d-74e5757ca860-metrics-tls") pod "dns-default-jg5kg" (UID: "38cfd0e8-69ad-4b63-bd2d-74e5757ca860") : secret "dns-default-metrics-tls" not found Jul 06 19:10:18.829923 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:18.829880 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:10:18.931444 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:18.931418 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:10:19.028132 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:19.028014 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c4564790-651c-4af2-a036-c285bb57f10a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-z9c9q\" (UID: \"c4564790-651c-4af2-a036-c285bb57f10a\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-z9c9q" Jul 06 19:10:19.028542 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:10:19.028178 2566 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 06 19:10:19.028542 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:10:19.028256 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c4564790-651c-4af2-a036-c285bb57f10a-cluster-monitoring-operator-tls podName:c4564790-651c-4af2-a036-c285bb57f10a nodeName:}" failed. No retries permitted until 2026-07-06 19:10:20.028240735 +0000 UTC m=+45.335599661 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/c4564790-651c-4af2-a036-c285bb57f10a-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-z9c9q" (UID: "c4564790-651c-4af2-a036-c285bb57f10a") : secret "cluster-monitoring-operator-tls" not found Jul 06 19:10:19.230147 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:19.230104 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1ddf025a-43f4-4d0a-826e-06e5137b740f-registry-tls\") pod \"image-registry-666cddf4fd-nks7n\" (UID: \"1ddf025a-43f4-4d0a-826e-06e5137b740f\") " pod="openshift-image-registry/image-registry-666cddf4fd-nks7n" Jul 06 19:10:19.230290 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:10:19.230225 2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 19:10:19.230290 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:10:19.230238 2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-666cddf4fd-nks7n: secret "image-registry-tls" not found Jul 06 19:10:19.230290 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:10:19.230289 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1ddf025a-43f4-4d0a-826e-06e5137b740f-registry-tls podName:1ddf025a-43f4-4d0a-826e-06e5137b740f nodeName:}" failed. No retries permitted until 2026-07-06 19:10:20.230275149 +0000 UTC m=+45.537634080 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1ddf025a-43f4-4d0a-826e-06e5137b740f-registry-tls") pod "image-registry-666cddf4fd-nks7n" (UID: "1ddf025a-43f4-4d0a-826e-06e5137b740f") : secret "image-registry-tls" not found Jul 06 19:10:19.262628 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:19.262602 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:10:19.359942 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:19.359920 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:10:19.437888 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:19.437841 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-65k2z" event={"ID":"e9a3a524-5e1b-48db-ba03-750883088470","Type":"ContainerStarted","Data":"59782d13567055b5d38296a9ed942aaa68807a8d187bd8330e2c817940895e79"} Jul 06 19:10:19.459893 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:19.459838 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-65k2z" podStartSLOduration=9.512300504 podStartE2EDuration="44.459816927s" totalImagesPullingTime="34.947516423s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 19:09:35 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:10:19.458357832 +0000 UTC m=+44.765716779" watchObservedRunningTime="2026-07-06 19:10:19.459816927 +0000 UTC m=+44.767175892" Jul 06 19:10:20.035692 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:20.035655 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c4564790-651c-4af2-a036-c285bb57f10a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-z9c9q\" (UID: \"c4564790-651c-4af2-a036-c285bb57f10a\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-z9c9q" Jul 06 19:10:20.036111 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:10:20.035811 2566 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 06 19:10:20.036111 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:10:20.035872 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c4564790-651c-4af2-a036-c285bb57f10a-cluster-monitoring-operator-tls podName:c4564790-651c-4af2-a036-c285bb57f10a nodeName:}" failed. No retries permitted until 2026-07-06 19:10:22.035856073 +0000 UTC m=+47.343215018 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/c4564790-651c-4af2-a036-c285bb57f10a-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-z9c9q" (UID: "c4564790-651c-4af2-a036-c285bb57f10a") : secret "cluster-monitoring-operator-tls" not found Jul 06 19:10:20.055031 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:20.054998 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:10:20.115250 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:20.115213 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:10:20.237260 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:20.237220 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1ddf025a-43f4-4d0a-826e-06e5137b740f-registry-tls\") pod \"image-registry-666cddf4fd-nks7n\" (UID: \"1ddf025a-43f4-4d0a-826e-06e5137b740f\") " pod="openshift-image-registry/image-registry-666cddf4fd-nks7n" Jul 06 19:10:20.237417 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:10:20.237356 2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 19:10:20.237417 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:10:20.237374 2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-666cddf4fd-nks7n: secret "image-registry-tls" not found Jul 06 19:10:20.237518 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:10:20.237427 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1ddf025a-43f4-4d0a-826e-06e5137b740f-registry-tls podName:1ddf025a-43f4-4d0a-826e-06e5137b740f nodeName:}" failed. No retries permitted until 2026-07-06 19:10:22.23741186 +0000 UTC m=+47.544770786 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1ddf025a-43f4-4d0a-826e-06e5137b740f-registry-tls") pod "image-registry-666cddf4fd-nks7n" (UID: "1ddf025a-43f4-4d0a-826e-06e5137b740f") : secret "image-registry-tls" not found Jul 06 19:10:20.301383 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:20.301314 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:10:20.484972 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:20.484939 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:10:20.540314 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:20.540280 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:10:21.254115 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:21.254084 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:10:21.310031 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:21.309995 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:10:21.667533 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:21.667497 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:10:21.734758 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:21.734694 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:10:22.050253 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:22.050219 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c4564790-651c-4af2-a036-c285bb57f10a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-z9c9q\" (UID: \"c4564790-651c-4af2-a036-c285bb57f10a\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-z9c9q" Jul 06 19:10:22.050411 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:10:22.050340 2566 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 06 19:10:22.050411 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:10:22.050394 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c4564790-651c-4af2-a036-c285bb57f10a-cluster-monitoring-operator-tls podName:c4564790-651c-4af2-a036-c285bb57f10a nodeName:}" failed. No retries permitted until 2026-07-06 19:10:26.050382005 +0000 UTC m=+51.357740931 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/c4564790-651c-4af2-a036-c285bb57f10a-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-z9c9q" (UID: "c4564790-651c-4af2-a036-c285bb57f10a") : secret "cluster-monitoring-operator-tls" not found Jul 06 19:10:22.252519 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:22.252480 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1ddf025a-43f4-4d0a-826e-06e5137b740f-registry-tls\") pod \"image-registry-666cddf4fd-nks7n\" (UID: \"1ddf025a-43f4-4d0a-826e-06e5137b740f\") " pod="openshift-image-registry/image-registry-666cddf4fd-nks7n" Jul 06 19:10:22.252690 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:10:22.252608 2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 19:10:22.252690 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:10:22.252620 2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-666cddf4fd-nks7n: secret "image-registry-tls" not found Jul 06 19:10:22.252690 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:10:22.252670 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1ddf025a-43f4-4d0a-826e-06e5137b740f-registry-tls podName:1ddf025a-43f4-4d0a-826e-06e5137b740f nodeName:}" failed. No retries permitted until 2026-07-06 19:10:26.252658567 +0000 UTC m=+51.560017494 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1ddf025a-43f4-4d0a-826e-06e5137b740f-registry-tls") pod "image-registry-666cddf4fd-nks7n" (UID: "1ddf025a-43f4-4d0a-826e-06e5137b740f") : secret "image-registry-tls" not found Jul 06 19:10:23.705491 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:23.705459 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:10:23.727018 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:23.726984 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-kgvj2"] Jul 06 19:10:23.740770 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:23.740741 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-kgvj2"] Jul 06 19:10:23.740895 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:23.740873 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-kgvj2" Jul 06 19:10:23.743259 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:23.743238 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 06 19:10:23.743397 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:23.743239 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 06 19:10:23.743870 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:23.743854 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-dr7bb\"" Jul 06 19:10:23.763987 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:23.763958 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-22s2r\" (UniqueName: \"kubernetes.io/projected/d4495041-9b50-48f1-93fc-631d2979b534-kube-api-access-22s2r\") pod \"migrator-d57d58cc9-kgvj2\" (UID: \"d4495041-9b50-48f1-93fc-631d2979b534\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-kgvj2" Jul 06 19:10:23.780674 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:23.780646 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:10:23.839912 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:23.839883 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:10:23.864500 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:23.864451 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-22s2r\" (UniqueName: \"kubernetes.io/projected/d4495041-9b50-48f1-93fc-631d2979b534-kube-api-access-22s2r\") pod \"migrator-d57d58cc9-kgvj2\" (UID: \"d4495041-9b50-48f1-93fc-631d2979b534\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-kgvj2" Jul 06 19:10:23.872193 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:23.872163 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-22s2r\" (UniqueName: \"kubernetes.io/projected/d4495041-9b50-48f1-93fc-631d2979b534-kube-api-access-22s2r\") pod \"migrator-d57d58cc9-kgvj2\" (UID: \"d4495041-9b50-48f1-93fc-631d2979b534\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-kgvj2" Jul 06 19:10:24.050322 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:24.050289 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-kgvj2" Jul 06 19:10:24.193767 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:24.193734 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-kgvj2"] Jul 06 19:10:24.197443 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:10:24.197418 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4495041_9b50_48f1_93fc_631d2979b534.slice/crio-2e3de1b0937db9102afaac796c6e6dfe21da83627028d7f05d334146f9e458df WatchSource:0}: Error finding container 2e3de1b0937db9102afaac796c6e6dfe21da83627028d7f05d334146f9e458df: Status 404 returned error can't find the container with id 2e3de1b0937db9102afaac796c6e6dfe21da83627028d7f05d334146f9e458df Jul 06 19:10:24.419242 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:24.419163 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-hqnrn"] Jul 06 19:10:24.450166 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:24.450131 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-hqnrn"] Jul 06 19:10:24.450332 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:24.450278 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-hqnrn" Jul 06 19:10:24.450381 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:24.450342 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-kgvj2" event={"ID":"d4495041-9b50-48f1-93fc-631d2979b534","Type":"ContainerStarted","Data":"2e3de1b0937db9102afaac796c6e6dfe21da83627028d7f05d334146f9e458df"} Jul 06 19:10:24.452535 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:24.452478 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 06 19:10:24.452776 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:24.452716 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-jdr5p\"" Jul 06 19:10:24.452776 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:24.452770 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 06 19:10:24.453057 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:24.453041 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 06 19:10:24.453452 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:24.453430 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 06 19:10:24.461403 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:24.461379 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:10:24.469940 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:24.469919 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/1a6ef785-86f6-412f-8ef3-300db1b24c2c-data-volume\") pod \"insights-runtime-extractor-hqnrn\" (UID: \"1a6ef785-86f6-412f-8ef3-300db1b24c2c\") " pod="openshift-insights/insights-runtime-extractor-hqnrn" Jul 06 19:10:24.470026 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:24.469955 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1a6ef785-86f6-412f-8ef3-300db1b24c2c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hqnrn\" (UID: \"1a6ef785-86f6-412f-8ef3-300db1b24c2c\") " pod="openshift-insights/insights-runtime-extractor-hqnrn" Jul 06 19:10:24.470026 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:24.469987 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/1a6ef785-86f6-412f-8ef3-300db1b24c2c-crio-socket\") pod \"insights-runtime-extractor-hqnrn\" (UID: \"1a6ef785-86f6-412f-8ef3-300db1b24c2c\") " pod="openshift-insights/insights-runtime-extractor-hqnrn" Jul 06 19:10:24.470115 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:24.470057 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lrxv4\" (UniqueName: \"kubernetes.io/projected/1a6ef785-86f6-412f-8ef3-300db1b24c2c-kube-api-access-lrxv4\") pod \"insights-runtime-extractor-hqnrn\" (UID: \"1a6ef785-86f6-412f-8ef3-300db1b24c2c\") " pod="openshift-insights/insights-runtime-extractor-hqnrn" Jul 06 19:10:24.470115 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:24.470107 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1a6ef785-86f6-412f-8ef3-300db1b24c2c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hqnrn\" (UID: \"1a6ef785-86f6-412f-8ef3-300db1b24c2c\") " pod="openshift-insights/insights-runtime-extractor-hqnrn" Jul 06 19:10:24.534553 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:24.534523 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:10:24.571291 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:24.571254 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1a6ef785-86f6-412f-8ef3-300db1b24c2c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hqnrn\" (UID: \"1a6ef785-86f6-412f-8ef3-300db1b24c2c\") " pod="openshift-insights/insights-runtime-extractor-hqnrn" Jul 06 19:10:24.571464 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:24.571295 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/1a6ef785-86f6-412f-8ef3-300db1b24c2c-crio-socket\") pod \"insights-runtime-extractor-hqnrn\" (UID: \"1a6ef785-86f6-412f-8ef3-300db1b24c2c\") " pod="openshift-insights/insights-runtime-extractor-hqnrn" Jul 06 19:10:24.571464 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:10:24.571415 2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 19:10:24.571464 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:24.571436 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lrxv4\" (UniqueName: \"kubernetes.io/projected/1a6ef785-86f6-412f-8ef3-300db1b24c2c-kube-api-access-lrxv4\") pod \"insights-runtime-extractor-hqnrn\" (UID: \"1a6ef785-86f6-412f-8ef3-300db1b24c2c\") " pod="openshift-insights/insights-runtime-extractor-hqnrn" Jul 06 19:10:24.571608 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:24.571473 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/1a6ef785-86f6-412f-8ef3-300db1b24c2c-crio-socket\") pod \"insights-runtime-extractor-hqnrn\" (UID: \"1a6ef785-86f6-412f-8ef3-300db1b24c2c\") " pod="openshift-insights/insights-runtime-extractor-hqnrn" Jul 06 19:10:24.571608 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:10:24.571484 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1a6ef785-86f6-412f-8ef3-300db1b24c2c-insights-runtime-extractor-tls podName:1a6ef785-86f6-412f-8ef3-300db1b24c2c nodeName:}" failed. No retries permitted until 2026-07-06 19:10:25.071468185 +0000 UTC m=+50.378827111 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/1a6ef785-86f6-412f-8ef3-300db1b24c2c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hqnrn" (UID: "1a6ef785-86f6-412f-8ef3-300db1b24c2c") : secret "insights-runtime-extractor-tls" not found Jul 06 19:10:24.571608 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:24.571500 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1a6ef785-86f6-412f-8ef3-300db1b24c2c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hqnrn\" (UID: \"1a6ef785-86f6-412f-8ef3-300db1b24c2c\") " pod="openshift-insights/insights-runtime-extractor-hqnrn" Jul 06 19:10:24.571608 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:24.571602 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/1a6ef785-86f6-412f-8ef3-300db1b24c2c-data-volume\") pod \"insights-runtime-extractor-hqnrn\" (UID: \"1a6ef785-86f6-412f-8ef3-300db1b24c2c\") " pod="openshift-insights/insights-runtime-extractor-hqnrn" Jul 06 19:10:24.571747 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:10:24.571601 2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 19:10:24.571747 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:10:24.571668 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1a6ef785-86f6-412f-8ef3-300db1b24c2c-kube-rbac-proxy-cm podName:1a6ef785-86f6-412f-8ef3-300db1b24c2c nodeName:}" failed. No retries permitted until 2026-07-06 19:10:25.071659131 +0000 UTC m=+50.379018092 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/1a6ef785-86f6-412f-8ef3-300db1b24c2c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hqnrn" (UID: "1a6ef785-86f6-412f-8ef3-300db1b24c2c") : configmap "kube-rbac-proxy" not found Jul 06 19:10:24.571923 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:24.571906 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/1a6ef785-86f6-412f-8ef3-300db1b24c2c-data-volume\") pod \"insights-runtime-extractor-hqnrn\" (UID: \"1a6ef785-86f6-412f-8ef3-300db1b24c2c\") " pod="openshift-insights/insights-runtime-extractor-hqnrn" Jul 06 19:10:24.581849 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:24.581812 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lrxv4\" (UniqueName: \"kubernetes.io/projected/1a6ef785-86f6-412f-8ef3-300db1b24c2c-kube-api-access-lrxv4\") pod \"insights-runtime-extractor-hqnrn\" (UID: \"1a6ef785-86f6-412f-8ef3-300db1b24c2c\") " pod="openshift-insights/insights-runtime-extractor-hqnrn" Jul 06 19:10:25.075856 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:25.075820 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1a6ef785-86f6-412f-8ef3-300db1b24c2c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hqnrn\" (UID: \"1a6ef785-86f6-412f-8ef3-300db1b24c2c\") " pod="openshift-insights/insights-runtime-extractor-hqnrn" Jul 06 19:10:25.076336 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:25.075887 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1a6ef785-86f6-412f-8ef3-300db1b24c2c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hqnrn\" (UID: \"1a6ef785-86f6-412f-8ef3-300db1b24c2c\") " pod="openshift-insights/insights-runtime-extractor-hqnrn" Jul 06 19:10:25.076336 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:10:25.075976 2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 06 19:10:25.076336 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:10:25.075981 2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 19:10:25.076336 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:10:25.076062 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1a6ef785-86f6-412f-8ef3-300db1b24c2c-kube-rbac-proxy-cm podName:1a6ef785-86f6-412f-8ef3-300db1b24c2c nodeName:}" failed. No retries permitted until 2026-07-06 19:10:26.076040684 +0000 UTC m=+51.383399610 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/1a6ef785-86f6-412f-8ef3-300db1b24c2c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hqnrn" (UID: "1a6ef785-86f6-412f-8ef3-300db1b24c2c") : configmap "kube-rbac-proxy" not found Jul 06 19:10:25.076336 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:10:25.076153 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1a6ef785-86f6-412f-8ef3-300db1b24c2c-insights-runtime-extractor-tls podName:1a6ef785-86f6-412f-8ef3-300db1b24c2c nodeName:}" failed. No retries permitted until 2026-07-06 19:10:26.076135862 +0000 UTC m=+51.383494789 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/1a6ef785-86f6-412f-8ef3-300db1b24c2c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hqnrn" (UID: "1a6ef785-86f6-412f-8ef3-300db1b24c2c") : secret "insights-runtime-extractor-tls" not found Jul 06 19:10:25.515772 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:25.515681 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:10:25.938894 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:25.938866 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:10:26.039815 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:26.039787 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:10:26.084601 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:26.084562 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c4564790-651c-4af2-a036-c285bb57f10a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-z9c9q\" (UID: \"c4564790-651c-4af2-a036-c285bb57f10a\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-z9c9q" Jul 06 19:10:26.084967 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:26.084619 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1a6ef785-86f6-412f-8ef3-300db1b24c2c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hqnrn\" (UID: \"1a6ef785-86f6-412f-8ef3-300db1b24c2c\") " pod="openshift-insights/insights-runtime-extractor-hqnrn" Jul 06 19:10:26.084967 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:26.084662 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1a6ef785-86f6-412f-8ef3-300db1b24c2c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hqnrn\" (UID: \"1a6ef785-86f6-412f-8ef3-300db1b24c2c\") " pod="openshift-insights/insights-runtime-extractor-hqnrn" Jul 06 19:10:26.084967 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:10:26.084708 2566 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 06 19:10:26.084967 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:10:26.084774 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c4564790-651c-4af2-a036-c285bb57f10a-cluster-monitoring-operator-tls podName:c4564790-651c-4af2-a036-c285bb57f10a nodeName:}" failed. No retries permitted until 2026-07-06 19:10:34.084756026 +0000 UTC m=+59.392114953 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/c4564790-651c-4af2-a036-c285bb57f10a-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-z9c9q" (UID: "c4564790-651c-4af2-a036-c285bb57f10a") : secret "cluster-monitoring-operator-tls" not found Jul 06 19:10:26.084967 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:10:26.084778 2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 19:10:26.084967 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:10:26.084832 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1a6ef785-86f6-412f-8ef3-300db1b24c2c-insights-runtime-extractor-tls podName:1a6ef785-86f6-412f-8ef3-300db1b24c2c nodeName:}" failed. No retries permitted until 2026-07-06 19:10:28.084819288 +0000 UTC m=+53.392178214 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/1a6ef785-86f6-412f-8ef3-300db1b24c2c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hqnrn" (UID: "1a6ef785-86f6-412f-8ef3-300db1b24c2c") : secret "insights-runtime-extractor-tls" not found Jul 06 19:10:26.085180 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:26.085130 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/1a6ef785-86f6-412f-8ef3-300db1b24c2c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hqnrn\" (UID: \"1a6ef785-86f6-412f-8ef3-300db1b24c2c\") " pod="openshift-insights/insights-runtime-extractor-hqnrn" Jul 06 19:10:26.216957 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:26.216859 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:10:26.287009 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:26.286973 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1ddf025a-43f4-4d0a-826e-06e5137b740f-registry-tls\") pod \"image-registry-666cddf4fd-nks7n\" (UID: \"1ddf025a-43f4-4d0a-826e-06e5137b740f\") " pod="openshift-image-registry/image-registry-666cddf4fd-nks7n" Jul 06 19:10:26.287227 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:10:26.287161 2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 19:10:26.287227 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:10:26.287180 2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-666cddf4fd-nks7n: secret "image-registry-tls" not found Jul 06 19:10:26.287297 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:10:26.287242 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1ddf025a-43f4-4d0a-826e-06e5137b740f-registry-tls podName:1ddf025a-43f4-4d0a-826e-06e5137b740f nodeName:}" failed. No retries permitted until 2026-07-06 19:10:34.287225631 +0000 UTC m=+59.594584557 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1ddf025a-43f4-4d0a-826e-06e5137b740f-registry-tls") pod "image-registry-666cddf4fd-nks7n" (UID: "1ddf025a-43f4-4d0a-826e-06e5137b740f") : secret "image-registry-tls" not found Jul 06 19:10:26.315827 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:26.315789 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:10:26.457579 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:26.457531 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-kgvj2" event={"ID":"d4495041-9b50-48f1-93fc-631d2979b534","Type":"ContainerStarted","Data":"21e478b011e1353d281cd06707e90ffa094acccd54065362575dcc794e56ed94"} Jul 06 19:10:26.457722 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:26.457594 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-kgvj2" event={"ID":"d4495041-9b50-48f1-93fc-631d2979b534","Type":"ContainerStarted","Data":"847e625d7be94f30550718ea69d12b600d0a2ea46793e376347d9d71f429f93d"} Jul 06 19:10:26.549663 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:26.549635 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:10:26.891276 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:26.891188 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/38cfd0e8-69ad-4b63-bd2d-74e5757ca860-metrics-tls\") pod \"dns-default-jg5kg\" (UID: \"38cfd0e8-69ad-4b63-bd2d-74e5757ca860\") " pod="openshift-dns/dns-default-jg5kg" Jul 06 19:10:26.891569 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:10:26.891298 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 06 19:10:26.891569 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:10:26.891349 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/38cfd0e8-69ad-4b63-bd2d-74e5757ca860-metrics-tls podName:38cfd0e8-69ad-4b63-bd2d-74e5757ca860 nodeName:}" failed. No retries permitted until 2026-07-06 19:10:42.89133648 +0000 UTC m=+68.198695407 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/38cfd0e8-69ad-4b63-bd2d-74e5757ca860-metrics-tls") pod "dns-default-jg5kg" (UID: "38cfd0e8-69ad-4b63-bd2d-74e5757ca860") : secret "dns-default-metrics-tls" not found Jul 06 19:10:27.412528 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:27.412493 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-8btbx_c75820ce-6423-4c3f-8e13-0ab0e5b00914/dns-node-resolver/0.log" Jul 06 19:10:27.819606 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:27.819549 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-kgvj2" podStartSLOduration=2.957493232 podStartE2EDuration="4.819531951s" totalImagesPullingTime="1.862038719s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:10:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:10:26.475494213 +0000 UTC m=+51.782853161" watchObservedRunningTime="2026-07-06 19:10:27.819531951 +0000 UTC m=+53.126890958" Jul 06 19:10:27.820363 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:27.820341 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-758f599c5-rvswn"] Jul 06 19:10:27.822213 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:27.822195 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-rvswn" Jul 06 19:10:27.824291 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:27.824269 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 06 19:10:27.824392 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:27.824328 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 06 19:10:27.824392 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:27.824355 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 06 19:10:27.824983 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:27.824964 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 06 19:10:27.825038 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:27.824969 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-5dg4j\"" Jul 06 19:10:27.832505 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:27.832474 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-rvswn"] Jul 06 19:10:27.856348 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:27.856320 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:10:27.898780 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:27.898749 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/b5e14011-0695-4515-b2e4-f162fe6c16fe-signing-key\") pod \"service-ca-758f599c5-rvswn\" (UID: \"b5e14011-0695-4515-b2e4-f162fe6c16fe\") " pod="openshift-service-ca/service-ca-758f599c5-rvswn" Jul 06 19:10:27.898902 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:27.898790 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/b5e14011-0695-4515-b2e4-f162fe6c16fe-signing-cabundle\") pod \"service-ca-758f599c5-rvswn\" (UID: \"b5e14011-0695-4515-b2e4-f162fe6c16fe\") " pod="openshift-service-ca/service-ca-758f599c5-rvswn" Jul 06 19:10:27.898902 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:27.898884 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mfjbp\" (UniqueName: \"kubernetes.io/projected/b5e14011-0695-4515-b2e4-f162fe6c16fe-kube-api-access-mfjbp\") pod \"service-ca-758f599c5-rvswn\" (UID: \"b5e14011-0695-4515-b2e4-f162fe6c16fe\") " pod="openshift-service-ca/service-ca-758f599c5-rvswn" Jul 06 19:10:27.912435 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:27.912400 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:10:28.000051 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:27.999997 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mfjbp\" (UniqueName: \"kubernetes.io/projected/b5e14011-0695-4515-b2e4-f162fe6c16fe-kube-api-access-mfjbp\") pod \"service-ca-758f599c5-rvswn\" (UID: \"b5e14011-0695-4515-b2e4-f162fe6c16fe\") " pod="openshift-service-ca/service-ca-758f599c5-rvswn" Jul 06 19:10:28.000189 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:28.000122 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/b5e14011-0695-4515-b2e4-f162fe6c16fe-signing-key\") pod \"service-ca-758f599c5-rvswn\" (UID: \"b5e14011-0695-4515-b2e4-f162fe6c16fe\") " pod="openshift-service-ca/service-ca-758f599c5-rvswn" Jul 06 19:10:28.000189 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:28.000146 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/b5e14011-0695-4515-b2e4-f162fe6c16fe-signing-cabundle\") pod \"service-ca-758f599c5-rvswn\" (UID: \"b5e14011-0695-4515-b2e4-f162fe6c16fe\") " pod="openshift-service-ca/service-ca-758f599c5-rvswn" Jul 06 19:10:28.000798 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:28.000779 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/b5e14011-0695-4515-b2e4-f162fe6c16fe-signing-cabundle\") pod \"service-ca-758f599c5-rvswn\" (UID: \"b5e14011-0695-4515-b2e4-f162fe6c16fe\") " pod="openshift-service-ca/service-ca-758f599c5-rvswn" Jul 06 19:10:28.002612 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:28.002594 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/b5e14011-0695-4515-b2e4-f162fe6c16fe-signing-key\") pod \"service-ca-758f599c5-rvswn\" (UID: \"b5e14011-0695-4515-b2e4-f162fe6c16fe\") " pod="openshift-service-ca/service-ca-758f599c5-rvswn" Jul 06 19:10:28.007720 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:28.007698 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mfjbp\" (UniqueName: \"kubernetes.io/projected/b5e14011-0695-4515-b2e4-f162fe6c16fe-kube-api-access-mfjbp\") pod \"service-ca-758f599c5-rvswn\" (UID: \"b5e14011-0695-4515-b2e4-f162fe6c16fe\") " pod="openshift-service-ca/service-ca-758f599c5-rvswn" Jul 06 19:10:28.101171 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:28.101050 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1a6ef785-86f6-412f-8ef3-300db1b24c2c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hqnrn\" (UID: \"1a6ef785-86f6-412f-8ef3-300db1b24c2c\") " pod="openshift-insights/insights-runtime-extractor-hqnrn" Jul 06 19:10:28.101327 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:10:28.101211 2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 19:10:28.101327 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:10:28.101282 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1a6ef785-86f6-412f-8ef3-300db1b24c2c-insights-runtime-extractor-tls podName:1a6ef785-86f6-412f-8ef3-300db1b24c2c nodeName:}" failed. No retries permitted until 2026-07-06 19:10:32.101266438 +0000 UTC m=+57.408625365 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/1a6ef785-86f6-412f-8ef3-300db1b24c2c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hqnrn" (UID: "1a6ef785-86f6-412f-8ef3-300db1b24c2c") : secret "insights-runtime-extractor-tls" not found Jul 06 19:10:28.130772 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:28.130726 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-rvswn" Jul 06 19:10:28.212236 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:28.212209 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-qbwhx_e37e30a5-e12f-4c79-a1ff-28c9460d44b3/node-ca/0.log" Jul 06 19:10:28.264244 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:28.264213 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-rvswn"] Jul 06 19:10:28.267537 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:10:28.267508 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5e14011_0695_4515_b2e4_f162fe6c16fe.slice/crio-b5579366a8c93bde851e2a12e467612a40df2cf02e42d97d1502fff604ad479a WatchSource:0}: Error finding container b5579366a8c93bde851e2a12e467612a40df2cf02e42d97d1502fff604ad479a: Status 404 returned error can't find the container with id b5579366a8c93bde851e2a12e467612a40df2cf02e42d97d1502fff604ad479a Jul 06 19:10:28.463130 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:28.463029 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-rvswn" event={"ID":"b5e14011-0695-4515-b2e4-f162fe6c16fe","Type":"ContainerStarted","Data":"b5579366a8c93bde851e2a12e467612a40df2cf02e42d97d1502fff604ad479a"} Jul 06 19:10:29.144657 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:29.144623 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:10:29.213646 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:29.213613 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-lcp9g_0315bc99-2d7d-4988-a643-58c780793906/serve-healthcheck-canary/0.log" Jul 06 19:10:29.413148 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:29.413076 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-kgvj2_d4495041-9b50-48f1-93fc-631d2979b534/migrator/0.log" Jul 06 19:10:29.612998 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:29.612965 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-kgvj2_d4495041-9b50-48f1-93fc-631d2979b534/graceful-termination/0.log" Jul 06 19:10:31.472265 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:31.472230 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-rvswn" event={"ID":"b5e14011-0695-4515-b2e4-f162fe6c16fe","Type":"ContainerStarted","Data":"ea05451aec9c7bc39fa2b8768baf9e7f59345d6cada4de3c0547a6654adf592e"} Jul 06 19:10:31.491306 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:31.489893 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-758f599c5-rvswn" podStartSLOduration=2.195790372 podStartE2EDuration="4.489870634s" totalImagesPullingTime="2.294080262s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:10:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:10:31.487675545 +0000 UTC m=+56.795034494" watchObservedRunningTime="2026-07-06 19:10:31.489870634 +0000 UTC m=+56.797229582" Jul 06 19:10:31.675195 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:31.675165 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:10:31.779537 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:31.779441 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:10:31.880895 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:31.880838 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:10:31.976540 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:31.976500 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:10:32.078193 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:32.078158 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:10:32.135660 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:32.135619 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1a6ef785-86f6-412f-8ef3-300db1b24c2c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hqnrn\" (UID: \"1a6ef785-86f6-412f-8ef3-300db1b24c2c\") " pod="openshift-insights/insights-runtime-extractor-hqnrn" Jul 06 19:10:32.135870 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:10:32.135790 2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 06 19:10:32.135944 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:10:32.135874 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1a6ef785-86f6-412f-8ef3-300db1b24c2c-insights-runtime-extractor-tls podName:1a6ef785-86f6-412f-8ef3-300db1b24c2c nodeName:}" failed. No retries permitted until 2026-07-06 19:10:40.135850047 +0000 UTC m=+65.443208988 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/1a6ef785-86f6-412f-8ef3-300db1b24c2c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hqnrn" (UID: "1a6ef785-86f6-412f-8ef3-300db1b24c2c") : secret "insights-runtime-extractor-tls" not found Jul 06 19:10:32.376059 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:32.375973 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:10:33.382060 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:33.382031 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:10:33.782038 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:33.781953 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:10:33.984701 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:33.984669 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:10:34.154487 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:34.154435 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c4564790-651c-4af2-a036-c285bb57f10a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-z9c9q\" (UID: \"c4564790-651c-4af2-a036-c285bb57f10a\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-z9c9q" Jul 06 19:10:34.154666 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:10:34.154608 2566 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 06 19:10:34.154710 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:10:34.154689 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c4564790-651c-4af2-a036-c285bb57f10a-cluster-monitoring-operator-tls podName:c4564790-651c-4af2-a036-c285bb57f10a nodeName:}" failed. No retries permitted until 2026-07-06 19:10:50.154669331 +0000 UTC m=+75.462028262 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/c4564790-651c-4af2-a036-c285bb57f10a-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-64786d9cbb-z9c9q" (UID: "c4564790-651c-4af2-a036-c285bb57f10a") : secret "cluster-monitoring-operator-tls" not found Jul 06 19:10:34.176689 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:34.176656 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:10:34.356531 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:34.356472 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1ddf025a-43f4-4d0a-826e-06e5137b740f-registry-tls\") pod \"image-registry-666cddf4fd-nks7n\" (UID: \"1ddf025a-43f4-4d0a-826e-06e5137b740f\") " pod="openshift-image-registry/image-registry-666cddf4fd-nks7n" Jul 06 19:10:34.356708 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:10:34.356651 2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 06 19:10:34.356708 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:10:34.356677 2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-666cddf4fd-nks7n: secret "image-registry-tls" not found Jul 06 19:10:34.356778 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:10:34.356759 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1ddf025a-43f4-4d0a-826e-06e5137b740f-registry-tls podName:1ddf025a-43f4-4d0a-826e-06e5137b740f nodeName:}" failed. No retries permitted until 2026-07-06 19:10:50.356737648 +0000 UTC m=+75.664096587 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/1ddf025a-43f4-4d0a-826e-06e5137b740f-registry-tls") pod "image-registry-666cddf4fd-nks7n" (UID: "1ddf025a-43f4-4d0a-826e-06e5137b740f") : secret "image-registry-tls" not found Jul 06 19:10:34.381515 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:34.381469 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:10:35.211037 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:35.211006 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:10:35.310744 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:35.310713 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:10:35.394298 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:35.394274 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4g4j2" Jul 06 19:10:35.410792 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:35.410751 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:10:35.469388 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:35.469301 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:10:35.783300 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:35.783219 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:10:35.873629 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:35.873602 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:10:35.990116 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:35.990091 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:10:36.174901 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:36.174865 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:10:36.231573 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:36.231542 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:10:36.574781 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:36.574747 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:10:37.373903 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:37.373866 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:10:37.582575 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:37.582539 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:10:37.988295 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:37.988262 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:10:38.041589 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:38.041562 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:10:38.172319 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:38.172281 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:10:38.379184 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:38.379148 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:10:39.578744 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:39.578705 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:10:40.054314 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:40.054284 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:10:40.118337 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:40.118301 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:10:40.187607 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:40.187570 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:10:40.199632 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:40.199591 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1a6ef785-86f6-412f-8ef3-300db1b24c2c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hqnrn\" (UID: \"1a6ef785-86f6-412f-8ef3-300db1b24c2c\") " pod="openshift-insights/insights-runtime-extractor-hqnrn" Jul 06 19:10:40.202281 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:40.202247 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/1a6ef785-86f6-412f-8ef3-300db1b24c2c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hqnrn\" (UID: \"1a6ef785-86f6-412f-8ef3-300db1b24c2c\") " pod="openshift-insights/insights-runtime-extractor-hqnrn" Jul 06 19:10:40.364430 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:40.364344 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-jdr5p\"" Jul 06 19:10:40.373166 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:40.373130 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-hqnrn" Jul 06 19:10:40.376908 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:40.376879 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:10:40.511089 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:40.511044 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-hqnrn"] Jul 06 19:10:40.514550 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:10:40.514520 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a6ef785_86f6_412f_8ef3_300db1b24c2c.slice/crio-b50700029667d6d9371d8ab790902e6968b7409e1fc313a7bc16abff4b0472e6 WatchSource:0}: Error finding container b50700029667d6d9371d8ab790902e6968b7409e1fc313a7bc16abff4b0472e6: Status 404 returned error can't find the container with id b50700029667d6d9371d8ab790902e6968b7409e1fc313a7bc16abff4b0472e6 Jul 06 19:10:40.904390 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:40.904354 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6b76700e-1870-40a7-a1f2-ab3c1332058c-metrics-certs\") pod \"network-metrics-daemon-8mnph\" (UID: \"6b76700e-1870-40a7-a1f2-ab3c1332058c\") " pod="openshift-multus/network-metrics-daemon-8mnph" Jul 06 19:10:40.906916 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:40.906897 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 06 19:10:40.917319 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:40.917278 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6b76700e-1870-40a7-a1f2-ab3c1332058c-metrics-certs\") pod \"network-metrics-daemon-8mnph\" (UID: \"6b76700e-1870-40a7-a1f2-ab3c1332058c\") " pod="openshift-multus/network-metrics-daemon-8mnph" Jul 06 19:10:41.005370 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:41.005335 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8sdmq\" (UniqueName: \"kubernetes.io/projected/2f2852c7-f9ef-445c-aa57-e5c697cb2c4b-kube-api-access-8sdmq\") pod \"network-check-target-f9z4j\" (UID: \"2f2852c7-f9ef-445c-aa57-e5c697cb2c4b\") " pod="openshift-network-diagnostics/network-check-target-f9z4j" Jul 06 19:10:41.005539 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:41.005391 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e8c2f347-1744-49ff-b46e-f036d27a5e46-original-pull-secret\") pod \"global-pull-secret-syncer-svgl2\" (UID: \"e8c2f347-1744-49ff-b46e-f036d27a5e46\") " pod="kube-system/global-pull-secret-syncer-svgl2" Jul 06 19:10:41.007985 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:41.007963 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 06 19:10:41.008041 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:41.008025 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 06 19:10:41.018579 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:41.018555 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 06 19:10:41.019006 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:41.018986 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e8c2f347-1744-49ff-b46e-f036d27a5e46-original-pull-secret\") pod \"global-pull-secret-syncer-svgl2\" (UID: \"e8c2f347-1744-49ff-b46e-f036d27a5e46\") " pod="kube-system/global-pull-secret-syncer-svgl2" Jul 06 19:10:41.030091 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:41.030051 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8sdmq\" (UniqueName: \"kubernetes.io/projected/2f2852c7-f9ef-445c-aa57-e5c697cb2c4b-kube-api-access-8sdmq\") pod \"network-check-target-f9z4j\" (UID: \"2f2852c7-f9ef-445c-aa57-e5c697cb2c4b\") " pod="openshift-network-diagnostics/network-check-target-f9z4j" Jul 06 19:10:41.046419 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:41.046393 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-qqjhw\"" Jul 06 19:10:41.054480 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:41.054450 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-8mnph" Jul 06 19:10:41.055631 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:41.055610 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-9tqph\"" Jul 06 19:10:41.058393 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:41.058371 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-svgl2" Jul 06 19:10:41.064364 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:41.064338 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-f9z4j" Jul 06 19:10:41.177474 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:41.177439 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:10:41.203802 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:41.203770 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-8mnph"] Jul 06 19:10:41.207536 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:10:41.207500 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b76700e_1870_40a7_a1f2_ab3c1332058c.slice/crio-f287f6c2364ac19965bb54dcae0e508625e65bd5773404666ec68d4488733c06 WatchSource:0}: Error finding container f287f6c2364ac19965bb54dcae0e508625e65bd5773404666ec68d4488733c06: Status 404 returned error can't find the container with id f287f6c2364ac19965bb54dcae0e508625e65bd5773404666ec68d4488733c06 Jul 06 19:10:41.431262 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:41.431130 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-svgl2"] Jul 06 19:10:41.431262 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:41.431188 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-f9z4j"] Jul 06 19:10:41.435450 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:10:41.435318 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f2852c7_f9ef_445c_aa57_e5c697cb2c4b.slice/crio-0a17b274ef1bc4bb15e7050f39d4e48398e0cd7ced7e08b12d0e1f81c9c56215 WatchSource:0}: Error finding container 0a17b274ef1bc4bb15e7050f39d4e48398e0cd7ced7e08b12d0e1f81c9c56215: Status 404 returned error can't find the container with id 0a17b274ef1bc4bb15e7050f39d4e48398e0cd7ced7e08b12d0e1f81c9c56215 Jul 06 19:10:41.436290 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:10:41.436162 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8c2f347_1744_49ff_b46e_f036d27a5e46.slice/crio-2daab486dcbddde2a703d769c0a095398537627f3242ed4eb894a5bbdc18b454 WatchSource:0}: Error finding container 2daab486dcbddde2a703d769c0a095398537627f3242ed4eb894a5bbdc18b454: Status 404 returned error can't find the container with id 2daab486dcbddde2a703d769c0a095398537627f3242ed4eb894a5bbdc18b454 Jul 06 19:10:41.500465 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:41.500415 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-svgl2" event={"ID":"e8c2f347-1744-49ff-b46e-f036d27a5e46","Type":"ContainerStarted","Data":"2daab486dcbddde2a703d769c0a095398537627f3242ed4eb894a5bbdc18b454"} Jul 06 19:10:41.502929 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:41.502892 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hqnrn" event={"ID":"1a6ef785-86f6-412f-8ef3-300db1b24c2c","Type":"ContainerStarted","Data":"b50700029667d6d9371d8ab790902e6968b7409e1fc313a7bc16abff4b0472e6"} Jul 06 19:10:41.504937 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:41.504902 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-8mnph" event={"ID":"6b76700e-1870-40a7-a1f2-ab3c1332058c","Type":"ContainerStarted","Data":"f287f6c2364ac19965bb54dcae0e508625e65bd5773404666ec68d4488733c06"} Jul 06 19:10:41.506385 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:41.506351 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-f9z4j" event={"ID":"2f2852c7-f9ef-445c-aa57-e5c697cb2c4b","Type":"ContainerStarted","Data":"0a17b274ef1bc4bb15e7050f39d4e48398e0cd7ced7e08b12d0e1f81c9c56215"} Jul 06 19:10:41.580492 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:41.580419 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:10:42.390226 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:42.390175 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:10:42.462983 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:42.462891 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:10:42.924493 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:42.924393 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/38cfd0e8-69ad-4b63-bd2d-74e5757ca860-metrics-tls\") pod \"dns-default-jg5kg\" (UID: \"38cfd0e8-69ad-4b63-bd2d-74e5757ca860\") " pod="openshift-dns/dns-default-jg5kg" Jul 06 19:10:42.927511 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:42.927480 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/38cfd0e8-69ad-4b63-bd2d-74e5757ca860-metrics-tls\") pod \"dns-default-jg5kg\" (UID: \"38cfd0e8-69ad-4b63-bd2d-74e5757ca860\") " pod="openshift-dns/dns-default-jg5kg" Jul 06 19:10:43.141677 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:43.141642 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-2k95g\"" Jul 06 19:10:43.149494 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:43.149460 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-jg5kg" Jul 06 19:10:43.518559 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:43.518515 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-8mnph" event={"ID":"6b76700e-1870-40a7-a1f2-ab3c1332058c","Type":"ContainerStarted","Data":"b49c835b22d998bb72ac4f17a37dd3b5f700bb49751acbf03edbf924152ac891"} Jul 06 19:10:43.519028 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:43.518632 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-8mnph" event={"ID":"6b76700e-1870-40a7-a1f2-ab3c1332058c","Type":"ContainerStarted","Data":"dc88b49fda8278a5470207f20c7f3ae12c64110eb95f0c8879497b3c3a404574"} Jul 06 19:10:43.536027 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:43.535880 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-8mnph" podStartSLOduration=67.178763842 podStartE2EDuration="1m8.535857367s" totalImagesPullingTime="1.357093525s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:09:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:10:43.534850019 +0000 UTC m=+68.842208968" watchObservedRunningTime="2026-07-06 19:10:43.535857367 +0000 UTC m=+68.843216318" Jul 06 19:10:45.197388 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:45.197345 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:10:45.296736 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:45.296692 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:10:45.398380 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:45.398339 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:10:45.498824 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:45.498745 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:10:45.636230 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:45.636194 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:10:45.864231 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:45.864188 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:10:46.040772 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:46.040740 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:10:46.095952 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:46.095922 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:10:47.067049 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:47.067009 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:10:47.272179 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:47.272130 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:10:47.373380 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:47.373293 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:10:47.441787 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:47.441743 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:10:47.919397 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:47.919358 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-jg5kg"] Jul 06 19:10:47.924370 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:10:47.924340 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38cfd0e8_69ad_4b63_bd2d_74e5757ca860.slice/crio-dc3d4e23816831340a5fef3da225464a7ea0b2af84573d03918b87f3481dd4d5 WatchSource:0}: Error finding container dc3d4e23816831340a5fef3da225464a7ea0b2af84573d03918b87f3481dd4d5: Status 404 returned error can't find the container with id dc3d4e23816831340a5fef3da225464a7ea0b2af84573d03918b87f3481dd4d5 Jul 06 19:10:48.537928 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:48.537891 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-jg5kg" event={"ID":"38cfd0e8-69ad-4b63-bd2d-74e5757ca860","Type":"ContainerStarted","Data":"dc3d4e23816831340a5fef3da225464a7ea0b2af84573d03918b87f3481dd4d5"} Jul 06 19:10:48.539617 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:48.539583 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-f9z4j" event={"ID":"2f2852c7-f9ef-445c-aa57-e5c697cb2c4b","Type":"ContainerStarted","Data":"53b085b438051b1f049989535fc46a4c00bb30384277e03a9dc3782ddc17e27f"} Jul 06 19:10:48.539761 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:48.539643 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-f9z4j" Jul 06 19:10:48.541436 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:48.541405 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-svgl2" event={"ID":"e8c2f347-1744-49ff-b46e-f036d27a5e46","Type":"ContainerStarted","Data":"3ead636b54bbcb7de76597fe09aa43fe340503a3aabd9f124ae1541cc7e59c6d"} Jul 06 19:10:48.543705 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:48.543681 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hqnrn" event={"ID":"1a6ef785-86f6-412f-8ef3-300db1b24c2c","Type":"ContainerStarted","Data":"8565b62d7920d7ce70192776e2bf6eedbe11c5d54a7e0a668655339f43c90a81"} Jul 06 19:10:48.543810 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:48.543731 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hqnrn" event={"ID":"1a6ef785-86f6-412f-8ef3-300db1b24c2c","Type":"ContainerStarted","Data":"7cb5cd805d9547f5ac192057139cf581288f400c5e9b0a3f1a3a54b930c2e866"} Jul 06 19:10:48.557411 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:48.557351 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-f9z4j" podStartSLOduration=67.234913142 podStartE2EDuration="1m13.55733061s" totalImagesPullingTime="6.322417468s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:09:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:10:48.556256638 +0000 UTC m=+73.863615588" watchObservedRunningTime="2026-07-06 19:10:48.55733061 +0000 UTC m=+73.864689559" Jul 06 19:10:50.185060 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:50.185007 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c4564790-651c-4af2-a036-c285bb57f10a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-z9c9q\" (UID: \"c4564790-651c-4af2-a036-c285bb57f10a\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-z9c9q" Jul 06 19:10:50.185485 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:50.185461 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:10:50.187803 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:50.187777 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/c4564790-651c-4af2-a036-c285bb57f10a-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-64786d9cbb-z9c9q\" (UID: \"c4564790-651c-4af2-a036-c285bb57f10a\") " pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-z9c9q" Jul 06 19:10:50.217322 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:50.217291 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-kwlgh\"" Jul 06 19:10:50.224510 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:50.224473 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-z9c9q" Jul 06 19:10:50.286163 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:50.286122 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:10:50.386661 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:50.386583 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1ddf025a-43f4-4d0a-826e-06e5137b740f-registry-tls\") pod \"image-registry-666cddf4fd-nks7n\" (UID: \"1ddf025a-43f4-4d0a-826e-06e5137b740f\") " pod="openshift-image-registry/image-registry-666cddf4fd-nks7n" Jul 06 19:10:50.389331 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:50.389301 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1ddf025a-43f4-4d0a-826e-06e5137b740f-registry-tls\") pod \"image-registry-666cddf4fd-nks7n\" (UID: \"1ddf025a-43f4-4d0a-826e-06e5137b740f\") " pod="openshift-image-registry/image-registry-666cddf4fd-nks7n" Jul 06 19:10:50.407613 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:50.407575 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-5jctw\"" Jul 06 19:10:50.415457 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:50.415418 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-666cddf4fd-nks7n" Jul 06 19:10:50.488823 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:50.488789 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:10:50.542448 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:50.542414 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:10:50.556829 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:50.556792 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-jg5kg" event={"ID":"38cfd0e8-69ad-4b63-bd2d-74e5757ca860","Type":"ContainerStarted","Data":"8ed398feb670d7310252aa4504d2de9459e1d2a9562ac4c400d65bf233aa05c6"} Jul 06 19:10:50.953180 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:50.953138 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:10:51.004427 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:51.004395 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:10:51.551288 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:51.551229 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-svgl2" podStartSLOduration=70.232182066 podStartE2EDuration="1m16.551207386s" totalImagesPullingTime="6.31902532s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 19:09:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:10:48.571489991 +0000 UTC m=+73.878848940" watchObservedRunningTime="2026-07-06 19:10:51.551207386 +0000 UTC m=+76.858566335" Jul 06 19:10:51.551708 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:51.551684 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-666cddf4fd-nks7n"] Jul 06 19:10:51.554011 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:10:51.553979 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ddf025a_43f4_4d0a_826e_06e5137b740f.slice/crio-66fc40817100aaa67461a6e7578e666d28b18cc2bf087b758d941d268f2f2907 WatchSource:0}: Error finding container 66fc40817100aaa67461a6e7578e666d28b18cc2bf087b758d941d268f2f2907: Status 404 returned error can't find the container with id 66fc40817100aaa67461a6e7578e666d28b18cc2bf087b758d941d268f2f2907 Jul 06 19:10:51.561166 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:51.561139 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-64786d9cbb-z9c9q"] Jul 06 19:10:51.566955 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:51.566919 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-666cddf4fd-nks7n" event={"ID":"1ddf025a-43f4-4d0a-826e-06e5137b740f","Type":"ContainerStarted","Data":"66fc40817100aaa67461a6e7578e666d28b18cc2bf087b758d941d268f2f2907"} Jul 06 19:10:51.569202 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:51.569170 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-jg5kg" event={"ID":"38cfd0e8-69ad-4b63-bd2d-74e5757ca860","Type":"ContainerStarted","Data":"cca53ce2882e159a62b84fdfe3e2ccbf015a0c765b68e94d0bd63b08352e71dc"} Jul 06 19:10:51.569316 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:10:51.569218 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4564790_651c_4af2_a036_c285bb57f10a.slice/crio-abb4d3a4e8105c83f6d18b03c7c171eb2fb8318952652fbf00d12c6534239e82 WatchSource:0}: Error finding container abb4d3a4e8105c83f6d18b03c7c171eb2fb8318952652fbf00d12c6534239e82: Status 404 returned error can't find the container with id abb4d3a4e8105c83f6d18b03c7c171eb2fb8318952652fbf00d12c6534239e82 Jul 06 19:10:51.591765 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:51.591729 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5f6f6dc6cd-4ch72"] Jul 06 19:10:51.608333 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:51.608303 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5f6f6dc6cd-4ch72"] Jul 06 19:10:51.608512 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:51.608432 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5f6f6dc6cd-4ch72" Jul 06 19:10:51.612929 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:51.612899 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 06 19:10:51.613104 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:51.612943 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 06 19:10:51.613201 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:51.613128 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 06 19:10:51.613631 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:51.613617 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-2vghc\"" Jul 06 19:10:51.613698 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:51.613648 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 06 19:10:51.688695 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:51.688607 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-666cddf4fd-nks7n"] Jul 06 19:10:51.697896 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:51.697856 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hvx5n\" (UniqueName: \"kubernetes.io/projected/43d7d3d8-3483-4d0a-856e-1d1aeb5ad84c-kube-api-access-hvx5n\") pod \"managed-serviceaccount-addon-agent-5f6f6dc6cd-4ch72\" (UID: \"43d7d3d8-3483-4d0a-856e-1d1aeb5ad84c\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5f6f6dc6cd-4ch72" Jul 06 19:10:51.697896 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:51.697893 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/43d7d3d8-3483-4d0a-856e-1d1aeb5ad84c-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5f6f6dc6cd-4ch72\" (UID: \"43d7d3d8-3483-4d0a-856e-1d1aeb5ad84c\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5f6f6dc6cd-4ch72" Jul 06 19:10:51.766184 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:51.766148 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:10:51.792914 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:51.792873 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-788dd79d58-8qj85"] Jul 06 19:10:51.798865 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:51.798830 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hvx5n\" (UniqueName: \"kubernetes.io/projected/43d7d3d8-3483-4d0a-856e-1d1aeb5ad84c-kube-api-access-hvx5n\") pod \"managed-serviceaccount-addon-agent-5f6f6dc6cd-4ch72\" (UID: \"43d7d3d8-3483-4d0a-856e-1d1aeb5ad84c\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5f6f6dc6cd-4ch72" Jul 06 19:10:51.799013 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:51.798877 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/43d7d3d8-3483-4d0a-856e-1d1aeb5ad84c-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5f6f6dc6cd-4ch72\" (UID: \"43d7d3d8-3483-4d0a-856e-1d1aeb5ad84c\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5f6f6dc6cd-4ch72" Jul 06 19:10:51.801586 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:51.801560 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/43d7d3d8-3483-4d0a-856e-1d1aeb5ad84c-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5f6f6dc6cd-4ch72\" (UID: \"43d7d3d8-3483-4d0a-856e-1d1aeb5ad84c\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5f6f6dc6cd-4ch72" Jul 06 19:10:51.812411 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:51.812383 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-788dd79d58-8qj85"] Jul 06 19:10:51.812603 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:51.812487 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-788dd79d58-8qj85" Jul 06 19:10:51.815310 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:51.815286 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hvx5n\" (UniqueName: \"kubernetes.io/projected/43d7d3d8-3483-4d0a-856e-1d1aeb5ad84c-kube-api-access-hvx5n\") pod \"managed-serviceaccount-addon-agent-5f6f6dc6cd-4ch72\" (UID: \"43d7d3d8-3483-4d0a-856e-1d1aeb5ad84c\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5f6f6dc6cd-4ch72" Jul 06 19:10:51.842133 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:51.842101 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:10:51.900219 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:51.900184 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d7905780-4864-4b14-8c62-b56030fbbe81-registry-certificates\") pod \"image-registry-788dd79d58-8qj85\" (UID: \"d7905780-4864-4b14-8c62-b56030fbbe81\") " pod="openshift-image-registry/image-registry-788dd79d58-8qj85" Jul 06 19:10:51.900219 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:51.900233 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d7905780-4864-4b14-8c62-b56030fbbe81-trusted-ca\") pod \"image-registry-788dd79d58-8qj85\" (UID: \"d7905780-4864-4b14-8c62-b56030fbbe81\") " pod="openshift-image-registry/image-registry-788dd79d58-8qj85" Jul 06 19:10:51.900430 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:51.900255 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wfgmn\" (UniqueName: \"kubernetes.io/projected/d7905780-4864-4b14-8c62-b56030fbbe81-kube-api-access-wfgmn\") pod \"image-registry-788dd79d58-8qj85\" (UID: \"d7905780-4864-4b14-8c62-b56030fbbe81\") " pod="openshift-image-registry/image-registry-788dd79d58-8qj85" Jul 06 19:10:51.900430 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:51.900298 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d7905780-4864-4b14-8c62-b56030fbbe81-image-registry-private-configuration\") pod \"image-registry-788dd79d58-8qj85\" (UID: \"d7905780-4864-4b14-8c62-b56030fbbe81\") " pod="openshift-image-registry/image-registry-788dd79d58-8qj85" Jul 06 19:10:51.900430 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:51.900349 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d7905780-4864-4b14-8c62-b56030fbbe81-registry-tls\") pod \"image-registry-788dd79d58-8qj85\" (UID: \"d7905780-4864-4b14-8c62-b56030fbbe81\") " pod="openshift-image-registry/image-registry-788dd79d58-8qj85" Jul 06 19:10:51.900430 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:51.900407 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d7905780-4864-4b14-8c62-b56030fbbe81-ca-trust-extracted\") pod \"image-registry-788dd79d58-8qj85\" (UID: \"d7905780-4864-4b14-8c62-b56030fbbe81\") " pod="openshift-image-registry/image-registry-788dd79d58-8qj85" Jul 06 19:10:51.900575 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:51.900481 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d7905780-4864-4b14-8c62-b56030fbbe81-installation-pull-secrets\") pod \"image-registry-788dd79d58-8qj85\" (UID: \"d7905780-4864-4b14-8c62-b56030fbbe81\") " pod="openshift-image-registry/image-registry-788dd79d58-8qj85" Jul 06 19:10:51.900575 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:51.900512 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d7905780-4864-4b14-8c62-b56030fbbe81-bound-sa-token\") pod \"image-registry-788dd79d58-8qj85\" (UID: \"d7905780-4864-4b14-8c62-b56030fbbe81\") " pod="openshift-image-registry/image-registry-788dd79d58-8qj85" Jul 06 19:10:51.929942 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:51.929899 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5f6f6dc6cd-4ch72" Jul 06 19:10:51.941781 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:51.941756 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:10:52.001662 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:52.001630 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d7905780-4864-4b14-8c62-b56030fbbe81-registry-certificates\") pod \"image-registry-788dd79d58-8qj85\" (UID: \"d7905780-4864-4b14-8c62-b56030fbbe81\") " pod="openshift-image-registry/image-registry-788dd79d58-8qj85" Jul 06 19:10:52.001857 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:52.001687 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d7905780-4864-4b14-8c62-b56030fbbe81-trusted-ca\") pod \"image-registry-788dd79d58-8qj85\" (UID: \"d7905780-4864-4b14-8c62-b56030fbbe81\") " pod="openshift-image-registry/image-registry-788dd79d58-8qj85" Jul 06 19:10:52.001857 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:52.001708 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wfgmn\" (UniqueName: \"kubernetes.io/projected/d7905780-4864-4b14-8c62-b56030fbbe81-kube-api-access-wfgmn\") pod \"image-registry-788dd79d58-8qj85\" (UID: \"d7905780-4864-4b14-8c62-b56030fbbe81\") " pod="openshift-image-registry/image-registry-788dd79d58-8qj85" Jul 06 19:10:52.001857 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:52.001733 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d7905780-4864-4b14-8c62-b56030fbbe81-image-registry-private-configuration\") pod \"image-registry-788dd79d58-8qj85\" (UID: \"d7905780-4864-4b14-8c62-b56030fbbe81\") " pod="openshift-image-registry/image-registry-788dd79d58-8qj85" Jul 06 19:10:52.001857 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:52.001759 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d7905780-4864-4b14-8c62-b56030fbbe81-registry-tls\") pod \"image-registry-788dd79d58-8qj85\" (UID: \"d7905780-4864-4b14-8c62-b56030fbbe81\") " pod="openshift-image-registry/image-registry-788dd79d58-8qj85" Jul 06 19:10:52.001857 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:52.001792 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d7905780-4864-4b14-8c62-b56030fbbe81-ca-trust-extracted\") pod \"image-registry-788dd79d58-8qj85\" (UID: \"d7905780-4864-4b14-8c62-b56030fbbe81\") " pod="openshift-image-registry/image-registry-788dd79d58-8qj85" Jul 06 19:10:52.001857 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:52.001831 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d7905780-4864-4b14-8c62-b56030fbbe81-installation-pull-secrets\") pod \"image-registry-788dd79d58-8qj85\" (UID: \"d7905780-4864-4b14-8c62-b56030fbbe81\") " pod="openshift-image-registry/image-registry-788dd79d58-8qj85" Jul 06 19:10:52.001857 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:52.001855 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d7905780-4864-4b14-8c62-b56030fbbe81-bound-sa-token\") pod \"image-registry-788dd79d58-8qj85\" (UID: \"d7905780-4864-4b14-8c62-b56030fbbe81\") " pod="openshift-image-registry/image-registry-788dd79d58-8qj85" Jul 06 19:10:52.002652 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:52.002619 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d7905780-4864-4b14-8c62-b56030fbbe81-ca-trust-extracted\") pod \"image-registry-788dd79d58-8qj85\" (UID: \"d7905780-4864-4b14-8c62-b56030fbbe81\") " pod="openshift-image-registry/image-registry-788dd79d58-8qj85" Jul 06 19:10:52.002949 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:52.002690 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d7905780-4864-4b14-8c62-b56030fbbe81-trusted-ca\") pod \"image-registry-788dd79d58-8qj85\" (UID: \"d7905780-4864-4b14-8c62-b56030fbbe81\") " pod="openshift-image-registry/image-registry-788dd79d58-8qj85" Jul 06 19:10:52.002949 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:52.002706 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d7905780-4864-4b14-8c62-b56030fbbe81-registry-certificates\") pod \"image-registry-788dd79d58-8qj85\" (UID: \"d7905780-4864-4b14-8c62-b56030fbbe81\") " pod="openshift-image-registry/image-registry-788dd79d58-8qj85" Jul 06 19:10:52.004700 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:52.004677 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d7905780-4864-4b14-8c62-b56030fbbe81-image-registry-private-configuration\") pod \"image-registry-788dd79d58-8qj85\" (UID: \"d7905780-4864-4b14-8c62-b56030fbbe81\") " pod="openshift-image-registry/image-registry-788dd79d58-8qj85" Jul 06 19:10:52.005322 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:52.005302 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d7905780-4864-4b14-8c62-b56030fbbe81-registry-tls\") pod \"image-registry-788dd79d58-8qj85\" (UID: \"d7905780-4864-4b14-8c62-b56030fbbe81\") " pod="openshift-image-registry/image-registry-788dd79d58-8qj85" Jul 06 19:10:52.005441 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:52.005416 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d7905780-4864-4b14-8c62-b56030fbbe81-installation-pull-secrets\") pod \"image-registry-788dd79d58-8qj85\" (UID: \"d7905780-4864-4b14-8c62-b56030fbbe81\") " pod="openshift-image-registry/image-registry-788dd79d58-8qj85" Jul 06 19:10:52.010203 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:52.010179 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d7905780-4864-4b14-8c62-b56030fbbe81-bound-sa-token\") pod \"image-registry-788dd79d58-8qj85\" (UID: \"d7905780-4864-4b14-8c62-b56030fbbe81\") " pod="openshift-image-registry/image-registry-788dd79d58-8qj85" Jul 06 19:10:52.010648 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:52.010626 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wfgmn\" (UniqueName: \"kubernetes.io/projected/d7905780-4864-4b14-8c62-b56030fbbe81-kube-api-access-wfgmn\") pod \"image-registry-788dd79d58-8qj85\" (UID: \"d7905780-4864-4b14-8c62-b56030fbbe81\") " pod="openshift-image-registry/image-registry-788dd79d58-8qj85" Jul 06 19:10:52.029326 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:52.029290 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:10:52.077448 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:52.077414 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5f6f6dc6cd-4ch72"] Jul 06 19:10:52.080339 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:10:52.080312 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod43d7d3d8_3483_4d0a_856e_1d1aeb5ad84c.slice/crio-1f3f6acb3ac0a0944fcc9843b6aa3574cc8c93031af013be2e5bfe367defcf40 WatchSource:0}: Error finding container 1f3f6acb3ac0a0944fcc9843b6aa3574cc8c93031af013be2e5bfe367defcf40: Status 404 returned error can't find the container with id 1f3f6acb3ac0a0944fcc9843b6aa3574cc8c93031af013be2e5bfe367defcf40 Jul 06 19:10:52.112489 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:52.112447 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:10:52.141790 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:52.141757 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-788dd79d58-8qj85" Jul 06 19:10:52.170966 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:52.170935 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:10:52.290546 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:52.290514 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-788dd79d58-8qj85"] Jul 06 19:10:52.293510 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:10:52.293484 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7905780_4864_4b14_8c62_b56030fbbe81.slice/crio-e71a86d212c1016981086b509f2a726e788adfb60468b63beb12e7c444f33aa5 WatchSource:0}: Error finding container e71a86d212c1016981086b509f2a726e788adfb60468b63beb12e7c444f33aa5: Status 404 returned error can't find the container with id e71a86d212c1016981086b509f2a726e788adfb60468b63beb12e7c444f33aa5 Jul 06 19:10:52.299305 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:52.299282 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:10:52.369124 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:52.369094 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:10:52.576115 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:52.576053 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5f6f6dc6cd-4ch72" event={"ID":"43d7d3d8-3483-4d0a-856e-1d1aeb5ad84c","Type":"ContainerStarted","Data":"1f3f6acb3ac0a0944fcc9843b6aa3574cc8c93031af013be2e5bfe367defcf40"} Jul 06 19:10:52.579843 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:52.579308 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hqnrn" event={"ID":"1a6ef785-86f6-412f-8ef3-300db1b24c2c","Type":"ContainerStarted","Data":"591039c0b1c29998d9dc94687487c5147c701d64d6c2c74c5731ccc7c81f9bc0"} Jul 06 19:10:52.584559 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:52.584056 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-666cddf4fd-nks7n" event={"ID":"1ddf025a-43f4-4d0a-826e-06e5137b740f","Type":"ContainerStarted","Data":"0ffd5be061bc9cc2a75c881b3988d243a2e5ee4944b58b7f5212ebd467d15b3d"} Jul 06 19:10:52.584559 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:52.584310 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-666cddf4fd-nks7n" Jul 06 19:10:52.587457 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:52.587425 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-788dd79d58-8qj85" event={"ID":"d7905780-4864-4b14-8c62-b56030fbbe81","Type":"ContainerStarted","Data":"6f27e0745ddd479b4f060b5e2155ea8a031f73625715e8f13c93840eaba65004"} Jul 06 19:10:52.587618 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:52.587481 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-788dd79d58-8qj85" event={"ID":"d7905780-4864-4b14-8c62-b56030fbbe81","Type":"ContainerStarted","Data":"e71a86d212c1016981086b509f2a726e788adfb60468b63beb12e7c444f33aa5"} Jul 06 19:10:52.587618 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:52.587602 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-788dd79d58-8qj85" Jul 06 19:10:52.589424 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:52.589317 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-z9c9q" event={"ID":"c4564790-651c-4af2-a036-c285bb57f10a","Type":"ContainerStarted","Data":"abb4d3a4e8105c83f6d18b03c7c171eb2fb8318952652fbf00d12c6534239e82"} Jul 06 19:10:52.589424 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:52.589368 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-jg5kg" Jul 06 19:10:52.597238 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:52.597169 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-hqnrn" podStartSLOduration=17.478057044 podStartE2EDuration="28.597149135s" totalImagesPullingTime="11.119092091s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 19:10:24 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:10:52.596319421 +0000 UTC m=+77.903678369" watchObservedRunningTime="2026-07-06 19:10:52.597149135 +0000 UTC m=+77.904508079" Jul 06 19:10:52.614242 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:52.614177 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-666cddf4fd-nks7n" podStartSLOduration=34.614154189 podStartE2EDuration="34.614154189s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:10:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:10:52.613028538 +0000 UTC m=+77.920387489" watchObservedRunningTime="2026-07-06 19:10:52.614154189 +0000 UTC m=+77.921513138" Jul 06 19:10:52.631752 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:52.631526 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-788dd79d58-8qj85" podStartSLOduration=1.6315047219999999 podStartE2EDuration="1.631504722s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:10:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:10:52.630877534 +0000 UTC m=+77.938236482" watchObservedRunningTime="2026-07-06 19:10:52.631504722 +0000 UTC m=+77.938863671" Jul 06 19:10:52.648282 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:52.648193 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-jg5kg" podStartSLOduration=40.503394313 podStartE2EDuration="42.648171172s" totalImagesPullingTime="2.144776859s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:10:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:10:52.646089936 +0000 UTC m=+77.953448883" watchObservedRunningTime="2026-07-06 19:10:52.648171172 +0000 UTC m=+77.955530122" Jul 06 19:10:52.728301 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:52.728263 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:10:53.504185 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:53.502447 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:10:53.894493 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:53.894449 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:10:53.994740 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:53.994714 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:10:54.503627 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:54.503560 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:10:54.570029 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:54.569997 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:10:55.141324 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:55.141230 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:10:55.284206 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:55.284015 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:10:55.469717 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:55.469574 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:10:55.602044 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:55.602004 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-z9c9q" event={"ID":"c4564790-651c-4af2-a036-c285bb57f10a","Type":"ContainerStarted","Data":"178cdee5de344aaa6e0335a7ef205f78c073d389daa7e940745c1a627311ff46"} Jul 06 19:10:55.603470 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:55.603447 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5f6f6dc6cd-4ch72" event={"ID":"43d7d3d8-3483-4d0a-856e-1d1aeb5ad84c","Type":"ContainerStarted","Data":"e8d39b6927cc90aac10abafa54999fd2d5b35fb5c8d43ddc104dcdb1e6626484"} Jul 06 19:10:55.617756 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:55.617697 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-64786d9cbb-z9c9q" podStartSLOduration=34.576499153 podStartE2EDuration="37.617683s" totalImagesPullingTime="3.041183847s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:10:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:10:55.616531584 +0000 UTC m=+80.923890533" watchObservedRunningTime="2026-07-06 19:10:55.617683 +0000 UTC m=+80.925041947" Jul 06 19:10:55.631739 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:10:55.631680 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5f6f6dc6cd-4ch72" podStartSLOduration=1.909839227 podStartE2EDuration="4.631663533s" totalImagesPullingTime="2.721824306s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:10:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:10:55.630621794 +0000 UTC m=+80.937980744" watchObservedRunningTime="2026-07-06 19:10:55.631663533 +0000 UTC m=+80.939022481" Jul 06 19:11:00.599121 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:00.599089 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-jg5kg" Jul 06 19:11:00.967283 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:00.967187 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:11:01.068437 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:01.068405 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:11:01.168823 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:01.168788 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:11:01.293111 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:01.293052 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:11:01.362302 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:01.362262 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-bkxlw"] Jul 06 19:11:01.365656 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:01.365632 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-bkxlw" Jul 06 19:11:01.366587 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:01.366567 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:11:01.368170 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:01.368152 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 06 19:11:01.368889 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:01.368872 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 06 19:11:01.368967 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:01.368952 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 06 19:11:01.369061 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:01.369038 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-c5rzc\"" Jul 06 19:11:01.375296 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:01.375269 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-bkxlw"] Jul 06 19:11:01.470771 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:01.470733 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/414f6ad7-4bba-4bff-99ec-9c29b9cb2c85-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-bkxlw\" (UID: \"414f6ad7-4bba-4bff-99ec-9c29b9cb2c85\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-bkxlw" Jul 06 19:11:01.470771 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:01.470778 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/414f6ad7-4bba-4bff-99ec-9c29b9cb2c85-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-bkxlw\" (UID: \"414f6ad7-4bba-4bff-99ec-9c29b9cb2c85\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-bkxlw" Jul 06 19:11:01.471021 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:01.470809 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/414f6ad7-4bba-4bff-99ec-9c29b9cb2c85-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-bkxlw\" (UID: \"414f6ad7-4bba-4bff-99ec-9c29b9cb2c85\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-bkxlw" Jul 06 19:11:01.471021 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:01.470872 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tz2xr\" (UniqueName: \"kubernetes.io/projected/414f6ad7-4bba-4bff-99ec-9c29b9cb2c85-kube-api-access-tz2xr\") pod \"prometheus-operator-567f7b8b7c-bkxlw\" (UID: \"414f6ad7-4bba-4bff-99ec-9c29b9cb2c85\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-bkxlw" Jul 06 19:11:01.571940 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:01.571855 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/414f6ad7-4bba-4bff-99ec-9c29b9cb2c85-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-bkxlw\" (UID: \"414f6ad7-4bba-4bff-99ec-9c29b9cb2c85\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-bkxlw" Jul 06 19:11:01.571940 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:01.571903 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/414f6ad7-4bba-4bff-99ec-9c29b9cb2c85-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-bkxlw\" (UID: \"414f6ad7-4bba-4bff-99ec-9c29b9cb2c85\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-bkxlw" Jul 06 19:11:01.571940 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:01.571928 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tz2xr\" (UniqueName: \"kubernetes.io/projected/414f6ad7-4bba-4bff-99ec-9c29b9cb2c85-kube-api-access-tz2xr\") pod \"prometheus-operator-567f7b8b7c-bkxlw\" (UID: \"414f6ad7-4bba-4bff-99ec-9c29b9cb2c85\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-bkxlw" Jul 06 19:11:01.572226 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:01.571966 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/414f6ad7-4bba-4bff-99ec-9c29b9cb2c85-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-bkxlw\" (UID: \"414f6ad7-4bba-4bff-99ec-9c29b9cb2c85\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-bkxlw" Jul 06 19:11:01.572226 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:11:01.572050 2566 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found Jul 06 19:11:01.572226 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:11:01.572165 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/414f6ad7-4bba-4bff-99ec-9c29b9cb2c85-prometheus-operator-tls podName:414f6ad7-4bba-4bff-99ec-9c29b9cb2c85 nodeName:}" failed. No retries permitted until 2026-07-06 19:11:02.072143321 +0000 UTC m=+87.379502262 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/414f6ad7-4bba-4bff-99ec-9c29b9cb2c85-prometheus-operator-tls") pod "prometheus-operator-567f7b8b7c-bkxlw" (UID: "414f6ad7-4bba-4bff-99ec-9c29b9cb2c85") : secret "prometheus-operator-tls" not found Jul 06 19:11:01.572680 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:01.572658 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/414f6ad7-4bba-4bff-99ec-9c29b9cb2c85-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-bkxlw\" (UID: \"414f6ad7-4bba-4bff-99ec-9c29b9cb2c85\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-bkxlw" Jul 06 19:11:01.574489 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:01.574467 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/414f6ad7-4bba-4bff-99ec-9c29b9cb2c85-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-bkxlw\" (UID: \"414f6ad7-4bba-4bff-99ec-9c29b9cb2c85\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-bkxlw" Jul 06 19:11:01.581168 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:01.581129 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tz2xr\" (UniqueName: \"kubernetes.io/projected/414f6ad7-4bba-4bff-99ec-9c29b9cb2c85-kube-api-access-tz2xr\") pod \"prometheus-operator-567f7b8b7c-bkxlw\" (UID: \"414f6ad7-4bba-4bff-99ec-9c29b9cb2c85\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-bkxlw" Jul 06 19:11:01.890240 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:01.890142 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:11:01.991039 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:01.991002 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:11:02.075186 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:02.075142 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/414f6ad7-4bba-4bff-99ec-9c29b9cb2c85-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-bkxlw\" (UID: \"414f6ad7-4bba-4bff-99ec-9c29b9cb2c85\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-bkxlw" Jul 06 19:11:02.077721 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:02.077677 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/414f6ad7-4bba-4bff-99ec-9c29b9cb2c85-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-bkxlw\" (UID: \"414f6ad7-4bba-4bff-99ec-9c29b9cb2c85\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-bkxlw" Jul 06 19:11:02.091737 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:02.091710 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:11:02.275533 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:02.275448 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-bkxlw" Jul 06 19:11:02.412783 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:02.412727 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-bkxlw"] Jul 06 19:11:02.416795 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:11:02.416759 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod414f6ad7_4bba_4bff_99ec_9c29b9cb2c85.slice/crio-b94dc68c491a468ae04d7d2dd6ac22a09a927d8ba579fbddad7c47ba27c2b6f6 WatchSource:0}: Error finding container b94dc68c491a468ae04d7d2dd6ac22a09a927d8ba579fbddad7c47ba27c2b6f6: Status 404 returned error can't find the container with id b94dc68c491a468ae04d7d2dd6ac22a09a927d8ba579fbddad7c47ba27c2b6f6 Jul 06 19:11:02.626832 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:02.626791 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-bkxlw" event={"ID":"414f6ad7-4bba-4bff-99ec-9c29b9cb2c85","Type":"ContainerStarted","Data":"b94dc68c491a468ae04d7d2dd6ac22a09a927d8ba579fbddad7c47ba27c2b6f6"} Jul 06 19:11:03.957427 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:03.957339 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:11:04.636027 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:04.635987 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-bkxlw" event={"ID":"414f6ad7-4bba-4bff-99ec-9c29b9cb2c85","Type":"ContainerStarted","Data":"42262126b80ee589c7d9c623a9550f2bb298a63cbc9a72310fec690caa3d17d7"} Jul 06 19:11:04.636220 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:04.636042 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-bkxlw" event={"ID":"414f6ad7-4bba-4bff-99ec-9c29b9cb2c85","Type":"ContainerStarted","Data":"6ec4159e45e1c27c07e5679c766d231a6eb2e45fb6b7cce7087161aa2492120a"} Jul 06 19:11:04.652387 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:04.652325 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-bkxlw" podStartSLOduration=2.387299531 podStartE2EDuration="3.652307407s" totalImagesPullingTime="1.265007876s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:11:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:11:04.651316238 +0000 UTC m=+89.958675188" watchObservedRunningTime="2026-07-06 19:11:04.652307407 +0000 UTC m=+89.959666355" Jul 06 19:11:05.494543 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:05.494507 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:11:05.594295 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:05.594265 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:11:06.453181 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:06.453149 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:11:06.671295 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:06.671262 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:11:06.779669 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:06.779584 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-qbxt4"] Jul 06 19:11:06.782058 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:06.782041 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-qbxt4" Jul 06 19:11:06.785025 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:06.785000 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-rphr7\"" Jul 06 19:11:06.785263 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:06.785150 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 06 19:11:06.785453 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:06.785244 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 06 19:11:06.785676 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:06.785646 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 06 19:11:06.786506 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:06.786486 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:11:06.882378 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:06.882348 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:11:06.913453 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:06.913422 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cc77fe73-70aa-4c81-8d65-fcf55a4b5f9c-sys\") pod \"node-exporter-qbxt4\" (UID: \"cc77fe73-70aa-4c81-8d65-fcf55a4b5f9c\") " pod="openshift-monitoring/node-exporter-qbxt4" Jul 06 19:11:06.913583 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:06.913458 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/cc77fe73-70aa-4c81-8d65-fcf55a4b5f9c-node-exporter-wtmp\") pod \"node-exporter-qbxt4\" (UID: \"cc77fe73-70aa-4c81-8d65-fcf55a4b5f9c\") " pod="openshift-monitoring/node-exporter-qbxt4" Jul 06 19:11:06.913583 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:06.913479 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/cc77fe73-70aa-4c81-8d65-fcf55a4b5f9c-node-exporter-tls\") pod \"node-exporter-qbxt4\" (UID: \"cc77fe73-70aa-4c81-8d65-fcf55a4b5f9c\") " pod="openshift-monitoring/node-exporter-qbxt4" Jul 06 19:11:06.913653 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:06.913607 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/cc77fe73-70aa-4c81-8d65-fcf55a4b5f9c-root\") pod \"node-exporter-qbxt4\" (UID: \"cc77fe73-70aa-4c81-8d65-fcf55a4b5f9c\") " pod="openshift-monitoring/node-exporter-qbxt4" Jul 06 19:11:06.913686 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:06.913660 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5wlcv\" (UniqueName: \"kubernetes.io/projected/cc77fe73-70aa-4c81-8d65-fcf55a4b5f9c-kube-api-access-5wlcv\") pod \"node-exporter-qbxt4\" (UID: \"cc77fe73-70aa-4c81-8d65-fcf55a4b5f9c\") " pod="openshift-monitoring/node-exporter-qbxt4" Jul 06 19:11:06.913734 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:06.913688 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/cc77fe73-70aa-4c81-8d65-fcf55a4b5f9c-node-exporter-textfile\") pod \"node-exporter-qbxt4\" (UID: \"cc77fe73-70aa-4c81-8d65-fcf55a4b5f9c\") " pod="openshift-monitoring/node-exporter-qbxt4" Jul 06 19:11:06.913766 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:06.913718 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/cc77fe73-70aa-4c81-8d65-fcf55a4b5f9c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-qbxt4\" (UID: \"cc77fe73-70aa-4c81-8d65-fcf55a4b5f9c\") " pod="openshift-monitoring/node-exporter-qbxt4" Jul 06 19:11:06.913800 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:06.913769 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cc77fe73-70aa-4c81-8d65-fcf55a4b5f9c-metrics-client-ca\") pod \"node-exporter-qbxt4\" (UID: \"cc77fe73-70aa-4c81-8d65-fcf55a4b5f9c\") " pod="openshift-monitoring/node-exporter-qbxt4" Jul 06 19:11:06.913832 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:06.913820 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/cc77fe73-70aa-4c81-8d65-fcf55a4b5f9c-node-exporter-accelerators-collector-config\") pod \"node-exporter-qbxt4\" (UID: \"cc77fe73-70aa-4c81-8d65-fcf55a4b5f9c\") " pod="openshift-monitoring/node-exporter-qbxt4" Jul 06 19:11:07.014547 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:07.014512 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/cc77fe73-70aa-4c81-8d65-fcf55a4b5f9c-node-exporter-accelerators-collector-config\") pod \"node-exporter-qbxt4\" (UID: \"cc77fe73-70aa-4c81-8d65-fcf55a4b5f9c\") " pod="openshift-monitoring/node-exporter-qbxt4" Jul 06 19:11:07.014718 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:07.014564 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cc77fe73-70aa-4c81-8d65-fcf55a4b5f9c-sys\") pod \"node-exporter-qbxt4\" (UID: \"cc77fe73-70aa-4c81-8d65-fcf55a4b5f9c\") " pod="openshift-monitoring/node-exporter-qbxt4" Jul 06 19:11:07.014718 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:07.014595 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/cc77fe73-70aa-4c81-8d65-fcf55a4b5f9c-node-exporter-wtmp\") pod \"node-exporter-qbxt4\" (UID: \"cc77fe73-70aa-4c81-8d65-fcf55a4b5f9c\") " pod="openshift-monitoring/node-exporter-qbxt4" Jul 06 19:11:07.014718 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:07.014668 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/cc77fe73-70aa-4c81-8d65-fcf55a4b5f9c-node-exporter-tls\") pod \"node-exporter-qbxt4\" (UID: \"cc77fe73-70aa-4c81-8d65-fcf55a4b5f9c\") " pod="openshift-monitoring/node-exporter-qbxt4" Jul 06 19:11:07.014718 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:07.014678 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cc77fe73-70aa-4c81-8d65-fcf55a4b5f9c-sys\") pod \"node-exporter-qbxt4\" (UID: \"cc77fe73-70aa-4c81-8d65-fcf55a4b5f9c\") " pod="openshift-monitoring/node-exporter-qbxt4" Jul 06 19:11:07.014917 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:07.014733 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/cc77fe73-70aa-4c81-8d65-fcf55a4b5f9c-root\") pod \"node-exporter-qbxt4\" (UID: \"cc77fe73-70aa-4c81-8d65-fcf55a4b5f9c\") " pod="openshift-monitoring/node-exporter-qbxt4" Jul 06 19:11:07.014917 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:07.014743 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/cc77fe73-70aa-4c81-8d65-fcf55a4b5f9c-node-exporter-wtmp\") pod \"node-exporter-qbxt4\" (UID: \"cc77fe73-70aa-4c81-8d65-fcf55a4b5f9c\") " pod="openshift-monitoring/node-exporter-qbxt4" Jul 06 19:11:07.014917 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:07.014772 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/cc77fe73-70aa-4c81-8d65-fcf55a4b5f9c-root\") pod \"node-exporter-qbxt4\" (UID: \"cc77fe73-70aa-4c81-8d65-fcf55a4b5f9c\") " pod="openshift-monitoring/node-exporter-qbxt4" Jul 06 19:11:07.014917 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:11:07.014772 2566 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 06 19:11:07.014917 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:11:07.014824 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cc77fe73-70aa-4c81-8d65-fcf55a4b5f9c-node-exporter-tls podName:cc77fe73-70aa-4c81-8d65-fcf55a4b5f9c nodeName:}" failed. No retries permitted until 2026-07-06 19:11:07.514808407 +0000 UTC m=+92.822167334 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/cc77fe73-70aa-4c81-8d65-fcf55a4b5f9c-node-exporter-tls") pod "node-exporter-qbxt4" (UID: "cc77fe73-70aa-4c81-8d65-fcf55a4b5f9c") : secret "node-exporter-tls" not found Jul 06 19:11:07.014917 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:07.014780 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5wlcv\" (UniqueName: \"kubernetes.io/projected/cc77fe73-70aa-4c81-8d65-fcf55a4b5f9c-kube-api-access-5wlcv\") pod \"node-exporter-qbxt4\" (UID: \"cc77fe73-70aa-4c81-8d65-fcf55a4b5f9c\") " pod="openshift-monitoring/node-exporter-qbxt4" Jul 06 19:11:07.014917 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:07.014861 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/cc77fe73-70aa-4c81-8d65-fcf55a4b5f9c-node-exporter-textfile\") pod \"node-exporter-qbxt4\" (UID: \"cc77fe73-70aa-4c81-8d65-fcf55a4b5f9c\") " pod="openshift-monitoring/node-exporter-qbxt4" Jul 06 19:11:07.014917 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:07.014886 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/cc77fe73-70aa-4c81-8d65-fcf55a4b5f9c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-qbxt4\" (UID: \"cc77fe73-70aa-4c81-8d65-fcf55a4b5f9c\") " pod="openshift-monitoring/node-exporter-qbxt4" Jul 06 19:11:07.014917 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:07.014915 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cc77fe73-70aa-4c81-8d65-fcf55a4b5f9c-metrics-client-ca\") pod \"node-exporter-qbxt4\" (UID: \"cc77fe73-70aa-4c81-8d65-fcf55a4b5f9c\") " pod="openshift-monitoring/node-exporter-qbxt4" Jul 06 19:11:07.015327 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:07.015194 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/cc77fe73-70aa-4c81-8d65-fcf55a4b5f9c-node-exporter-accelerators-collector-config\") pod \"node-exporter-qbxt4\" (UID: \"cc77fe73-70aa-4c81-8d65-fcf55a4b5f9c\") " pod="openshift-monitoring/node-exporter-qbxt4" Jul 06 19:11:07.015327 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:07.015196 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/cc77fe73-70aa-4c81-8d65-fcf55a4b5f9c-node-exporter-textfile\") pod \"node-exporter-qbxt4\" (UID: \"cc77fe73-70aa-4c81-8d65-fcf55a4b5f9c\") " pod="openshift-monitoring/node-exporter-qbxt4" Jul 06 19:11:07.015390 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:07.015360 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cc77fe73-70aa-4c81-8d65-fcf55a4b5f9c-metrics-client-ca\") pod \"node-exporter-qbxt4\" (UID: \"cc77fe73-70aa-4c81-8d65-fcf55a4b5f9c\") " pod="openshift-monitoring/node-exporter-qbxt4" Jul 06 19:11:07.017491 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:07.017463 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/cc77fe73-70aa-4c81-8d65-fcf55a4b5f9c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-qbxt4\" (UID: \"cc77fe73-70aa-4c81-8d65-fcf55a4b5f9c\") " pod="openshift-monitoring/node-exporter-qbxt4" Jul 06 19:11:07.028402 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:07.028379 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5wlcv\" (UniqueName: \"kubernetes.io/projected/cc77fe73-70aa-4c81-8d65-fcf55a4b5f9c-kube-api-access-5wlcv\") pod \"node-exporter-qbxt4\" (UID: \"cc77fe73-70aa-4c81-8d65-fcf55a4b5f9c\") " pod="openshift-monitoring/node-exporter-qbxt4" Jul 06 19:11:07.070849 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:07.070800 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:11:07.169409 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:07.169365 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:11:07.495803 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:07.495705 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:11:07.520054 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:07.520016 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/cc77fe73-70aa-4c81-8d65-fcf55a4b5f9c-node-exporter-tls\") pod \"node-exporter-qbxt4\" (UID: \"cc77fe73-70aa-4c81-8d65-fcf55a4b5f9c\") " pod="openshift-monitoring/node-exporter-qbxt4" Jul 06 19:11:07.522503 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:07.522467 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/cc77fe73-70aa-4c81-8d65-fcf55a4b5f9c-node-exporter-tls\") pod \"node-exporter-qbxt4\" (UID: \"cc77fe73-70aa-4c81-8d65-fcf55a4b5f9c\") " pod="openshift-monitoring/node-exporter-qbxt4" Jul 06 19:11:07.594717 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:07.594685 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:11:07.692677 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:07.692642 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-qbxt4" Jul 06 19:11:07.696039 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:07.695757 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:11:07.703832 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:11:07.703798 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc77fe73_70aa_4c81_8d65_fcf55a4b5f9c.slice/crio-c1707c1b05fa13337ec21157dd397b7f85b98c94badc80696db9e372c44edd5d WatchSource:0}: Error finding container c1707c1b05fa13337ec21157dd397b7f85b98c94badc80696db9e372c44edd5d: Status 404 returned error can't find the container with id c1707c1b05fa13337ec21157dd397b7f85b98c94badc80696db9e372c44edd5d Jul 06 19:11:07.779408 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:07.779322 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:11:07.910203 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:07.910167 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:11:08.207347 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:08.207312 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:11:08.262609 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:08.262577 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:11:08.654935 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:08.654899 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qbxt4" event={"ID":"cc77fe73-70aa-4c81-8d65-fcf55a4b5f9c","Type":"ContainerStarted","Data":"c1707c1b05fa13337ec21157dd397b7f85b98c94badc80696db9e372c44edd5d"} Jul 06 19:11:08.698290 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:08.698254 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-65c786fdd5-6rpm8"] Jul 06 19:11:08.702060 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:08.702034 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-65c786fdd5-6rpm8" Jul 06 19:11:08.704628 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:08.704343 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 06 19:11:08.704628 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:08.704343 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-fk0a3p3r5aone\"" Jul 06 19:11:08.704628 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:08.704400 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 06 19:11:08.705444 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:08.705424 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 06 19:11:08.705972 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:08.705951 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 06 19:11:08.706104 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:08.705993 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 06 19:11:08.706171 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:08.705780 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-8hdqp\"" Jul 06 19:11:08.714025 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:08.713998 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-65c786fdd5-6rpm8"] Jul 06 19:11:08.830080 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:08.830021 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/61470d37-f33e-46ee-9282-76d61d3a2b0e-secret-thanos-querier-tls\") pod \"thanos-querier-65c786fdd5-6rpm8\" (UID: \"61470d37-f33e-46ee-9282-76d61d3a2b0e\") " pod="openshift-monitoring/thanos-querier-65c786fdd5-6rpm8" Jul 06 19:11:08.830080 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:08.830058 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/61470d37-f33e-46ee-9282-76d61d3a2b0e-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-65c786fdd5-6rpm8\" (UID: \"61470d37-f33e-46ee-9282-76d61d3a2b0e\") " pod="openshift-monitoring/thanos-querier-65c786fdd5-6rpm8" Jul 06 19:11:08.830281 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:08.830104 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/61470d37-f33e-46ee-9282-76d61d3a2b0e-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-65c786fdd5-6rpm8\" (UID: \"61470d37-f33e-46ee-9282-76d61d3a2b0e\") " pod="openshift-monitoring/thanos-querier-65c786fdd5-6rpm8" Jul 06 19:11:08.830281 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:08.830129 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/61470d37-f33e-46ee-9282-76d61d3a2b0e-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-65c786fdd5-6rpm8\" (UID: \"61470d37-f33e-46ee-9282-76d61d3a2b0e\") " pod="openshift-monitoring/thanos-querier-65c786fdd5-6rpm8" Jul 06 19:11:08.830281 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:08.830158 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/61470d37-f33e-46ee-9282-76d61d3a2b0e-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-65c786fdd5-6rpm8\" (UID: \"61470d37-f33e-46ee-9282-76d61d3a2b0e\") " pod="openshift-monitoring/thanos-querier-65c786fdd5-6rpm8" Jul 06 19:11:08.830281 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:08.830179 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/61470d37-f33e-46ee-9282-76d61d3a2b0e-metrics-client-ca\") pod \"thanos-querier-65c786fdd5-6rpm8\" (UID: \"61470d37-f33e-46ee-9282-76d61d3a2b0e\") " pod="openshift-monitoring/thanos-querier-65c786fdd5-6rpm8" Jul 06 19:11:08.830281 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:08.830201 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sjctn\" (UniqueName: \"kubernetes.io/projected/61470d37-f33e-46ee-9282-76d61d3a2b0e-kube-api-access-sjctn\") pod \"thanos-querier-65c786fdd5-6rpm8\" (UID: \"61470d37-f33e-46ee-9282-76d61d3a2b0e\") " pod="openshift-monitoring/thanos-querier-65c786fdd5-6rpm8" Jul 06 19:11:08.830281 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:08.830266 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/61470d37-f33e-46ee-9282-76d61d3a2b0e-secret-grpc-tls\") pod \"thanos-querier-65c786fdd5-6rpm8\" (UID: \"61470d37-f33e-46ee-9282-76d61d3a2b0e\") " pod="openshift-monitoring/thanos-querier-65c786fdd5-6rpm8" Jul 06 19:11:08.931244 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:08.931215 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/61470d37-f33e-46ee-9282-76d61d3a2b0e-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-65c786fdd5-6rpm8\" (UID: \"61470d37-f33e-46ee-9282-76d61d3a2b0e\") " pod="openshift-monitoring/thanos-querier-65c786fdd5-6rpm8" Jul 06 19:11:08.931351 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:08.931256 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/61470d37-f33e-46ee-9282-76d61d3a2b0e-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-65c786fdd5-6rpm8\" (UID: \"61470d37-f33e-46ee-9282-76d61d3a2b0e\") " pod="openshift-monitoring/thanos-querier-65c786fdd5-6rpm8" Jul 06 19:11:08.931461 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:08.931438 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/61470d37-f33e-46ee-9282-76d61d3a2b0e-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-65c786fdd5-6rpm8\" (UID: \"61470d37-f33e-46ee-9282-76d61d3a2b0e\") " pod="openshift-monitoring/thanos-querier-65c786fdd5-6rpm8" Jul 06 19:11:08.931501 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:08.931486 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/61470d37-f33e-46ee-9282-76d61d3a2b0e-metrics-client-ca\") pod \"thanos-querier-65c786fdd5-6rpm8\" (UID: \"61470d37-f33e-46ee-9282-76d61d3a2b0e\") " pod="openshift-monitoring/thanos-querier-65c786fdd5-6rpm8" Jul 06 19:11:08.931539 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:08.931513 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sjctn\" (UniqueName: \"kubernetes.io/projected/61470d37-f33e-46ee-9282-76d61d3a2b0e-kube-api-access-sjctn\") pod \"thanos-querier-65c786fdd5-6rpm8\" (UID: \"61470d37-f33e-46ee-9282-76d61d3a2b0e\") " pod="openshift-monitoring/thanos-querier-65c786fdd5-6rpm8" Jul 06 19:11:08.931583 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:08.931542 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/61470d37-f33e-46ee-9282-76d61d3a2b0e-secret-grpc-tls\") pod \"thanos-querier-65c786fdd5-6rpm8\" (UID: \"61470d37-f33e-46ee-9282-76d61d3a2b0e\") " pod="openshift-monitoring/thanos-querier-65c786fdd5-6rpm8" Jul 06 19:11:08.931630 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:08.931581 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/61470d37-f33e-46ee-9282-76d61d3a2b0e-secret-thanos-querier-tls\") pod \"thanos-querier-65c786fdd5-6rpm8\" (UID: \"61470d37-f33e-46ee-9282-76d61d3a2b0e\") " pod="openshift-monitoring/thanos-querier-65c786fdd5-6rpm8" Jul 06 19:11:08.931676 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:08.931643 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/61470d37-f33e-46ee-9282-76d61d3a2b0e-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-65c786fdd5-6rpm8\" (UID: \"61470d37-f33e-46ee-9282-76d61d3a2b0e\") " pod="openshift-monitoring/thanos-querier-65c786fdd5-6rpm8" Jul 06 19:11:08.932523 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:08.932495 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/61470d37-f33e-46ee-9282-76d61d3a2b0e-metrics-client-ca\") pod \"thanos-querier-65c786fdd5-6rpm8\" (UID: \"61470d37-f33e-46ee-9282-76d61d3a2b0e\") " pod="openshift-monitoring/thanos-querier-65c786fdd5-6rpm8" Jul 06 19:11:08.933826 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:08.933798 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/61470d37-f33e-46ee-9282-76d61d3a2b0e-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-65c786fdd5-6rpm8\" (UID: \"61470d37-f33e-46ee-9282-76d61d3a2b0e\") " pod="openshift-monitoring/thanos-querier-65c786fdd5-6rpm8" Jul 06 19:11:08.934323 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:08.934282 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/61470d37-f33e-46ee-9282-76d61d3a2b0e-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-65c786fdd5-6rpm8\" (UID: \"61470d37-f33e-46ee-9282-76d61d3a2b0e\") " pod="openshift-monitoring/thanos-querier-65c786fdd5-6rpm8" Jul 06 19:11:08.934401 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:08.934293 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/61470d37-f33e-46ee-9282-76d61d3a2b0e-secret-grpc-tls\") pod \"thanos-querier-65c786fdd5-6rpm8\" (UID: \"61470d37-f33e-46ee-9282-76d61d3a2b0e\") " pod="openshift-monitoring/thanos-querier-65c786fdd5-6rpm8" Jul 06 19:11:08.934401 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:08.934346 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/61470d37-f33e-46ee-9282-76d61d3a2b0e-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-65c786fdd5-6rpm8\" (UID: \"61470d37-f33e-46ee-9282-76d61d3a2b0e\") " pod="openshift-monitoring/thanos-querier-65c786fdd5-6rpm8" Jul 06 19:11:08.934741 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:08.934719 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/61470d37-f33e-46ee-9282-76d61d3a2b0e-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-65c786fdd5-6rpm8\" (UID: \"61470d37-f33e-46ee-9282-76d61d3a2b0e\") " pod="openshift-monitoring/thanos-querier-65c786fdd5-6rpm8" Jul 06 19:11:08.935259 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:08.935235 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/61470d37-f33e-46ee-9282-76d61d3a2b0e-secret-thanos-querier-tls\") pod \"thanos-querier-65c786fdd5-6rpm8\" (UID: \"61470d37-f33e-46ee-9282-76d61d3a2b0e\") " pod="openshift-monitoring/thanos-querier-65c786fdd5-6rpm8" Jul 06 19:11:08.939645 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:08.939624 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sjctn\" (UniqueName: \"kubernetes.io/projected/61470d37-f33e-46ee-9282-76d61d3a2b0e-kube-api-access-sjctn\") pod \"thanos-querier-65c786fdd5-6rpm8\" (UID: \"61470d37-f33e-46ee-9282-76d61d3a2b0e\") " pod="openshift-monitoring/thanos-querier-65c786fdd5-6rpm8" Jul 06 19:11:09.015769 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:09.015734 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-65c786fdd5-6rpm8" Jul 06 19:11:09.148170 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:09.148111 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-65c786fdd5-6rpm8"] Jul 06 19:11:09.150229 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:11:09.150202 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61470d37_f33e_46ee_9282_76d61d3a2b0e.slice/crio-264e5b0efb86e647daa6010e50a91fe3e064c7e44c96fa451a155ab61de7114a WatchSource:0}: Error finding container 264e5b0efb86e647daa6010e50a91fe3e064c7e44c96fa451a155ab61de7114a: Status 404 returned error can't find the container with id 264e5b0efb86e647daa6010e50a91fe3e064c7e44c96fa451a155ab61de7114a Jul 06 19:11:09.622410 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:09.622378 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:11:09.660780 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:09.660740 2566 generic.go:356] "Generic (PLEG): container finished" podID="cc77fe73-70aa-4c81-8d65-fcf55a4b5f9c" containerID="b28420d065f18cabc148467e6e0fe1a9ebbc51d488f2f21f9d54fe9a44cd2bf9" exitCode=0 Jul 06 19:11:09.660992 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:09.660831 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qbxt4" event={"ID":"cc77fe73-70aa-4c81-8d65-fcf55a4b5f9c","Type":"ContainerDied","Data":"b28420d065f18cabc148467e6e0fe1a9ebbc51d488f2f21f9d54fe9a44cd2bf9"} Jul 06 19:11:09.662267 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:09.662229 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-65c786fdd5-6rpm8" event={"ID":"61470d37-f33e-46ee-9282-76d61d3a2b0e","Type":"ContainerStarted","Data":"264e5b0efb86e647daa6010e50a91fe3e064c7e44c96fa451a155ab61de7114a"} Jul 06 19:11:09.690778 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:09.690749 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:11:09.766203 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:09.766173 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:11:09.891763 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:09.891650 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:11:10.065685 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:10.065645 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:11:10.667529 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:10.667495 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qbxt4" event={"ID":"cc77fe73-70aa-4c81-8d65-fcf55a4b5f9c","Type":"ContainerStarted","Data":"a531e5dba4a5c2da606ad95c731818536ce0dbc5e2d33818432cdcf2c1fbcc23"} Jul 06 19:11:10.667693 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:10.667543 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qbxt4" event={"ID":"cc77fe73-70aa-4c81-8d65-fcf55a4b5f9c","Type":"ContainerStarted","Data":"7b7d67b989e7daef4c8362168a694dc4d9acdff128cdb994237d6e172865e010"} Jul 06 19:11:10.686922 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:10.686871 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-qbxt4" podStartSLOduration=3.5509835450000002 podStartE2EDuration="4.686854572s" totalImagesPullingTime="1.064253568s" totalInitContainerRuntime="71.617459ms" isStatefulPod=true podCreationTimestamp="2026-07-06 19:11:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:11:10.686809495 +0000 UTC m=+95.994168443" watchObservedRunningTime="2026-07-06 19:11:10.686854572 +0000 UTC m=+95.994213516" Jul 06 19:11:11.286740 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:11.286706 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:11:11.336903 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:11.336866 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:11:11.491495 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:11.491410 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:11:11.590493 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:11.590411 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:11:11.678875 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:11.678836 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-65c786fdd5-6rpm8" event={"ID":"61470d37-f33e-46ee-9282-76d61d3a2b0e","Type":"ContainerStarted","Data":"20352a8b2b3771a741b9110f1852694ad9e079859e2e3158f992ce73ef5d2225"} Jul 06 19:11:11.679018 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:11.678927 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-65c786fdd5-6rpm8" event={"ID":"61470d37-f33e-46ee-9282-76d61d3a2b0e","Type":"ContainerStarted","Data":"18c3695080da98b0091d88bd2c619aec0b20534d047b2894cd6dd060f4f8c5a7"} Jul 06 19:11:11.679018 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:11.678957 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-65c786fdd5-6rpm8" event={"ID":"61470d37-f33e-46ee-9282-76d61d3a2b0e","Type":"ContainerStarted","Data":"b0966f4e77f6a75c60051d37dcfe375364637236b7705222de8e4d2e489d14db"} Jul 06 19:11:11.744692 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:11.744660 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:11:12.590352 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:12.590323 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-666cddf4fd-nks7n" Jul 06 19:11:12.687337 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:12.687247 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-65c786fdd5-6rpm8" event={"ID":"61470d37-f33e-46ee-9282-76d61d3a2b0e","Type":"ContainerStarted","Data":"84ea7cae340e121b33f57baecaea1a2037060e74c0cc216c51d1b709a19a3df7"} Jul 06 19:11:12.687337 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:12.687320 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-65c786fdd5-6rpm8" event={"ID":"61470d37-f33e-46ee-9282-76d61d3a2b0e","Type":"ContainerStarted","Data":"49fac37f5678426cbbfa229b3db5c0e760178e6e0fbcb29d5f1b579fc2a002d2"} Jul 06 19:11:12.687538 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:12.687349 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-65c786fdd5-6rpm8" event={"ID":"61470d37-f33e-46ee-9282-76d61d3a2b0e","Type":"ContainerStarted","Data":"9dc62910e8f55e5a0544204ef5f24c93427aac5e541c21a03fec4149abc0d4e4"} Jul 06 19:11:12.687538 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:12.687405 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-65c786fdd5-6rpm8" Jul 06 19:11:12.709178 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:12.709114 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-65c786fdd5-6rpm8" podStartSLOduration=1.6710190649999999 podStartE2EDuration="4.709091955s" totalImagesPullingTime="3.03807289s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:11:08 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:11:12.708377234 +0000 UTC m=+98.015736197" watchObservedRunningTime="2026-07-06 19:11:12.709091955 +0000 UTC m=+98.016450906" Jul 06 19:11:12.886704 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:12.886670 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:11:13.598733 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:13.598704 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-788dd79d58-8qj85" Jul 06 19:11:13.722748 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:13.722697 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:11:13.822970 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:13.822936 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:11:13.923876 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:13.923785 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:11:14.023948 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:14.023919 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:11:15.900786 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:15.900751 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:11:16.001651 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:16.001622 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:11:17.611442 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:17.611403 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-666cddf4fd-nks7n" podUID="1ddf025a-43f4-4d0a-826e-06e5137b740f" containerName="registry" containerID="cri-o://0ffd5be061bc9cc2a75c881b3988d243a2e5ee4944b58b7f5212ebd467d15b3d" gracePeriod=30 Jul 06 19:11:17.866081 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:17.865966 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-666cddf4fd-nks7n" Jul 06 19:11:17.903669 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:17.903637 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1ddf025a-43f4-4d0a-826e-06e5137b740f-image-registry-private-configuration\") pod \"1ddf025a-43f4-4d0a-826e-06e5137b740f\" (UID: \"1ddf025a-43f4-4d0a-826e-06e5137b740f\") " Jul 06 19:11:17.903860 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:17.903681 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1ddf025a-43f4-4d0a-826e-06e5137b740f-registry-tls\") pod \"1ddf025a-43f4-4d0a-826e-06e5137b740f\" (UID: \"1ddf025a-43f4-4d0a-826e-06e5137b740f\") " Jul 06 19:11:17.903860 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:17.903736 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1ddf025a-43f4-4d0a-826e-06e5137b740f-ca-trust-extracted\") pod \"1ddf025a-43f4-4d0a-826e-06e5137b740f\" (UID: \"1ddf025a-43f4-4d0a-826e-06e5137b740f\") " Jul 06 19:11:17.903860 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:17.903757 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1ddf025a-43f4-4d0a-826e-06e5137b740f-trusted-ca\") pod \"1ddf025a-43f4-4d0a-826e-06e5137b740f\" (UID: \"1ddf025a-43f4-4d0a-826e-06e5137b740f\") " Jul 06 19:11:17.903860 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:17.903783 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1ddf025a-43f4-4d0a-826e-06e5137b740f-installation-pull-secrets\") pod \"1ddf025a-43f4-4d0a-826e-06e5137b740f\" (UID: \"1ddf025a-43f4-4d0a-826e-06e5137b740f\") " Jul 06 19:11:17.903860 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:17.903816 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1ddf025a-43f4-4d0a-826e-06e5137b740f-bound-sa-token\") pod \"1ddf025a-43f4-4d0a-826e-06e5137b740f\" (UID: \"1ddf025a-43f4-4d0a-826e-06e5137b740f\") " Jul 06 19:11:17.903860 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:17.903847 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v5kjx\" (UniqueName: \"kubernetes.io/projected/1ddf025a-43f4-4d0a-826e-06e5137b740f-kube-api-access-v5kjx\") pod \"1ddf025a-43f4-4d0a-826e-06e5137b740f\" (UID: \"1ddf025a-43f4-4d0a-826e-06e5137b740f\") " Jul 06 19:11:17.904280 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:17.904245 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1ddf025a-43f4-4d0a-826e-06e5137b740f-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "1ddf025a-43f4-4d0a-826e-06e5137b740f" (UID: "1ddf025a-43f4-4d0a-826e-06e5137b740f"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:11:17.906639 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:17.906604 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1ddf025a-43f4-4d0a-826e-06e5137b740f-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "1ddf025a-43f4-4d0a-826e-06e5137b740f" (UID: "1ddf025a-43f4-4d0a-826e-06e5137b740f"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:11:17.906763 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:17.906604 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1ddf025a-43f4-4d0a-826e-06e5137b740f-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "1ddf025a-43f4-4d0a-826e-06e5137b740f" (UID: "1ddf025a-43f4-4d0a-826e-06e5137b740f"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:11:17.906821 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:17.906773 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1ddf025a-43f4-4d0a-826e-06e5137b740f-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "1ddf025a-43f4-4d0a-826e-06e5137b740f" (UID: "1ddf025a-43f4-4d0a-826e-06e5137b740f"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:11:17.906871 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:17.906846 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1ddf025a-43f4-4d0a-826e-06e5137b740f-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "1ddf025a-43f4-4d0a-826e-06e5137b740f" (UID: "1ddf025a-43f4-4d0a-826e-06e5137b740f"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:11:17.907120 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:17.907099 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1ddf025a-43f4-4d0a-826e-06e5137b740f-kube-api-access-v5kjx" (OuterVolumeSpecName: "kube-api-access-v5kjx") pod "1ddf025a-43f4-4d0a-826e-06e5137b740f" (UID: "1ddf025a-43f4-4d0a-826e-06e5137b740f"). InnerVolumeSpecName "kube-api-access-v5kjx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:11:17.915379 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:17.915345 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1ddf025a-43f4-4d0a-826e-06e5137b740f-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "1ddf025a-43f4-4d0a-826e-06e5137b740f" (UID: "1ddf025a-43f4-4d0a-826e-06e5137b740f"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:11:18.004688 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:18.004645 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1ddf025a-43f4-4d0a-826e-06e5137b740f-registry-certificates\") pod \"1ddf025a-43f4-4d0a-826e-06e5137b740f\" (UID: \"1ddf025a-43f4-4d0a-826e-06e5137b740f\") " Jul 06 19:11:18.004892 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:18.004868 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v5kjx\" (UniqueName: \"kubernetes.io/projected/1ddf025a-43f4-4d0a-826e-06e5137b740f-kube-api-access-v5kjx\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:11:18.004892 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:18.004889 2566 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1ddf025a-43f4-4d0a-826e-06e5137b740f-image-registry-private-configuration\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:11:18.004992 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:18.004902 2566 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1ddf025a-43f4-4d0a-826e-06e5137b740f-registry-tls\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:11:18.004992 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:18.004917 2566 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1ddf025a-43f4-4d0a-826e-06e5137b740f-ca-trust-extracted\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:11:18.004992 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:18.004928 2566 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1ddf025a-43f4-4d0a-826e-06e5137b740f-trusted-ca\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:11:18.004992 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:18.004941 2566 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1ddf025a-43f4-4d0a-826e-06e5137b740f-installation-pull-secrets\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:11:18.004992 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:18.004952 2566 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1ddf025a-43f4-4d0a-826e-06e5137b740f-bound-sa-token\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:11:18.005241 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:18.005016 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1ddf025a-43f4-4d0a-826e-06e5137b740f-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "1ddf025a-43f4-4d0a-826e-06e5137b740f" (UID: "1ddf025a-43f4-4d0a-826e-06e5137b740f"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:11:18.105949 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:18.105911 2566 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1ddf025a-43f4-4d0a-826e-06e5137b740f-registry-certificates\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:11:18.696315 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:18.696287 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-65c786fdd5-6rpm8" Jul 06 19:11:18.713811 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:18.713778 2566 generic.go:356] "Generic (PLEG): container finished" podID="1ddf025a-43f4-4d0a-826e-06e5137b740f" containerID="0ffd5be061bc9cc2a75c881b3988d243a2e5ee4944b58b7f5212ebd467d15b3d" exitCode=0 Jul 06 19:11:18.713811 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:18.713816 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-666cddf4fd-nks7n" Jul 06 19:11:18.714025 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:18.713861 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-666cddf4fd-nks7n" event={"ID":"1ddf025a-43f4-4d0a-826e-06e5137b740f","Type":"ContainerDied","Data":"0ffd5be061bc9cc2a75c881b3988d243a2e5ee4944b58b7f5212ebd467d15b3d"} Jul 06 19:11:18.714025 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:18.713916 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-666cddf4fd-nks7n" event={"ID":"1ddf025a-43f4-4d0a-826e-06e5137b740f","Type":"ContainerDied","Data":"66fc40817100aaa67461a6e7578e666d28b18cc2bf087b758d941d268f2f2907"} Jul 06 19:11:18.714025 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:18.713951 2566 scope.go:117] "RemoveContainer" containerID="0ffd5be061bc9cc2a75c881b3988d243a2e5ee4944b58b7f5212ebd467d15b3d" Jul 06 19:11:18.727731 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:18.727696 2566 scope.go:117] "RemoveContainer" containerID="0ffd5be061bc9cc2a75c881b3988d243a2e5ee4944b58b7f5212ebd467d15b3d" Jul 06 19:11:18.728539 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:11:18.728500 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0ffd5be061bc9cc2a75c881b3988d243a2e5ee4944b58b7f5212ebd467d15b3d\": container with ID starting with 0ffd5be061bc9cc2a75c881b3988d243a2e5ee4944b58b7f5212ebd467d15b3d not found: ID does not exist" containerID="0ffd5be061bc9cc2a75c881b3988d243a2e5ee4944b58b7f5212ebd467d15b3d" Jul 06 19:11:18.728679 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:18.728544 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ffd5be061bc9cc2a75c881b3988d243a2e5ee4944b58b7f5212ebd467d15b3d"} err="failed to get container status \"0ffd5be061bc9cc2a75c881b3988d243a2e5ee4944b58b7f5212ebd467d15b3d\": rpc error: code = NotFound desc = could not find container \"0ffd5be061bc9cc2a75c881b3988d243a2e5ee4944b58b7f5212ebd467d15b3d\": container with ID starting with 0ffd5be061bc9cc2a75c881b3988d243a2e5ee4944b58b7f5212ebd467d15b3d not found: ID does not exist" Jul 06 19:11:18.737719 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:18.737685 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-666cddf4fd-nks7n"] Jul 06 19:11:18.744130 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:18.744100 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-666cddf4fd-nks7n"] Jul 06 19:11:18.744276 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:18.744260 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:11:18.809561 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:18.809527 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:11:18.936762 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:18.936727 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:11:19.038589 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:19.038559 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:11:19.235277 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:19.235246 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1ddf025a-43f4-4d0a-826e-06e5137b740f" path="/var/lib/kubelet/pods/1ddf025a-43f4-4d0a-826e-06e5137b740f/volumes" Jul 06 19:11:19.553082 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:19.553044 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-f9z4j" Jul 06 19:11:20.259658 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:20.259610 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:11:20.359742 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:20.359713 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:11:22.119228 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:22.119195 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:11:22.178724 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:22.178673 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:11:23.373138 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:23.373101 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:11:23.473649 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:23.473614 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:11:23.575371 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:23.575340 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:11:29.349127 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:29.349089 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:11:29.442196 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:29.442132 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:11:35.211129 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:35.211096 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:11:35.283464 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:35.283431 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:11:43.827184 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:43.827148 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:11:50.187112 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:50.187063 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:11:50.245024 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:11:50.244992 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:12:13.973166 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:12:13.973134 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:12:17.997611 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:12:17.997578 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:12:25.883397 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:12:25.883304 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:12:26.726214 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:12:26.726172 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:12:34.972960 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:12:34.972912 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:12:41.105483 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:12:41.105448 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:12:41.207411 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:12:41.207371 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:12:52.978019 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:12:52.977972 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:12:53.093819 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:12:53.093785 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:12:56.789418 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:12:56.789384 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:12:56.851467 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:12:56.851433 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:12:56.909273 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:12:56.909245 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:13:05.206015 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:13:05.205978 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:13:05.304892 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:13:05.304859 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:13:19.230767 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:13:19.230729 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:13:19.554661 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:13:19.554632 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:13:24.106625 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:13:24.106595 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:13:25.820251 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:13:25.820218 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:13:25.915880 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:13:25.915844 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:13:26.403517 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:13:26.403477 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ms6f9"] Jul 06 19:13:26.403771 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:13:26.403758 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ddf025a-43f4-4d0a-826e-06e5137b740f" containerName="registry" Jul 06 19:13:26.403816 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:13:26.403773 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ddf025a-43f4-4d0a-826e-06e5137b740f" containerName="registry" Jul 06 19:13:26.403855 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:13:26.403828 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="1ddf025a-43f4-4d0a-826e-06e5137b740f" containerName="registry" Jul 06 19:13:26.406517 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:13:26.406501 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ms6f9" Jul 06 19:13:26.408840 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:13:26.408816 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 06 19:13:26.409495 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:13:26.409481 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 06 19:13:26.409552 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:13:26.409486 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-dr45n\"" Jul 06 19:13:26.415900 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:13:26.415874 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ms6f9"] Jul 06 19:13:26.480307 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:13:26.480271 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/36917fc6-e393-444d-b300-16374e6bac3c-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ms6f9\" (UID: \"36917fc6-e393-444d-b300-16374e6bac3c\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ms6f9" Jul 06 19:13:26.480482 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:13:26.480325 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4wt5d\" (UniqueName: \"kubernetes.io/projected/36917fc6-e393-444d-b300-16374e6bac3c-kube-api-access-4wt5d\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ms6f9\" (UID: \"36917fc6-e393-444d-b300-16374e6bac3c\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ms6f9" Jul 06 19:13:26.480482 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:13:26.480381 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/36917fc6-e393-444d-b300-16374e6bac3c-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ms6f9\" (UID: \"36917fc6-e393-444d-b300-16374e6bac3c\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ms6f9" Jul 06 19:13:26.580757 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:13:26.580724 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/36917fc6-e393-444d-b300-16374e6bac3c-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ms6f9\" (UID: \"36917fc6-e393-444d-b300-16374e6bac3c\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ms6f9" Jul 06 19:13:26.580928 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:13:26.580776 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/36917fc6-e393-444d-b300-16374e6bac3c-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ms6f9\" (UID: \"36917fc6-e393-444d-b300-16374e6bac3c\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ms6f9" Jul 06 19:13:26.580928 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:13:26.580809 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4wt5d\" (UniqueName: \"kubernetes.io/projected/36917fc6-e393-444d-b300-16374e6bac3c-kube-api-access-4wt5d\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ms6f9\" (UID: \"36917fc6-e393-444d-b300-16374e6bac3c\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ms6f9" Jul 06 19:13:26.581131 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:13:26.581110 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/36917fc6-e393-444d-b300-16374e6bac3c-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ms6f9\" (UID: \"36917fc6-e393-444d-b300-16374e6bac3c\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ms6f9" Jul 06 19:13:26.581166 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:13:26.581151 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/36917fc6-e393-444d-b300-16374e6bac3c-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ms6f9\" (UID: \"36917fc6-e393-444d-b300-16374e6bac3c\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ms6f9" Jul 06 19:13:26.588935 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:13:26.588904 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4wt5d\" (UniqueName: \"kubernetes.io/projected/36917fc6-e393-444d-b300-16374e6bac3c-kube-api-access-4wt5d\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ms6f9\" (UID: \"36917fc6-e393-444d-b300-16374e6bac3c\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ms6f9" Jul 06 19:13:26.719383 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:13:26.719300 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ms6f9" Jul 06 19:13:26.845679 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:13:26.845549 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ms6f9"] Jul 06 19:13:26.848346 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:13:26.848318 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36917fc6_e393_444d_b300_16374e6bac3c.slice/crio-dc408447a9b491da80d11e8ca170e4b1a135385a37422d4225b934403a3420d0 WatchSource:0}: Error finding container dc408447a9b491da80d11e8ca170e4b1a135385a37422d4225b934403a3420d0: Status 404 returned error can't find the container with id dc408447a9b491da80d11e8ca170e4b1a135385a37422d4225b934403a3420d0 Jul 06 19:13:27.169090 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:13:27.169042 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ms6f9" event={"ID":"36917fc6-e393-444d-b300-16374e6bac3c","Type":"ContainerStarted","Data":"dc408447a9b491da80d11e8ca170e4b1a135385a37422d4225b934403a3420d0"} Jul 06 19:13:35.199190 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:13:35.199159 2566 generic.go:356] "Generic (PLEG): container finished" podID="36917fc6-e393-444d-b300-16374e6bac3c" containerID="bbdefcc7479f7f6c2644d3f91e85a81b00f4a2aceb85ee3e2996867d4a4cc7aa" exitCode=0 Jul 06 19:13:35.199537 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:13:35.199207 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ms6f9" event={"ID":"36917fc6-e393-444d-b300-16374e6bac3c","Type":"ContainerDied","Data":"bbdefcc7479f7f6c2644d3f91e85a81b00f4a2aceb85ee3e2996867d4a4cc7aa"} Jul 06 19:13:39.222055 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:13:39.222017 2566 generic.go:356] "Generic (PLEG): container finished" podID="36917fc6-e393-444d-b300-16374e6bac3c" containerID="4f261c426201c832ed9967327c18a64777e29b453f3187bb96cce1a12e962bc2" exitCode=0 Jul 06 19:13:39.222436 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:13:39.222104 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ms6f9" event={"ID":"36917fc6-e393-444d-b300-16374e6bac3c","Type":"ContainerDied","Data":"4f261c426201c832ed9967327c18a64777e29b453f3187bb96cce1a12e962bc2"} Jul 06 19:13:47.209597 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:13:47.209556 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:13:47.268653 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:13:47.268613 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:13:48.256810 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:13:48.256778 2566 generic.go:356] "Generic (PLEG): container finished" podID="36917fc6-e393-444d-b300-16374e6bac3c" containerID="222458fcd92e7b9ff61038fa438f5600d2c513389bb06448c0e7f9853e5ede2a" exitCode=0 Jul 06 19:13:48.257262 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:13:48.256870 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ms6f9" event={"ID":"36917fc6-e393-444d-b300-16374e6bac3c","Type":"ContainerDied","Data":"222458fcd92e7b9ff61038fa438f5600d2c513389bb06448c0e7f9853e5ede2a"} Jul 06 19:13:49.436826 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:13:49.436792 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ms6f9" Jul 06 19:13:49.466337 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:13:49.466306 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/36917fc6-e393-444d-b300-16374e6bac3c-bundle\") pod \"36917fc6-e393-444d-b300-16374e6bac3c\" (UID: \"36917fc6-e393-444d-b300-16374e6bac3c\") " Jul 06 19:13:49.466507 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:13:49.466355 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/36917fc6-e393-444d-b300-16374e6bac3c-util\") pod \"36917fc6-e393-444d-b300-16374e6bac3c\" (UID: \"36917fc6-e393-444d-b300-16374e6bac3c\") " Jul 06 19:13:49.466507 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:13:49.466400 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4wt5d\" (UniqueName: \"kubernetes.io/projected/36917fc6-e393-444d-b300-16374e6bac3c-kube-api-access-4wt5d\") pod \"36917fc6-e393-444d-b300-16374e6bac3c\" (UID: \"36917fc6-e393-444d-b300-16374e6bac3c\") " Jul 06 19:13:49.466940 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:13:49.466914 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36917fc6-e393-444d-b300-16374e6bac3c-bundle" (OuterVolumeSpecName: "bundle") pod "36917fc6-e393-444d-b300-16374e6bac3c" (UID: "36917fc6-e393-444d-b300-16374e6bac3c"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:13:49.468671 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:13:49.468648 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/36917fc6-e393-444d-b300-16374e6bac3c-kube-api-access-4wt5d" (OuterVolumeSpecName: "kube-api-access-4wt5d") pod "36917fc6-e393-444d-b300-16374e6bac3c" (UID: "36917fc6-e393-444d-b300-16374e6bac3c"). InnerVolumeSpecName "kube-api-access-4wt5d". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:13:49.470541 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:13:49.470516 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36917fc6-e393-444d-b300-16374e6bac3c-util" (OuterVolumeSpecName: "util") pod "36917fc6-e393-444d-b300-16374e6bac3c" (UID: "36917fc6-e393-444d-b300-16374e6bac3c"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:13:49.567920 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:13:49.567882 2566 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/36917fc6-e393-444d-b300-16374e6bac3c-bundle\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:13:49.567920 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:13:49.567910 2566 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/36917fc6-e393-444d-b300-16374e6bac3c-util\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:13:49.567920 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:13:49.567918 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4wt5d\" (UniqueName: \"kubernetes.io/projected/36917fc6-e393-444d-b300-16374e6bac3c-kube-api-access-4wt5d\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:13:50.270413 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:13:50.270387 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ms6f9" Jul 06 19:13:50.270574 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:13:50.270381 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540ms6f9" event={"ID":"36917fc6-e393-444d-b300-16374e6bac3c","Type":"ContainerDied","Data":"dc408447a9b491da80d11e8ca170e4b1a135385a37422d4225b934403a3420d0"} Jul 06 19:13:50.270574 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:13:50.270489 2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dc408447a9b491da80d11e8ca170e4b1a135385a37422d4225b934403a3420d0" Jul 06 19:13:52.411299 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:13:52.411271 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:13:54.219273 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:13:54.219241 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:13:54.404641 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:13:54.404605 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jctnn"] Jul 06 19:13:54.404946 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:13:54.404930 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36917fc6-e393-444d-b300-16374e6bac3c" containerName="extract" Jul 06 19:13:54.404995 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:13:54.404948 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="36917fc6-e393-444d-b300-16374e6bac3c" containerName="extract" Jul 06 19:13:54.404995 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:13:54.404964 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36917fc6-e393-444d-b300-16374e6bac3c" containerName="pull" Jul 06 19:13:54.404995 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:13:54.404969 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="36917fc6-e393-444d-b300-16374e6bac3c" containerName="pull" Jul 06 19:13:54.404995 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:13:54.404983 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36917fc6-e393-444d-b300-16374e6bac3c" containerName="util" Jul 06 19:13:54.404995 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:13:54.404988 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="36917fc6-e393-444d-b300-16374e6bac3c" containerName="util" Jul 06 19:13:54.405171 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:13:54.405030 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="36917fc6-e393-444d-b300-16374e6bac3c" containerName="extract" Jul 06 19:13:54.408238 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:13:54.408224 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jctnn" Jul 06 19:13:54.410410 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:13:54.410387 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-xrz6b\"" Jul 06 19:13:54.410524 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:13:54.410387 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 06 19:13:54.410524 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:13:54.410387 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 06 19:13:54.418685 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:13:54.418664 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jctnn"] Jul 06 19:13:54.502643 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:13:54.502553 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b7eaa17d-85b9-41a0-ae24-a8a2023b7d00-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-jctnn\" (UID: \"b7eaa17d-85b9-41a0-ae24-a8a2023b7d00\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jctnn" Jul 06 19:13:54.502643 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:13:54.502602 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6kdlw\" (UniqueName: \"kubernetes.io/projected/b7eaa17d-85b9-41a0-ae24-a8a2023b7d00-kube-api-access-6kdlw\") pod \"cert-manager-operator-controller-manager-55c444bf86-jctnn\" (UID: \"b7eaa17d-85b9-41a0-ae24-a8a2023b7d00\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jctnn" Jul 06 19:13:54.603274 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:13:54.603228 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b7eaa17d-85b9-41a0-ae24-a8a2023b7d00-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-jctnn\" (UID: \"b7eaa17d-85b9-41a0-ae24-a8a2023b7d00\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jctnn" Jul 06 19:13:54.603446 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:13:54.603295 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6kdlw\" (UniqueName: \"kubernetes.io/projected/b7eaa17d-85b9-41a0-ae24-a8a2023b7d00-kube-api-access-6kdlw\") pod \"cert-manager-operator-controller-manager-55c444bf86-jctnn\" (UID: \"b7eaa17d-85b9-41a0-ae24-a8a2023b7d00\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jctnn" Jul 06 19:13:54.603605 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:13:54.603588 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b7eaa17d-85b9-41a0-ae24-a8a2023b7d00-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-jctnn\" (UID: \"b7eaa17d-85b9-41a0-ae24-a8a2023b7d00\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jctnn" Jul 06 19:13:54.611192 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:13:54.611172 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6kdlw\" (UniqueName: \"kubernetes.io/projected/b7eaa17d-85b9-41a0-ae24-a8a2023b7d00-kube-api-access-6kdlw\") pod \"cert-manager-operator-controller-manager-55c444bf86-jctnn\" (UID: \"b7eaa17d-85b9-41a0-ae24-a8a2023b7d00\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jctnn" Jul 06 19:13:54.717393 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:13:54.717347 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jctnn" Jul 06 19:13:54.847970 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:13:54.847946 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jctnn"] Jul 06 19:13:54.851111 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:13:54.851060 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7eaa17d_85b9_41a0_ae24_a8a2023b7d00.slice/crio-208ac7a6d58eb531e64e5daef7bc9a3683f41d60eec896787a25ddb9ee454868 WatchSource:0}: Error finding container 208ac7a6d58eb531e64e5daef7bc9a3683f41d60eec896787a25ddb9ee454868: Status 404 returned error can't find the container with id 208ac7a6d58eb531e64e5daef7bc9a3683f41d60eec896787a25ddb9ee454868 Jul 06 19:13:55.290629 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:13:55.290547 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jctnn" event={"ID":"b7eaa17d-85b9-41a0-ae24-a8a2023b7d00","Type":"ContainerStarted","Data":"208ac7a6d58eb531e64e5daef7bc9a3683f41d60eec896787a25ddb9ee454868"} Jul 06 19:13:56.031121 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:13:56.031079 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:13:56.302688 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:13:56.302663 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:13:57.298827 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:13:57.298791 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jctnn" event={"ID":"b7eaa17d-85b9-41a0-ae24-a8a2023b7d00","Type":"ContainerStarted","Data":"62f56483bd3903073e39abfa6ff6a45e9f68771861b19fde09c0203bbfcc30f5"} Jul 06 19:13:57.318377 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:13:57.318317 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-jctnn" podStartSLOduration=1.903985228 podStartE2EDuration="3.318299958s" totalImagesPullingTime="1.41431473s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:13:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:13:57.316206201 +0000 UTC m=+262.623565149" watchObservedRunningTime="2026-07-06 19:13:57.318299958 +0000 UTC m=+262.625658905" Jul 06 19:13:57.762962 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:13:57.762873 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:13:57.894081 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:13:57.894032 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:13:58.086490 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:13:58.086458 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:13:58.187446 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:13:58.187413 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-r72kd"] Jul 06 19:13:58.189447 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:13:58.189431 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-r72kd" Jul 06 19:13:58.191709 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:13:58.191681 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 06 19:13:58.191887 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:13:58.191836 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 06 19:13:58.192608 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:13:58.192590 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-b76dz\"" Jul 06 19:13:58.201232 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:13:58.201199 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:13:58.201394 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:13:58.201377 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-r72kd"] Jul 06 19:13:58.234567 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:13:58.234533 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6d77w\" (UniqueName: \"kubernetes.io/projected/1b7392c7-2339-4ae7-bd32-bbfa731c6c1f-kube-api-access-6d77w\") pod \"cert-manager-webhook-6486ffd997-r72kd\" (UID: \"1b7392c7-2339-4ae7-bd32-bbfa731c6c1f\") " pod="cert-manager/cert-manager-webhook-6486ffd997-r72kd" Jul 06 19:13:58.234737 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:13:58.234591 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1b7392c7-2339-4ae7-bd32-bbfa731c6c1f-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-r72kd\" (UID: \"1b7392c7-2339-4ae7-bd32-bbfa731c6c1f\") " pod="cert-manager/cert-manager-webhook-6486ffd997-r72kd" Jul 06 19:13:58.335641 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:13:58.335602 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1b7392c7-2339-4ae7-bd32-bbfa731c6c1f-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-r72kd\" (UID: \"1b7392c7-2339-4ae7-bd32-bbfa731c6c1f\") " pod="cert-manager/cert-manager-webhook-6486ffd997-r72kd" Jul 06 19:13:58.335641 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:13:58.335651 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6d77w\" (UniqueName: \"kubernetes.io/projected/1b7392c7-2339-4ae7-bd32-bbfa731c6c1f-kube-api-access-6d77w\") pod \"cert-manager-webhook-6486ffd997-r72kd\" (UID: \"1b7392c7-2339-4ae7-bd32-bbfa731c6c1f\") " pod="cert-manager/cert-manager-webhook-6486ffd997-r72kd" Jul 06 19:13:58.343751 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:13:58.343686 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6d77w\" (UniqueName: \"kubernetes.io/projected/1b7392c7-2339-4ae7-bd32-bbfa731c6c1f-kube-api-access-6d77w\") pod \"cert-manager-webhook-6486ffd997-r72kd\" (UID: \"1b7392c7-2339-4ae7-bd32-bbfa731c6c1f\") " pod="cert-manager/cert-manager-webhook-6486ffd997-r72kd" Jul 06 19:13:58.343751 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:13:58.343692 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1b7392c7-2339-4ae7-bd32-bbfa731c6c1f-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-r72kd\" (UID: \"1b7392c7-2339-4ae7-bd32-bbfa731c6c1f\") " pod="cert-manager/cert-manager-webhook-6486ffd997-r72kd" Jul 06 19:13:58.498651 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:13:58.498612 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-r72kd" Jul 06 19:13:58.628588 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:13:58.628493 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-r72kd"] Jul 06 19:13:58.632482 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:13:58.632456 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1b7392c7_2339_4ae7_bd32_bbfa731c6c1f.slice/crio-9404db3fc4f46316f73aaa50045aa9aad78bd02a4ac9724c88310f996baac2c7 WatchSource:0}: Error finding container 9404db3fc4f46316f73aaa50045aa9aad78bd02a4ac9724c88310f996baac2c7: Status 404 returned error can't find the container with id 9404db3fc4f46316f73aaa50045aa9aad78bd02a4ac9724c88310f996baac2c7 Jul 06 19:13:58.665981 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:13:58.665946 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkvjpk"] Jul 06 19:13:58.668642 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:13:58.668625 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkvjpk" Jul 06 19:13:58.671030 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:13:58.671005 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 06 19:13:58.671172 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:13:58.671043 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 06 19:13:58.671172 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:13:58.671087 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-dr45n\"" Jul 06 19:13:58.677632 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:13:58.677597 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkvjpk"] Jul 06 19:13:58.738989 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:13:58.738949 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hrskh\" (UniqueName: \"kubernetes.io/projected/d277e4ce-118b-4ccb-9ed6-fc6ecff1326f-kube-api-access-hrskh\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkvjpk\" (UID: \"d277e4ce-118b-4ccb-9ed6-fc6ecff1326f\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkvjpk" Jul 06 19:13:58.738989 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:13:58.738986 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d277e4ce-118b-4ccb-9ed6-fc6ecff1326f-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkvjpk\" (UID: \"d277e4ce-118b-4ccb-9ed6-fc6ecff1326f\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkvjpk" Jul 06 19:13:58.739243 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:13:58.739014 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d277e4ce-118b-4ccb-9ed6-fc6ecff1326f-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkvjpk\" (UID: \"d277e4ce-118b-4ccb-9ed6-fc6ecff1326f\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkvjpk" Jul 06 19:13:58.840400 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:13:58.840356 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d277e4ce-118b-4ccb-9ed6-fc6ecff1326f-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkvjpk\" (UID: \"d277e4ce-118b-4ccb-9ed6-fc6ecff1326f\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkvjpk" Jul 06 19:13:58.840612 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:13:58.840472 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hrskh\" (UniqueName: \"kubernetes.io/projected/d277e4ce-118b-4ccb-9ed6-fc6ecff1326f-kube-api-access-hrskh\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkvjpk\" (UID: \"d277e4ce-118b-4ccb-9ed6-fc6ecff1326f\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkvjpk" Jul 06 19:13:58.840612 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:13:58.840503 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d277e4ce-118b-4ccb-9ed6-fc6ecff1326f-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkvjpk\" (UID: \"d277e4ce-118b-4ccb-9ed6-fc6ecff1326f\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkvjpk" Jul 06 19:13:58.840790 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:13:58.840768 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d277e4ce-118b-4ccb-9ed6-fc6ecff1326f-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkvjpk\" (UID: \"d277e4ce-118b-4ccb-9ed6-fc6ecff1326f\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkvjpk" Jul 06 19:13:58.840860 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:13:58.840837 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d277e4ce-118b-4ccb-9ed6-fc6ecff1326f-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkvjpk\" (UID: \"d277e4ce-118b-4ccb-9ed6-fc6ecff1326f\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkvjpk" Jul 06 19:13:58.849815 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:13:58.849781 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hrskh\" (UniqueName: \"kubernetes.io/projected/d277e4ce-118b-4ccb-9ed6-fc6ecff1326f-kube-api-access-hrskh\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkvjpk\" (UID: \"d277e4ce-118b-4ccb-9ed6-fc6ecff1326f\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkvjpk" Jul 06 19:13:58.978537 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:13:58.978443 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkvjpk" Jul 06 19:13:59.124690 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:13:59.124511 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkvjpk"] Jul 06 19:13:59.127956 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:13:59.127925 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd277e4ce_118b_4ccb_9ed6_fc6ecff1326f.slice/crio-61c0b941518237699644ee43af11e38ca30cb7f285bae5b0cbf6ef7c3ba6ee59 WatchSource:0}: Error finding container 61c0b941518237699644ee43af11e38ca30cb7f285bae5b0cbf6ef7c3ba6ee59: Status 404 returned error can't find the container with id 61c0b941518237699644ee43af11e38ca30cb7f285bae5b0cbf6ef7c3ba6ee59 Jul 06 19:13:59.309503 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:13:59.309463 2566 generic.go:356] "Generic (PLEG): container finished" podID="d277e4ce-118b-4ccb-9ed6-fc6ecff1326f" containerID="b928b0a2411fb4ff1fa3a862c6baf31c426526f40e751c497e107f5d9867f300" exitCode=0 Jul 06 19:13:59.309686 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:13:59.309552 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkvjpk" event={"ID":"d277e4ce-118b-4ccb-9ed6-fc6ecff1326f","Type":"ContainerDied","Data":"b928b0a2411fb4ff1fa3a862c6baf31c426526f40e751c497e107f5d9867f300"} Jul 06 19:13:59.309686 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:13:59.309607 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkvjpk" event={"ID":"d277e4ce-118b-4ccb-9ed6-fc6ecff1326f","Type":"ContainerStarted","Data":"61c0b941518237699644ee43af11e38ca30cb7f285bae5b0cbf6ef7c3ba6ee59"} Jul 06 19:13:59.310858 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:13:59.310833 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-r72kd" event={"ID":"1b7392c7-2339-4ae7-bd32-bbfa731c6c1f","Type":"ContainerStarted","Data":"9404db3fc4f46316f73aaa50045aa9aad78bd02a4ac9724c88310f996baac2c7"} Jul 06 19:14:00.389705 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:00.389672 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:14:00.610158 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:00.610117 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:14:02.325664 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:02.325628 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-r72kd" event={"ID":"1b7392c7-2339-4ae7-bd32-bbfa731c6c1f","Type":"ContainerStarted","Data":"cc0d29d5dcff16edf1509e449e486801cfa9ed8c16c1b7a309a2546bc63c25c8"} Jul 06 19:14:02.326057 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:02.325766 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-r72kd" Jul 06 19:14:02.327438 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:02.327413 2566 generic.go:356] "Generic (PLEG): container finished" podID="d277e4ce-118b-4ccb-9ed6-fc6ecff1326f" containerID="130a35aeffe0c596337ed29aa4b4cd160ded24bca6347c073b4a2bd941d00283" exitCode=0 Jul 06 19:14:02.327549 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:02.327447 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkvjpk" event={"ID":"d277e4ce-118b-4ccb-9ed6-fc6ecff1326f","Type":"ContainerDied","Data":"130a35aeffe0c596337ed29aa4b4cd160ded24bca6347c073b4a2bd941d00283"} Jul 06 19:14:02.341109 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:02.341044 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-r72kd" podStartSLOduration=1.2547886080000001 podStartE2EDuration="4.341030375s" totalImagesPullingTime="3.086241767s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:13:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:14:02.340164289 +0000 UTC m=+267.647523233" watchObservedRunningTime="2026-07-06 19:14:02.341030375 +0000 UTC m=+267.648389320" Jul 06 19:14:03.014459 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:03.014425 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:14:03.068604 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:03.068564 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:14:03.333751 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:03.333712 2566 generic.go:356] "Generic (PLEG): container finished" podID="d277e4ce-118b-4ccb-9ed6-fc6ecff1326f" containerID="8d01c4d122b12eacade3e1b67d7a4b1f556a3520ff87f6171134a69189a917c3" exitCode=0 Jul 06 19:14:03.334194 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:03.333786 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkvjpk" event={"ID":"d277e4ce-118b-4ccb-9ed6-fc6ecff1326f","Type":"ContainerDied","Data":"8d01c4d122b12eacade3e1b67d7a4b1f556a3520ff87f6171134a69189a917c3"} Jul 06 19:14:04.467853 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:04.467823 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkvjpk" Jul 06 19:14:04.590726 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:04.590613 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d277e4ce-118b-4ccb-9ed6-fc6ecff1326f-util\") pod \"d277e4ce-118b-4ccb-9ed6-fc6ecff1326f\" (UID: \"d277e4ce-118b-4ccb-9ed6-fc6ecff1326f\") " Jul 06 19:14:04.590726 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:04.590700 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hrskh\" (UniqueName: \"kubernetes.io/projected/d277e4ce-118b-4ccb-9ed6-fc6ecff1326f-kube-api-access-hrskh\") pod \"d277e4ce-118b-4ccb-9ed6-fc6ecff1326f\" (UID: \"d277e4ce-118b-4ccb-9ed6-fc6ecff1326f\") " Jul 06 19:14:04.590947 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:04.590763 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d277e4ce-118b-4ccb-9ed6-fc6ecff1326f-bundle\") pod \"d277e4ce-118b-4ccb-9ed6-fc6ecff1326f\" (UID: \"d277e4ce-118b-4ccb-9ed6-fc6ecff1326f\") " Jul 06 19:14:04.591225 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:04.591196 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d277e4ce-118b-4ccb-9ed6-fc6ecff1326f-bundle" (OuterVolumeSpecName: "bundle") pod "d277e4ce-118b-4ccb-9ed6-fc6ecff1326f" (UID: "d277e4ce-118b-4ccb-9ed6-fc6ecff1326f"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:14:04.592923 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:04.592903 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d277e4ce-118b-4ccb-9ed6-fc6ecff1326f-kube-api-access-hrskh" (OuterVolumeSpecName: "kube-api-access-hrskh") pod "d277e4ce-118b-4ccb-9ed6-fc6ecff1326f" (UID: "d277e4ce-118b-4ccb-9ed6-fc6ecff1326f"). InnerVolumeSpecName "kube-api-access-hrskh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:14:04.595971 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:04.595944 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d277e4ce-118b-4ccb-9ed6-fc6ecff1326f-util" (OuterVolumeSpecName: "util") pod "d277e4ce-118b-4ccb-9ed6-fc6ecff1326f" (UID: "d277e4ce-118b-4ccb-9ed6-fc6ecff1326f"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:14:04.692134 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:04.692059 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hrskh\" (UniqueName: \"kubernetes.io/projected/d277e4ce-118b-4ccb-9ed6-fc6ecff1326f-kube-api-access-hrskh\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:14:04.692134 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:04.692125 2566 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d277e4ce-118b-4ccb-9ed6-fc6ecff1326f-bundle\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:14:04.692134 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:04.692134 2566 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d277e4ce-118b-4ccb-9ed6-fc6ecff1326f-util\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:14:05.343724 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:05.343689 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkvjpk" Jul 06 19:14:05.343882 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:05.343688 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fkvjpk" event={"ID":"d277e4ce-118b-4ccb-9ed6-fc6ecff1326f","Type":"ContainerDied","Data":"61c0b941518237699644ee43af11e38ca30cb7f285bae5b0cbf6ef7c3ba6ee59"} Jul 06 19:14:05.343882 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:05.343806 2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="61c0b941518237699644ee43af11e38ca30cb7f285bae5b0cbf6ef7c3ba6ee59" Jul 06 19:14:06.281792 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:06.281754 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-4vsw7"] Jul 06 19:14:06.282184 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:06.282057 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d277e4ce-118b-4ccb-9ed6-fc6ecff1326f" containerName="extract" Jul 06 19:14:06.282184 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:06.282082 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="d277e4ce-118b-4ccb-9ed6-fc6ecff1326f" containerName="extract" Jul 06 19:14:06.282184 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:06.282090 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d277e4ce-118b-4ccb-9ed6-fc6ecff1326f" containerName="util" Jul 06 19:14:06.282184 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:06.282095 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="d277e4ce-118b-4ccb-9ed6-fc6ecff1326f" containerName="util" Jul 06 19:14:06.282184 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:06.282102 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d277e4ce-118b-4ccb-9ed6-fc6ecff1326f" containerName="pull" Jul 06 19:14:06.282184 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:06.282107 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="d277e4ce-118b-4ccb-9ed6-fc6ecff1326f" containerName="pull" Jul 06 19:14:06.282184 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:06.282152 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="d277e4ce-118b-4ccb-9ed6-fc6ecff1326f" containerName="extract" Jul 06 19:14:06.283966 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:06.283950 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-4vsw7" Jul 06 19:14:06.286185 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:06.286162 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-7db8h\"" Jul 06 19:14:06.295468 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:06.295447 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-4vsw7"] Jul 06 19:14:06.403673 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:06.403634 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d98fx\" (UniqueName: \"kubernetes.io/projected/7de5f8c2-5974-4b0b-8a59-809e3388b26f-kube-api-access-d98fx\") pod \"cert-manager-cainjector-559cdd8c87-4vsw7\" (UID: \"7de5f8c2-5974-4b0b-8a59-809e3388b26f\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-4vsw7" Jul 06 19:14:06.403864 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:06.403691 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7de5f8c2-5974-4b0b-8a59-809e3388b26f-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-4vsw7\" (UID: \"7de5f8c2-5974-4b0b-8a59-809e3388b26f\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-4vsw7" Jul 06 19:14:06.504781 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:06.504745 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d98fx\" (UniqueName: \"kubernetes.io/projected/7de5f8c2-5974-4b0b-8a59-809e3388b26f-kube-api-access-d98fx\") pod \"cert-manager-cainjector-559cdd8c87-4vsw7\" (UID: \"7de5f8c2-5974-4b0b-8a59-809e3388b26f\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-4vsw7" Jul 06 19:14:06.504932 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:06.504790 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7de5f8c2-5974-4b0b-8a59-809e3388b26f-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-4vsw7\" (UID: \"7de5f8c2-5974-4b0b-8a59-809e3388b26f\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-4vsw7" Jul 06 19:14:06.513397 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:06.513367 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7de5f8c2-5974-4b0b-8a59-809e3388b26f-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-4vsw7\" (UID: \"7de5f8c2-5974-4b0b-8a59-809e3388b26f\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-4vsw7" Jul 06 19:14:06.513550 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:06.513431 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d98fx\" (UniqueName: \"kubernetes.io/projected/7de5f8c2-5974-4b0b-8a59-809e3388b26f-kube-api-access-d98fx\") pod \"cert-manager-cainjector-559cdd8c87-4vsw7\" (UID: \"7de5f8c2-5974-4b0b-8a59-809e3388b26f\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-4vsw7" Jul 06 19:14:06.592601 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:06.592565 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-4vsw7" Jul 06 19:14:06.745125 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:06.745100 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-4vsw7"] Jul 06 19:14:06.747543 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:14:06.747514 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7de5f8c2_5974_4b0b_8a59_809e3388b26f.slice/crio-d7ab3251b0988f42b39f1a3efcf020700740cf590c985acf4d56ea652f3b2644 WatchSource:0}: Error finding container d7ab3251b0988f42b39f1a3efcf020700740cf590c985acf4d56ea652f3b2644: Status 404 returned error can't find the container with id d7ab3251b0988f42b39f1a3efcf020700740cf590c985acf4d56ea652f3b2644 Jul 06 19:14:07.353810 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:07.353774 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-4vsw7" event={"ID":"7de5f8c2-5974-4b0b-8a59-809e3388b26f","Type":"ContainerStarted","Data":"ae88658d6b3095c0289630790af08a7e8eafe8994328d708aba0d2cac035d44b"} Jul 06 19:14:07.354218 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:07.353822 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-4vsw7" event={"ID":"7de5f8c2-5974-4b0b-8a59-809e3388b26f","Type":"ContainerStarted","Data":"d7ab3251b0988f42b39f1a3efcf020700740cf590c985acf4d56ea652f3b2644"} Jul 06 19:14:07.369948 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:07.369890 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-4vsw7" podStartSLOduration=1.369870331 podStartE2EDuration="1.369870331s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:14:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:14:07.368659047 +0000 UTC m=+272.676017996" watchObservedRunningTime="2026-07-06 19:14:07.369870331 +0000 UTC m=+272.677229280" Jul 06 19:14:07.510815 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:07.510783 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:14:07.610062 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:07.609968 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:14:07.998473 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:07.998395 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:14:08.337131 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:08.337098 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-r72kd" Jul 06 19:14:09.264606 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:09.264565 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:14:09.361077 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:09.361043 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:14:11.247086 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:11.247049 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zkrxk"] Jul 06 19:14:11.249161 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:11.249140 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zkrxk" Jul 06 19:14:11.251283 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:11.251261 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 06 19:14:11.251283 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:11.251282 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 06 19:14:11.251456 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:11.251401 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-c47zj\"" Jul 06 19:14:11.263650 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:11.260137 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zkrxk"] Jul 06 19:14:11.352274 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:11.352224 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9j7zm\" (UniqueName: \"kubernetes.io/projected/cfefa92a-56b5-4754-8269-4e840af96bc5-kube-api-access-9j7zm\") pod \"openshift-lws-operator-fd8ccff4c-zkrxk\" (UID: \"cfefa92a-56b5-4754-8269-4e840af96bc5\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zkrxk" Jul 06 19:14:11.352482 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:11.352297 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cfefa92a-56b5-4754-8269-4e840af96bc5-tmp\") pod \"openshift-lws-operator-fd8ccff4c-zkrxk\" (UID: \"cfefa92a-56b5-4754-8269-4e840af96bc5\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zkrxk" Jul 06 19:14:11.453423 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:11.453385 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9j7zm\" (UniqueName: \"kubernetes.io/projected/cfefa92a-56b5-4754-8269-4e840af96bc5-kube-api-access-9j7zm\") pod \"openshift-lws-operator-fd8ccff4c-zkrxk\" (UID: \"cfefa92a-56b5-4754-8269-4e840af96bc5\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zkrxk" Jul 06 19:14:11.453614 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:11.453435 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cfefa92a-56b5-4754-8269-4e840af96bc5-tmp\") pod \"openshift-lws-operator-fd8ccff4c-zkrxk\" (UID: \"cfefa92a-56b5-4754-8269-4e840af96bc5\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zkrxk" Jul 06 19:14:11.453788 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:11.453769 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cfefa92a-56b5-4754-8269-4e840af96bc5-tmp\") pod \"openshift-lws-operator-fd8ccff4c-zkrxk\" (UID: \"cfefa92a-56b5-4754-8269-4e840af96bc5\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zkrxk" Jul 06 19:14:11.461829 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:11.461798 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9j7zm\" (UniqueName: \"kubernetes.io/projected/cfefa92a-56b5-4754-8269-4e840af96bc5-kube-api-access-9j7zm\") pod \"openshift-lws-operator-fd8ccff4c-zkrxk\" (UID: \"cfefa92a-56b5-4754-8269-4e840af96bc5\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zkrxk" Jul 06 19:14:11.558789 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:11.558734 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zkrxk" Jul 06 19:14:11.685641 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:11.685617 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zkrxk"] Jul 06 19:14:11.688267 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:14:11.688228 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcfefa92a_56b5_4754_8269_4e840af96bc5.slice/crio-38b10ed9c9bdfbf34d5067055598ebc6f20f9f30447812fd42d485381220735b WatchSource:0}: Error finding container 38b10ed9c9bdfbf34d5067055598ebc6f20f9f30447812fd42d485381220735b: Status 404 returned error can't find the container with id 38b10ed9c9bdfbf34d5067055598ebc6f20f9f30447812fd42d485381220735b Jul 06 19:14:12.377710 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:12.377674 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zkrxk" event={"ID":"cfefa92a-56b5-4754-8269-4e840af96bc5","Type":"ContainerStarted","Data":"38b10ed9c9bdfbf34d5067055598ebc6f20f9f30447812fd42d485381220735b"} Jul 06 19:14:13.385613 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:13.385478 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zkrxk" event={"ID":"cfefa92a-56b5-4754-8269-4e840af96bc5","Type":"ContainerStarted","Data":"b8506fd71f21defcd9fce9a3a7849b3c3afd4c930bf980d7f1bb6f9c47d575d0"} Jul 06 19:14:13.407304 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:13.407248 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-zkrxk" podStartSLOduration=0.895788432 podStartE2EDuration="2.407229871s" totalImagesPullingTime="1.511441439s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:14:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:14:13.405713977 +0000 UTC m=+278.713072926" watchObservedRunningTime="2026-07-06 19:14:13.407229871 +0000 UTC m=+278.714588829" Jul 06 19:14:14.107379 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:14.107296 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:14:14.209134 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:14.209100 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:14:16.135979 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:16.135947 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbfxj4"] Jul 06 19:14:16.138254 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:16.138238 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbfxj4" Jul 06 19:14:16.140486 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:16.140464 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 06 19:14:16.141350 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:16.141317 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-dr45n\"" Jul 06 19:14:16.141350 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:16.141319 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 06 19:14:16.147273 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:16.147249 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbfxj4"] Jul 06 19:14:16.241533 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:16.241498 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:14:16.296656 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:16.296614 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f2766740-2a0b-4970-96ba-04ba0314011d-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbfxj4\" (UID: \"f2766740-2a0b-4970-96ba-04ba0314011d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbfxj4" Jul 06 19:14:16.296840 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:16.296668 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cjzf2\" (UniqueName: \"kubernetes.io/projected/f2766740-2a0b-4970-96ba-04ba0314011d-kube-api-access-cjzf2\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbfxj4\" (UID: \"f2766740-2a0b-4970-96ba-04ba0314011d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbfxj4" Jul 06 19:14:16.296840 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:16.296738 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f2766740-2a0b-4970-96ba-04ba0314011d-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbfxj4\" (UID: \"f2766740-2a0b-4970-96ba-04ba0314011d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbfxj4" Jul 06 19:14:16.397164 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:16.397058 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f2766740-2a0b-4970-96ba-04ba0314011d-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbfxj4\" (UID: \"f2766740-2a0b-4970-96ba-04ba0314011d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbfxj4" Jul 06 19:14:16.397164 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:16.397127 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f2766740-2a0b-4970-96ba-04ba0314011d-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbfxj4\" (UID: \"f2766740-2a0b-4970-96ba-04ba0314011d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbfxj4" Jul 06 19:14:16.397164 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:16.397155 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cjzf2\" (UniqueName: \"kubernetes.io/projected/f2766740-2a0b-4970-96ba-04ba0314011d-kube-api-access-cjzf2\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbfxj4\" (UID: \"f2766740-2a0b-4970-96ba-04ba0314011d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbfxj4" Jul 06 19:14:16.397441 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:16.397423 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f2766740-2a0b-4970-96ba-04ba0314011d-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbfxj4\" (UID: \"f2766740-2a0b-4970-96ba-04ba0314011d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbfxj4" Jul 06 19:14:16.397475 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:16.397449 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f2766740-2a0b-4970-96ba-04ba0314011d-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbfxj4\" (UID: \"f2766740-2a0b-4970-96ba-04ba0314011d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbfxj4" Jul 06 19:14:16.405786 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:16.405758 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cjzf2\" (UniqueName: \"kubernetes.io/projected/f2766740-2a0b-4970-96ba-04ba0314011d-kube-api-access-cjzf2\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbfxj4\" (UID: \"f2766740-2a0b-4970-96ba-04ba0314011d\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbfxj4" Jul 06 19:14:16.447768 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:16.447729 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbfxj4" Jul 06 19:14:16.577390 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:16.577361 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbfxj4"] Jul 06 19:14:16.579710 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:14:16.579683 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2766740_2a0b_4970_96ba_04ba0314011d.slice/crio-7ee7f1df1e611bae34c5357ff602402fa26d45ce1e831ac1691935f7febadf6d WatchSource:0}: Error finding container 7ee7f1df1e611bae34c5357ff602402fa26d45ce1e831ac1691935f7febadf6d: Status 404 returned error can't find the container with id 7ee7f1df1e611bae34c5357ff602402fa26d45ce1e831ac1691935f7febadf6d Jul 06 19:14:17.402291 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:17.402253 2566 generic.go:356] "Generic (PLEG): container finished" podID="f2766740-2a0b-4970-96ba-04ba0314011d" containerID="1579e147cefec216af47469fcdc07ee00497eabacb037762babf36271753251d" exitCode=0 Jul 06 19:14:17.402671 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:17.402326 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbfxj4" event={"ID":"f2766740-2a0b-4970-96ba-04ba0314011d","Type":"ContainerDied","Data":"1579e147cefec216af47469fcdc07ee00497eabacb037762babf36271753251d"} Jul 06 19:14:17.402671 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:17.402375 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbfxj4" event={"ID":"f2766740-2a0b-4970-96ba-04ba0314011d","Type":"ContainerStarted","Data":"7ee7f1df1e611bae34c5357ff602402fa26d45ce1e831ac1691935f7febadf6d"} Jul 06 19:14:19.412278 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:19.412240 2566 generic.go:356] "Generic (PLEG): container finished" podID="f2766740-2a0b-4970-96ba-04ba0314011d" containerID="0b1a0885be7c7682b1b4a1184c877c199fef3b5a390c37382870f2ed1b612e2e" exitCode=0 Jul 06 19:14:19.412278 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:19.412283 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbfxj4" event={"ID":"f2766740-2a0b-4970-96ba-04ba0314011d","Type":"ContainerDied","Data":"0b1a0885be7c7682b1b4a1184c877c199fef3b5a390c37382870f2ed1b612e2e"} Jul 06 19:14:20.418115 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:20.418059 2566 generic.go:356] "Generic (PLEG): container finished" podID="f2766740-2a0b-4970-96ba-04ba0314011d" containerID="ea93ac589587bf439aad12bc4178b9027a3797a1a708341245444ef9a7fc3781" exitCode=0 Jul 06 19:14:20.418115 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:20.418099 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbfxj4" event={"ID":"f2766740-2a0b-4970-96ba-04ba0314011d","Type":"ContainerDied","Data":"ea93ac589587bf439aad12bc4178b9027a3797a1a708341245444ef9a7fc3781"} Jul 06 19:14:21.548361 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:21.548336 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbfxj4" Jul 06 19:14:21.641661 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:21.641622 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f2766740-2a0b-4970-96ba-04ba0314011d-util\") pod \"f2766740-2a0b-4970-96ba-04ba0314011d\" (UID: \"f2766740-2a0b-4970-96ba-04ba0314011d\") " Jul 06 19:14:21.641661 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:21.641661 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f2766740-2a0b-4970-96ba-04ba0314011d-bundle\") pod \"f2766740-2a0b-4970-96ba-04ba0314011d\" (UID: \"f2766740-2a0b-4970-96ba-04ba0314011d\") " Jul 06 19:14:21.641905 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:21.641725 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cjzf2\" (UniqueName: \"kubernetes.io/projected/f2766740-2a0b-4970-96ba-04ba0314011d-kube-api-access-cjzf2\") pod \"f2766740-2a0b-4970-96ba-04ba0314011d\" (UID: \"f2766740-2a0b-4970-96ba-04ba0314011d\") " Jul 06 19:14:21.642410 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:21.642384 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f2766740-2a0b-4970-96ba-04ba0314011d-bundle" (OuterVolumeSpecName: "bundle") pod "f2766740-2a0b-4970-96ba-04ba0314011d" (UID: "f2766740-2a0b-4970-96ba-04ba0314011d"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:14:21.643898 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:21.643877 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f2766740-2a0b-4970-96ba-04ba0314011d-kube-api-access-cjzf2" (OuterVolumeSpecName: "kube-api-access-cjzf2") pod "f2766740-2a0b-4970-96ba-04ba0314011d" (UID: "f2766740-2a0b-4970-96ba-04ba0314011d"). InnerVolumeSpecName "kube-api-access-cjzf2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:14:21.646808 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:21.646770 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f2766740-2a0b-4970-96ba-04ba0314011d-util" (OuterVolumeSpecName: "util") pod "f2766740-2a0b-4970-96ba-04ba0314011d" (UID: "f2766740-2a0b-4970-96ba-04ba0314011d"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:14:21.742565 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:21.742475 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cjzf2\" (UniqueName: \"kubernetes.io/projected/f2766740-2a0b-4970-96ba-04ba0314011d-kube-api-access-cjzf2\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:14:21.742565 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:21.742504 2566 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f2766740-2a0b-4970-96ba-04ba0314011d-util\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:14:21.742565 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:21.742517 2566 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f2766740-2a0b-4970-96ba-04ba0314011d-bundle\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:14:22.428486 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:22.428458 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbfxj4" Jul 06 19:14:22.428486 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:22.428475 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbfxj4" event={"ID":"f2766740-2a0b-4970-96ba-04ba0314011d","Type":"ContainerDied","Data":"7ee7f1df1e611bae34c5357ff602402fa26d45ce1e831ac1691935f7febadf6d"} Jul 06 19:14:22.428744 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:22.428516 2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7ee7f1df1e611bae34c5357ff602402fa26d45ce1e831ac1691935f7febadf6d" Jul 06 19:14:34.510709 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:34.510665 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:14:34.608844 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:34.608809 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:14:34.843183 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:34.843150 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:14:34.956624 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:34.956571 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:14:35.081153 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:35.081128 2566 kubelet.go:1648] "Image garbage collection succeeded" Jul 06 19:14:36.043278 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:36.043247 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-55c47dfb7f-tbk57"] Jul 06 19:14:36.044500 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:36.043537 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f2766740-2a0b-4970-96ba-04ba0314011d" containerName="util" Jul 06 19:14:36.044500 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:36.043548 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="f2766740-2a0b-4970-96ba-04ba0314011d" containerName="util" Jul 06 19:14:36.044500 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:36.043561 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f2766740-2a0b-4970-96ba-04ba0314011d" containerName="pull" Jul 06 19:14:36.044500 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:36.043566 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="f2766740-2a0b-4970-96ba-04ba0314011d" containerName="pull" Jul 06 19:14:36.044500 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:36.043576 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f2766740-2a0b-4970-96ba-04ba0314011d" containerName="extract" Jul 06 19:14:36.044500 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:36.043581 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="f2766740-2a0b-4970-96ba-04ba0314011d" containerName="extract" Jul 06 19:14:36.044500 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:36.043630 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="f2766740-2a0b-4970-96ba-04ba0314011d" containerName="extract" Jul 06 19:14:36.045246 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:36.045231 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-55c47dfb7f-tbk57" Jul 06 19:14:36.047535 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:36.047508 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 06 19:14:36.047754 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:36.047739 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 06 19:14:36.047880 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:36.047864 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 06 19:14:36.047947 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:36.047924 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 06 19:14:36.048020 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:36.048000 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-wpwt9\"" Jul 06 19:14:36.054699 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:36.054677 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f4693d1d-13dd-4b20-a725-f722f97cee41-apiservice-cert\") pod \"opendatahub-operator-controller-manager-55c47dfb7f-tbk57\" (UID: \"f4693d1d-13dd-4b20-a725-f722f97cee41\") " pod="opendatahub/opendatahub-operator-controller-manager-55c47dfb7f-tbk57" Jul 06 19:14:36.054784 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:36.054711 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f4693d1d-13dd-4b20-a725-f722f97cee41-webhook-cert\") pod \"opendatahub-operator-controller-manager-55c47dfb7f-tbk57\" (UID: \"f4693d1d-13dd-4b20-a725-f722f97cee41\") " pod="opendatahub/opendatahub-operator-controller-manager-55c47dfb7f-tbk57" Jul 06 19:14:36.054784 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:36.054744 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f4693d1d-13dd-4b20-a725-f722f97cee41-manifests\") pod \"opendatahub-operator-controller-manager-55c47dfb7f-tbk57\" (UID: \"f4693d1d-13dd-4b20-a725-f722f97cee41\") " pod="opendatahub/opendatahub-operator-controller-manager-55c47dfb7f-tbk57" Jul 06 19:14:36.054784 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:36.054764 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tjjhq\" (UniqueName: \"kubernetes.io/projected/f4693d1d-13dd-4b20-a725-f722f97cee41-kube-api-access-tjjhq\") pod \"opendatahub-operator-controller-manager-55c47dfb7f-tbk57\" (UID: \"f4693d1d-13dd-4b20-a725-f722f97cee41\") " pod="opendatahub/opendatahub-operator-controller-manager-55c47dfb7f-tbk57" Jul 06 19:14:36.054882 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:36.054786 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f4693d1d-13dd-4b20-a725-f722f97cee41-tmp\") pod \"opendatahub-operator-controller-manager-55c47dfb7f-tbk57\" (UID: \"f4693d1d-13dd-4b20-a725-f722f97cee41\") " pod="opendatahub/opendatahub-operator-controller-manager-55c47dfb7f-tbk57" Jul 06 19:14:36.067564 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:36.067532 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-55c47dfb7f-tbk57"] Jul 06 19:14:36.155848 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:36.155803 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f4693d1d-13dd-4b20-a725-f722f97cee41-apiservice-cert\") pod \"opendatahub-operator-controller-manager-55c47dfb7f-tbk57\" (UID: \"f4693d1d-13dd-4b20-a725-f722f97cee41\") " pod="opendatahub/opendatahub-operator-controller-manager-55c47dfb7f-tbk57" Jul 06 19:14:36.156051 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:36.155861 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f4693d1d-13dd-4b20-a725-f722f97cee41-webhook-cert\") pod \"opendatahub-operator-controller-manager-55c47dfb7f-tbk57\" (UID: \"f4693d1d-13dd-4b20-a725-f722f97cee41\") " pod="opendatahub/opendatahub-operator-controller-manager-55c47dfb7f-tbk57" Jul 06 19:14:36.156051 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:36.155893 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f4693d1d-13dd-4b20-a725-f722f97cee41-manifests\") pod \"opendatahub-operator-controller-manager-55c47dfb7f-tbk57\" (UID: \"f4693d1d-13dd-4b20-a725-f722f97cee41\") " pod="opendatahub/opendatahub-operator-controller-manager-55c47dfb7f-tbk57" Jul 06 19:14:36.156051 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:36.155919 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tjjhq\" (UniqueName: \"kubernetes.io/projected/f4693d1d-13dd-4b20-a725-f722f97cee41-kube-api-access-tjjhq\") pod \"opendatahub-operator-controller-manager-55c47dfb7f-tbk57\" (UID: \"f4693d1d-13dd-4b20-a725-f722f97cee41\") " pod="opendatahub/opendatahub-operator-controller-manager-55c47dfb7f-tbk57" Jul 06 19:14:36.156051 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:36.155951 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f4693d1d-13dd-4b20-a725-f722f97cee41-tmp\") pod \"opendatahub-operator-controller-manager-55c47dfb7f-tbk57\" (UID: \"f4693d1d-13dd-4b20-a725-f722f97cee41\") " pod="opendatahub/opendatahub-operator-controller-manager-55c47dfb7f-tbk57" Jul 06 19:14:36.156385 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:36.156355 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f4693d1d-13dd-4b20-a725-f722f97cee41-manifests\") pod \"opendatahub-operator-controller-manager-55c47dfb7f-tbk57\" (UID: \"f4693d1d-13dd-4b20-a725-f722f97cee41\") " pod="opendatahub/opendatahub-operator-controller-manager-55c47dfb7f-tbk57" Jul 06 19:14:36.158245 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:36.158224 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f4693d1d-13dd-4b20-a725-f722f97cee41-tmp\") pod \"opendatahub-operator-controller-manager-55c47dfb7f-tbk57\" (UID: \"f4693d1d-13dd-4b20-a725-f722f97cee41\") " pod="opendatahub/opendatahub-operator-controller-manager-55c47dfb7f-tbk57" Jul 06 19:14:36.158365 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:36.158347 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f4693d1d-13dd-4b20-a725-f722f97cee41-apiservice-cert\") pod \"opendatahub-operator-controller-manager-55c47dfb7f-tbk57\" (UID: \"f4693d1d-13dd-4b20-a725-f722f97cee41\") " pod="opendatahub/opendatahub-operator-controller-manager-55c47dfb7f-tbk57" Jul 06 19:14:36.158549 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:36.158530 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f4693d1d-13dd-4b20-a725-f722f97cee41-webhook-cert\") pod \"opendatahub-operator-controller-manager-55c47dfb7f-tbk57\" (UID: \"f4693d1d-13dd-4b20-a725-f722f97cee41\") " pod="opendatahub/opendatahub-operator-controller-manager-55c47dfb7f-tbk57" Jul 06 19:14:36.168463 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:36.168432 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tjjhq\" (UniqueName: \"kubernetes.io/projected/f4693d1d-13dd-4b20-a725-f722f97cee41-kube-api-access-tjjhq\") pod \"opendatahub-operator-controller-manager-55c47dfb7f-tbk57\" (UID: \"f4693d1d-13dd-4b20-a725-f722f97cee41\") " pod="opendatahub/opendatahub-operator-controller-manager-55c47dfb7f-tbk57" Jul 06 19:14:36.218938 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:36.218903 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:14:36.317043 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:36.316966 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:14:36.354999 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:36.354965 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-55c47dfb7f-tbk57" Jul 06 19:14:36.495529 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:36.495499 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-55c47dfb7f-tbk57"] Jul 06 19:14:36.498021 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:14:36.497994 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4693d1d_13dd_4b20_a725_f722f97cee41.slice/crio-51740595c26e759291e7f3fea1aca145f37c77fb30d08b7787e01289ae5b4afb WatchSource:0}: Error finding container 51740595c26e759291e7f3fea1aca145f37c77fb30d08b7787e01289ae5b4afb: Status 404 returned error can't find the container with id 51740595c26e759291e7f3fea1aca145f37c77fb30d08b7787e01289ae5b4afb Jul 06 19:14:36.499738 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:36.499723 2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 19:14:36.856273 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:36.856238 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:14:37.493753 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:37.493710 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-55c47dfb7f-tbk57" event={"ID":"f4693d1d-13dd-4b20-a725-f722f97cee41","Type":"ContainerStarted","Data":"51740595c26e759291e7f3fea1aca145f37c77fb30d08b7787e01289ae5b4afb"} Jul 06 19:14:38.028085 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:38.028035 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:14:40.508951 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:40.508911 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-55c47dfb7f-tbk57" event={"ID":"f4693d1d-13dd-4b20-a725-f722f97cee41","Type":"ContainerStarted","Data":"0d88d07f014080f1f6ee40e2fcd33d419246f1cc411c2f7c541d03f8c8a6e7a7"} Jul 06 19:14:41.514403 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:41.514368 2566 generic.go:356] "Generic (PLEG): container finished" podID="f4693d1d-13dd-4b20-a725-f722f97cee41" containerID="0d88d07f014080f1f6ee40e2fcd33d419246f1cc411c2f7c541d03f8c8a6e7a7" exitCode=0 Jul 06 19:14:41.514771 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:41.514451 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-55c47dfb7f-tbk57" event={"ID":"f4693d1d-13dd-4b20-a725-f722f97cee41","Type":"ContainerDied","Data":"0d88d07f014080f1f6ee40e2fcd33d419246f1cc411c2f7c541d03f8c8a6e7a7"} Jul 06 19:14:42.521128 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:42.521022 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-55c47dfb7f-tbk57" event={"ID":"f4693d1d-13dd-4b20-a725-f722f97cee41","Type":"ContainerStarted","Data":"21b2ca856637450547dc88c307515480039c218e53123db5a605b217d2a45c17"} Jul 06 19:14:42.521467 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:42.521208 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-55c47dfb7f-tbk57" Jul 06 19:14:42.542173 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:42.542120 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-55c47dfb7f-tbk57" podStartSLOduration=0.982591335 podStartE2EDuration="6.542102335s" totalImagesPullingTime="4.474543644s" totalInitContainerRuntime="1.084967356s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:14:36 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:14:42.540207306 +0000 UTC m=+307.847566255" watchObservedRunningTime="2026-07-06 19:14:42.542102335 +0000 UTC m=+307.849461281" Jul 06 19:14:47.835006 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:47.834972 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:14:47.937870 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:47.937837 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:14:53.252773 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:53.252737 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:14:53.352997 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:53.352964 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:14:53.528654 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:53.528566 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-55c47dfb7f-tbk57" Jul 06 19:14:56.577496 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:56.577459 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:14:56.718176 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:56.718137 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:14:57.038608 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:57.038573 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:14:57.150634 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:57.150591 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-754576df6f-jrtrg"] Jul 06 19:14:57.154120 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:57.154100 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-754576df6f-jrtrg" Jul 06 19:14:57.157870 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:57.157844 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 06 19:14:57.158047 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:57.157847 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 06 19:14:57.158555 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:57.158535 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 06 19:14:57.158646 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:57.158538 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-rdfhv\"" Jul 06 19:14:57.158646 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:57.158572 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 06 19:14:57.174369 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:57.174349 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-754576df6f-jrtrg"] Jul 06 19:14:57.213388 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:57.213358 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e1be9243-b0bd-493c-8e06-e3f6c2c8a9a7-tls-certs\") pod \"kube-auth-proxy-754576df6f-jrtrg\" (UID: \"e1be9243-b0bd-493c-8e06-e3f6c2c8a9a7\") " pod="openshift-ingress/kube-auth-proxy-754576df6f-jrtrg" Jul 06 19:14:57.213537 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:57.213401 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4xrf8\" (UniqueName: \"kubernetes.io/projected/e1be9243-b0bd-493c-8e06-e3f6c2c8a9a7-kube-api-access-4xrf8\") pod \"kube-auth-proxy-754576df6f-jrtrg\" (UID: \"e1be9243-b0bd-493c-8e06-e3f6c2c8a9a7\") " pod="openshift-ingress/kube-auth-proxy-754576df6f-jrtrg" Jul 06 19:14:57.213537 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:57.213425 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e1be9243-b0bd-493c-8e06-e3f6c2c8a9a7-tmp\") pod \"kube-auth-proxy-754576df6f-jrtrg\" (UID: \"e1be9243-b0bd-493c-8e06-e3f6c2c8a9a7\") " pod="openshift-ingress/kube-auth-proxy-754576df6f-jrtrg" Jul 06 19:14:57.314608 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:57.314501 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e1be9243-b0bd-493c-8e06-e3f6c2c8a9a7-tls-certs\") pod \"kube-auth-proxy-754576df6f-jrtrg\" (UID: \"e1be9243-b0bd-493c-8e06-e3f6c2c8a9a7\") " pod="openshift-ingress/kube-auth-proxy-754576df6f-jrtrg" Jul 06 19:14:57.314608 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:57.314559 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4xrf8\" (UniqueName: \"kubernetes.io/projected/e1be9243-b0bd-493c-8e06-e3f6c2c8a9a7-kube-api-access-4xrf8\") pod \"kube-auth-proxy-754576df6f-jrtrg\" (UID: \"e1be9243-b0bd-493c-8e06-e3f6c2c8a9a7\") " pod="openshift-ingress/kube-auth-proxy-754576df6f-jrtrg" Jul 06 19:14:57.314608 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:57.314594 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e1be9243-b0bd-493c-8e06-e3f6c2c8a9a7-tmp\") pod \"kube-auth-proxy-754576df6f-jrtrg\" (UID: \"e1be9243-b0bd-493c-8e06-e3f6c2c8a9a7\") " pod="openshift-ingress/kube-auth-proxy-754576df6f-jrtrg" Jul 06 19:14:57.314903 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:14:57.314672 2566 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found Jul 06 19:14:57.314903 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:14:57.314762 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1be9243-b0bd-493c-8e06-e3f6c2c8a9a7-tls-certs podName:e1be9243-b0bd-493c-8e06-e3f6c2c8a9a7 nodeName:}" failed. No retries permitted until 2026-07-06 19:14:57.814740286 +0000 UTC m=+323.122099220 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/e1be9243-b0bd-493c-8e06-e3f6c2c8a9a7-tls-certs") pod "kube-auth-proxy-754576df6f-jrtrg" (UID: "e1be9243-b0bd-493c-8e06-e3f6c2c8a9a7") : secret "kube-auth-proxy-tls" not found Jul 06 19:14:57.317017 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:57.316992 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e1be9243-b0bd-493c-8e06-e3f6c2c8a9a7-tmp\") pod \"kube-auth-proxy-754576df6f-jrtrg\" (UID: \"e1be9243-b0bd-493c-8e06-e3f6c2c8a9a7\") " pod="openshift-ingress/kube-auth-proxy-754576df6f-jrtrg" Jul 06 19:14:57.334110 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:57.334080 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4xrf8\" (UniqueName: \"kubernetes.io/projected/e1be9243-b0bd-493c-8e06-e3f6c2c8a9a7-kube-api-access-4xrf8\") pod \"kube-auth-proxy-754576df6f-jrtrg\" (UID: \"e1be9243-b0bd-493c-8e06-e3f6c2c8a9a7\") " pod="openshift-ingress/kube-auth-proxy-754576df6f-jrtrg" Jul 06 19:14:57.404550 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:57.404519 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:14:57.457380 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:57.457339 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-sdhp4"] Jul 06 19:14:57.460743 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:57.460719 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-sdhp4" Jul 06 19:14:57.462971 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:57.462947 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-bvgls\"" Jul 06 19:14:57.463102 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:57.462982 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jul 06 19:14:57.470890 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:57.470861 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-sdhp4"] Jul 06 19:14:57.503755 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:57.503723 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:14:57.515821 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:57.515790 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c10a5b93-4175-4ac7-a141-6c248f02bd8c-cert\") pod \"odh-model-controller-7998c67dc4-sdhp4\" (UID: \"c10a5b93-4175-4ac7-a141-6c248f02bd8c\") " pod="opendatahub/odh-model-controller-7998c67dc4-sdhp4" Jul 06 19:14:57.515989 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:57.515829 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-drskf\" (UniqueName: \"kubernetes.io/projected/c10a5b93-4175-4ac7-a141-6c248f02bd8c-kube-api-access-drskf\") pod \"odh-model-controller-7998c67dc4-sdhp4\" (UID: \"c10a5b93-4175-4ac7-a141-6c248f02bd8c\") " pod="opendatahub/odh-model-controller-7998c67dc4-sdhp4" Jul 06 19:14:57.609429 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:57.609340 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:14:57.616404 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:57.616375 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c10a5b93-4175-4ac7-a141-6c248f02bd8c-cert\") pod \"odh-model-controller-7998c67dc4-sdhp4\" (UID: \"c10a5b93-4175-4ac7-a141-6c248f02bd8c\") " pod="opendatahub/odh-model-controller-7998c67dc4-sdhp4" Jul 06 19:14:57.616516 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:57.616417 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-drskf\" (UniqueName: \"kubernetes.io/projected/c10a5b93-4175-4ac7-a141-6c248f02bd8c-kube-api-access-drskf\") pod \"odh-model-controller-7998c67dc4-sdhp4\" (UID: \"c10a5b93-4175-4ac7-a141-6c248f02bd8c\") " pod="opendatahub/odh-model-controller-7998c67dc4-sdhp4" Jul 06 19:14:57.616559 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:14:57.616520 2566 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jul 06 19:14:57.616592 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:14:57.616586 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c10a5b93-4175-4ac7-a141-6c248f02bd8c-cert podName:c10a5b93-4175-4ac7-a141-6c248f02bd8c nodeName:}" failed. No retries permitted until 2026-07-06 19:14:58.116571809 +0000 UTC m=+323.423930739 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/c10a5b93-4175-4ac7-a141-6c248f02bd8c-cert") pod "odh-model-controller-7998c67dc4-sdhp4" (UID: "c10a5b93-4175-4ac7-a141-6c248f02bd8c") : secret "odh-model-controller-webhook-cert" not found Jul 06 19:14:57.629815 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:57.629789 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-drskf\" (UniqueName: \"kubernetes.io/projected/c10a5b93-4175-4ac7-a141-6c248f02bd8c-kube-api-access-drskf\") pod \"odh-model-controller-7998c67dc4-sdhp4\" (UID: \"c10a5b93-4175-4ac7-a141-6c248f02bd8c\") " pod="opendatahub/odh-model-controller-7998c67dc4-sdhp4" Jul 06 19:14:57.704764 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:57.704733 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:14:57.817919 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:57.817878 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e1be9243-b0bd-493c-8e06-e3f6c2c8a9a7-tls-certs\") pod \"kube-auth-proxy-754576df6f-jrtrg\" (UID: \"e1be9243-b0bd-493c-8e06-e3f6c2c8a9a7\") " pod="openshift-ingress/kube-auth-proxy-754576df6f-jrtrg" Jul 06 19:14:57.820457 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:57.820413 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e1be9243-b0bd-493c-8e06-e3f6c2c8a9a7-tls-certs\") pod \"kube-auth-proxy-754576df6f-jrtrg\" (UID: \"e1be9243-b0bd-493c-8e06-e3f6c2c8a9a7\") " pod="openshift-ingress/kube-auth-proxy-754576df6f-jrtrg" Jul 06 19:14:57.827755 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:57.827730 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:14:58.064295 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:58.064261 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-754576df6f-jrtrg" Jul 06 19:14:58.120177 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:58.120133 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c10a5b93-4175-4ac7-a141-6c248f02bd8c-cert\") pod \"odh-model-controller-7998c67dc4-sdhp4\" (UID: \"c10a5b93-4175-4ac7-a141-6c248f02bd8c\") " pod="opendatahub/odh-model-controller-7998c67dc4-sdhp4" Jul 06 19:14:58.122868 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:58.122842 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c10a5b93-4175-4ac7-a141-6c248f02bd8c-cert\") pod \"odh-model-controller-7998c67dc4-sdhp4\" (UID: \"c10a5b93-4175-4ac7-a141-6c248f02bd8c\") " pod="opendatahub/odh-model-controller-7998c67dc4-sdhp4" Jul 06 19:14:58.200754 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:58.200730 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-754576df6f-jrtrg"] Jul 06 19:14:58.203051 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:14:58.203027 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1be9243_b0bd_493c_8e06_e3f6c2c8a9a7.slice/crio-e7cc0c8d5cfc7db91013514f2450d48e6ad6a855191da6c2778a8a2a0f3371fc WatchSource:0}: Error finding container e7cc0c8d5cfc7db91013514f2450d48e6ad6a855191da6c2778a8a2a0f3371fc: Status 404 returned error can't find the container with id e7cc0c8d5cfc7db91013514f2450d48e6ad6a855191da6c2778a8a2a0f3371fc Jul 06 19:14:58.307813 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:58.307779 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-bl6x9"] Jul 06 19:14:58.312049 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:58.312032 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-bl6x9" Jul 06 19:14:58.316338 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:58.316273 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 06 19:14:58.342235 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:58.342205 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-bl6x9"] Jul 06 19:14:58.348682 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:58.348656 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:14:58.371836 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:58.371796 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-sdhp4" Jul 06 19:14:58.422909 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:58.422870 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b2160904-d8c6-4fd7-a41f-a10ff617640f-tls-certs\") pod \"model-serving-api-54d89bd5fc-bl6x9\" (UID: \"b2160904-d8c6-4fd7-a41f-a10ff617640f\") " pod="opendatahub/model-serving-api-54d89bd5fc-bl6x9" Jul 06 19:14:58.423112 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:58.422924 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hrcvf\" (UniqueName: \"kubernetes.io/projected/b2160904-d8c6-4fd7-a41f-a10ff617640f-kube-api-access-hrcvf\") pod \"model-serving-api-54d89bd5fc-bl6x9\" (UID: \"b2160904-d8c6-4fd7-a41f-a10ff617640f\") " pod="opendatahub/model-serving-api-54d89bd5fc-bl6x9" Jul 06 19:14:58.513500 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:58.513466 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-sdhp4"] Jul 06 19:14:58.514820 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:14:58.514795 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc10a5b93_4175_4ac7_a141_6c248f02bd8c.slice/crio-8b451795499f7946c77ddd29930637c2ce6ea3d497274c358d0b8ab5208ce789 WatchSource:0}: Error finding container 8b451795499f7946c77ddd29930637c2ce6ea3d497274c358d0b8ab5208ce789: Status 404 returned error can't find the container with id 8b451795499f7946c77ddd29930637c2ce6ea3d497274c358d0b8ab5208ce789 Jul 06 19:14:58.524269 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:58.524236 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hrcvf\" (UniqueName: \"kubernetes.io/projected/b2160904-d8c6-4fd7-a41f-a10ff617640f-kube-api-access-hrcvf\") pod \"model-serving-api-54d89bd5fc-bl6x9\" (UID: \"b2160904-d8c6-4fd7-a41f-a10ff617640f\") " pod="opendatahub/model-serving-api-54d89bd5fc-bl6x9" Jul 06 19:14:58.524358 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:58.524349 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b2160904-d8c6-4fd7-a41f-a10ff617640f-tls-certs\") pod \"model-serving-api-54d89bd5fc-bl6x9\" (UID: \"b2160904-d8c6-4fd7-a41f-a10ff617640f\") " pod="opendatahub/model-serving-api-54d89bd5fc-bl6x9" Jul 06 19:14:58.524476 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:14:58.524463 2566 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found Jul 06 19:14:58.524540 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:14:58.524521 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b2160904-d8c6-4fd7-a41f-a10ff617640f-tls-certs podName:b2160904-d8c6-4fd7-a41f-a10ff617640f nodeName:}" failed. No retries permitted until 2026-07-06 19:14:59.024506254 +0000 UTC m=+324.331865180 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/b2160904-d8c6-4fd7-a41f-a10ff617640f-tls-certs") pod "model-serving-api-54d89bd5fc-bl6x9" (UID: "b2160904-d8c6-4fd7-a41f-a10ff617640f") : secret "model-serving-api-tls" not found Jul 06 19:14:58.539541 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:58.539513 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hrcvf\" (UniqueName: \"kubernetes.io/projected/b2160904-d8c6-4fd7-a41f-a10ff617640f-kube-api-access-hrcvf\") pod \"model-serving-api-54d89bd5fc-bl6x9\" (UID: \"b2160904-d8c6-4fd7-a41f-a10ff617640f\") " pod="opendatahub/model-serving-api-54d89bd5fc-bl6x9" Jul 06 19:14:58.596041 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:58.595940 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-sdhp4" event={"ID":"c10a5b93-4175-4ac7-a141-6c248f02bd8c","Type":"ContainerStarted","Data":"8b451795499f7946c77ddd29930637c2ce6ea3d497274c358d0b8ab5208ce789"} Jul 06 19:14:58.597386 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:58.597359 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-754576df6f-jrtrg" event={"ID":"e1be9243-b0bd-493c-8e06-e3f6c2c8a9a7","Type":"ContainerStarted","Data":"e7cc0c8d5cfc7db91013514f2450d48e6ad6a855191da6c2778a8a2a0f3371fc"} Jul 06 19:14:58.653199 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:58.653164 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-tj6gd"] Jul 06 19:14:58.657419 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:58.657400 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-tj6gd" Jul 06 19:14:58.658529 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:58.658506 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:14:58.659918 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:58.659893 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 06 19:14:58.660030 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:58.659897 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 06 19:14:58.660030 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:58.659925 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 06 19:14:58.660197 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:58.660089 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-6hkf6\"" Jul 06 19:14:58.660271 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:58.660257 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 06 19:14:58.671324 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:58.671299 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-tj6gd"] Jul 06 19:14:58.725977 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:58.725936 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5j7mw\" (UniqueName: \"kubernetes.io/projected/e2443d8a-b4df-4de7-a2f6-7ade12827727-kube-api-access-5j7mw\") pod \"istiod-openshift-gateway-7574459656-tj6gd\" (UID: \"e2443d8a-b4df-4de7-a2f6-7ade12827727\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-tj6gd" Jul 06 19:14:58.725977 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:58.725984 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e2443d8a-b4df-4de7-a2f6-7ade12827727-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-tj6gd\" (UID: \"e2443d8a-b4df-4de7-a2f6-7ade12827727\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-tj6gd" Jul 06 19:14:58.726260 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:58.726029 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e2443d8a-b4df-4de7-a2f6-7ade12827727-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-tj6gd\" (UID: \"e2443d8a-b4df-4de7-a2f6-7ade12827727\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-tj6gd" Jul 06 19:14:58.726260 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:58.726134 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e2443d8a-b4df-4de7-a2f6-7ade12827727-cacerts\") pod \"istiod-openshift-gateway-7574459656-tj6gd\" (UID: \"e2443d8a-b4df-4de7-a2f6-7ade12827727\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-tj6gd" Jul 06 19:14:58.726260 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:58.726161 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e2443d8a-b4df-4de7-a2f6-7ade12827727-local-certs\") pod \"istiod-openshift-gateway-7574459656-tj6gd\" (UID: \"e2443d8a-b4df-4de7-a2f6-7ade12827727\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-tj6gd" Jul 06 19:14:58.726260 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:58.726184 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e2443d8a-b4df-4de7-a2f6-7ade12827727-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-tj6gd\" (UID: \"e2443d8a-b4df-4de7-a2f6-7ade12827727\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-tj6gd" Jul 06 19:14:58.726260 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:58.726214 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e2443d8a-b4df-4de7-a2f6-7ade12827727-istio-token\") pod \"istiod-openshift-gateway-7574459656-tj6gd\" (UID: \"e2443d8a-b4df-4de7-a2f6-7ade12827727\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-tj6gd" Jul 06 19:14:58.730567 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:58.730547 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:14:58.827116 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:58.827061 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5j7mw\" (UniqueName: \"kubernetes.io/projected/e2443d8a-b4df-4de7-a2f6-7ade12827727-kube-api-access-5j7mw\") pod \"istiod-openshift-gateway-7574459656-tj6gd\" (UID: \"e2443d8a-b4df-4de7-a2f6-7ade12827727\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-tj6gd" Jul 06 19:14:58.827291 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:58.827132 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e2443d8a-b4df-4de7-a2f6-7ade12827727-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-tj6gd\" (UID: \"e2443d8a-b4df-4de7-a2f6-7ade12827727\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-tj6gd" Jul 06 19:14:58.827362 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:58.827293 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e2443d8a-b4df-4de7-a2f6-7ade12827727-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-tj6gd\" (UID: \"e2443d8a-b4df-4de7-a2f6-7ade12827727\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-tj6gd" Jul 06 19:14:58.827419 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:58.827366 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e2443d8a-b4df-4de7-a2f6-7ade12827727-cacerts\") pod \"istiod-openshift-gateway-7574459656-tj6gd\" (UID: \"e2443d8a-b4df-4de7-a2f6-7ade12827727\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-tj6gd" Jul 06 19:14:58.827419 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:58.827389 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e2443d8a-b4df-4de7-a2f6-7ade12827727-local-certs\") pod \"istiod-openshift-gateway-7574459656-tj6gd\" (UID: \"e2443d8a-b4df-4de7-a2f6-7ade12827727\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-tj6gd" Jul 06 19:14:58.827419 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:58.827407 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e2443d8a-b4df-4de7-a2f6-7ade12827727-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-tj6gd\" (UID: \"e2443d8a-b4df-4de7-a2f6-7ade12827727\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-tj6gd" Jul 06 19:14:58.827562 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:58.827527 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e2443d8a-b4df-4de7-a2f6-7ade12827727-istio-token\") pod \"istiod-openshift-gateway-7574459656-tj6gd\" (UID: \"e2443d8a-b4df-4de7-a2f6-7ade12827727\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-tj6gd" Jul 06 19:14:58.828041 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:58.827958 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e2443d8a-b4df-4de7-a2f6-7ade12827727-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-tj6gd\" (UID: \"e2443d8a-b4df-4de7-a2f6-7ade12827727\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-tj6gd" Jul 06 19:14:58.829657 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:58.829628 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e2443d8a-b4df-4de7-a2f6-7ade12827727-local-certs\") pod \"istiod-openshift-gateway-7574459656-tj6gd\" (UID: \"e2443d8a-b4df-4de7-a2f6-7ade12827727\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-tj6gd" Jul 06 19:14:58.829846 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:58.829824 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e2443d8a-b4df-4de7-a2f6-7ade12827727-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-tj6gd\" (UID: \"e2443d8a-b4df-4de7-a2f6-7ade12827727\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-tj6gd" Jul 06 19:14:58.830057 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:58.830040 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e2443d8a-b4df-4de7-a2f6-7ade12827727-cacerts\") pod \"istiod-openshift-gateway-7574459656-tj6gd\" (UID: \"e2443d8a-b4df-4de7-a2f6-7ade12827727\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-tj6gd" Jul 06 19:14:58.830142 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:58.830126 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e2443d8a-b4df-4de7-a2f6-7ade12827727-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-tj6gd\" (UID: \"e2443d8a-b4df-4de7-a2f6-7ade12827727\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-tj6gd" Jul 06 19:14:58.834586 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:58.834565 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5j7mw\" (UniqueName: \"kubernetes.io/projected/e2443d8a-b4df-4de7-a2f6-7ade12827727-kube-api-access-5j7mw\") pod \"istiod-openshift-gateway-7574459656-tj6gd\" (UID: \"e2443d8a-b4df-4de7-a2f6-7ade12827727\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-tj6gd" Jul 06 19:14:58.834676 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:58.834597 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e2443d8a-b4df-4de7-a2f6-7ade12827727-istio-token\") pod \"istiod-openshift-gateway-7574459656-tj6gd\" (UID: \"e2443d8a-b4df-4de7-a2f6-7ade12827727\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-tj6gd" Jul 06 19:14:58.903189 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:58.903107 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:14:58.968485 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:58.968446 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-tj6gd" Jul 06 19:14:59.029479 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:59.029430 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b2160904-d8c6-4fd7-a41f-a10ff617640f-tls-certs\") pod \"model-serving-api-54d89bd5fc-bl6x9\" (UID: \"b2160904-d8c6-4fd7-a41f-a10ff617640f\") " pod="opendatahub/model-serving-api-54d89bd5fc-bl6x9" Jul 06 19:14:59.033010 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:59.032983 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b2160904-d8c6-4fd7-a41f-a10ff617640f-tls-certs\") pod \"model-serving-api-54d89bd5fc-bl6x9\" (UID: \"b2160904-d8c6-4fd7-a41f-a10ff617640f\") " pod="opendatahub/model-serving-api-54d89bd5fc-bl6x9" Jul 06 19:14:59.127242 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:59.127211 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-tj6gd"] Jul 06 19:14:59.131586 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:14:59.131553 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2443d8a_b4df_4de7_a2f6_7ade12827727.slice/crio-0aaf89a634a8fa5ff5f9c1b9c9edb2814da07561d0df02fc8d5da76ab07416e3 WatchSource:0}: Error finding container 0aaf89a634a8fa5ff5f9c1b9c9edb2814da07561d0df02fc8d5da76ab07416e3: Status 404 returned error can't find the container with id 0aaf89a634a8fa5ff5f9c1b9c9edb2814da07561d0df02fc8d5da76ab07416e3 Jul 06 19:14:59.231019 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:59.230740 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-bl6x9" Jul 06 19:14:59.402670 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:59.402641 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-bl6x9"] Jul 06 19:14:59.405989 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:14:59.405905 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2160904_d8c6_4fd7_a41f_a10ff617640f.slice/crio-e8437de9f1088bbd7e755c5728c1c9955fe8ce345e3abf32ae238e22fff47603 WatchSource:0}: Error finding container e8437de9f1088bbd7e755c5728c1c9955fe8ce345e3abf32ae238e22fff47603: Status 404 returned error can't find the container with id e8437de9f1088bbd7e755c5728c1c9955fe8ce345e3abf32ae238e22fff47603 Jul 06 19:14:59.536939 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:59.536908 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:14:59.605298 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:59.605254 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-bl6x9" event={"ID":"b2160904-d8c6-4fd7-a41f-a10ff617640f","Type":"ContainerStarted","Data":"e8437de9f1088bbd7e755c5728c1c9955fe8ce345e3abf32ae238e22fff47603"} Jul 06 19:14:59.606620 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:14:59.606591 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-tj6gd" event={"ID":"e2443d8a-b4df-4de7-a2f6-7ade12827727","Type":"ContainerStarted","Data":"0aaf89a634a8fa5ff5f9c1b9c9edb2814da07561d0df02fc8d5da76ab07416e3"} Jul 06 19:15:03.630420 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:03.630373 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 19:15:03.630765 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:03.630471 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 19:15:04.617801 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:04.617765 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:15:04.635178 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:04.635143 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-sdhp4" event={"ID":"c10a5b93-4175-4ac7-a141-6c248f02bd8c","Type":"ContainerStarted","Data":"0fc9cdc71f40888b08274f92c812a6eab5d46d1327eac4fe26d54d459ff38288"} Jul 06 19:15:04.635582 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:04.635251 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-sdhp4" Jul 06 19:15:04.636702 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:04.636680 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-bl6x9" event={"ID":"b2160904-d8c6-4fd7-a41f-a10ff617640f","Type":"ContainerStarted","Data":"063edaaced4c3095f9bfbc5c8f237b29fd4b7f7efc055410b21a37a8c1bc40cb"} Jul 06 19:15:04.636808 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:04.636737 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-bl6x9" Jul 06 19:15:04.638182 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:04.638163 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-tj6gd" event={"ID":"e2443d8a-b4df-4de7-a2f6-7ade12827727","Type":"ContainerStarted","Data":"4edd0deef81e1bf2c4655572feeae84d49b90bb1384bf99d6c3bd84d936acdf1"} Jul 06 19:15:04.638254 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:04.638199 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-tj6gd" Jul 06 19:15:04.639577 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:04.639557 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-754576df6f-jrtrg" event={"ID":"e1be9243-b0bd-493c-8e06-e3f6c2c8a9a7","Type":"ContainerStarted","Data":"6851dd40f78acf19132a1870e17a56932d8a856dcd58993cb3b9efce528536ea"} Jul 06 19:15:04.639986 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:04.639967 2566 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-tj6gd container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 06 19:15:04.640104 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:04.640013 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-tj6gd" podUID="e2443d8a-b4df-4de7-a2f6-7ade12827727" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 19:15:04.668847 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:04.668798 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-sdhp4" podStartSLOduration=2.555660709 podStartE2EDuration="7.66878194s" totalImagesPullingTime="5.113121231s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:14:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:15:04.66767044 +0000 UTC m=+329.975029388" watchObservedRunningTime="2026-07-06 19:15:04.66878194 +0000 UTC m=+329.976140888" Jul 06 19:15:04.701621 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:04.701556 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-tj6gd" podStartSLOduration=2.207178731 podStartE2EDuration="6.701535944s" totalImagesPullingTime="4.494357213s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:14:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:15:04.700373547 +0000 UTC m=+330.007732494" watchObservedRunningTime="2026-07-06 19:15:04.701535944 +0000 UTC m=+330.008894895" Jul 06 19:15:04.719813 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:04.719592 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-bl6x9" podStartSLOduration=2.498174963 podStartE2EDuration="6.719571403s" totalImagesPullingTime="4.22139644s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:14:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:15:04.718926331 +0000 UTC m=+330.026285279" watchObservedRunningTime="2026-07-06 19:15:04.719571403 +0000 UTC m=+330.026930354" Jul 06 19:15:04.724899 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:04.724873 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:15:04.739893 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:04.739843 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-754576df6f-jrtrg" podStartSLOduration=2.304782371 podStartE2EDuration="7.739827987s" totalImagesPullingTime="5.435045616s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:14:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:15:04.738214425 +0000 UTC m=+330.045573371" watchObservedRunningTime="2026-07-06 19:15:04.739827987 +0000 UTC m=+330.047186934" Jul 06 19:15:05.277925 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:05.277890 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4kzvjv"] Jul 06 19:15:05.281360 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:05.281341 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4kzvjv" Jul 06 19:15:05.283613 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:05.283591 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-p4f6x\"" Jul 06 19:15:05.294454 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:05.294429 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4kzvjv"] Jul 06 19:15:05.308504 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:05.308481 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:15:05.385293 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:05.385264 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc73ba95-f338-4688-a38a-ee10bbce473d-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4kzvjv\" (UID: \"fc73ba95-f338-4688-a38a-ee10bbce473d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4kzvjv" Jul 06 19:15:05.385293 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:05.385302 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fc73ba95-f338-4688-a38a-ee10bbce473d-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4kzvjv\" (UID: \"fc73ba95-f338-4688-a38a-ee10bbce473d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4kzvjv" Jul 06 19:15:05.385513 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:05.385418 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fc73ba95-f338-4688-a38a-ee10bbce473d-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4kzvjv\" (UID: \"fc73ba95-f338-4688-a38a-ee10bbce473d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4kzvjv" Jul 06 19:15:05.385513 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:05.385478 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-55znv\" (UniqueName: \"kubernetes.io/projected/fc73ba95-f338-4688-a38a-ee10bbce473d-kube-api-access-55znv\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4kzvjv\" (UID: \"fc73ba95-f338-4688-a38a-ee10bbce473d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4kzvjv" Jul 06 19:15:05.385599 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:05.385515 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fc73ba95-f338-4688-a38a-ee10bbce473d-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4kzvjv\" (UID: \"fc73ba95-f338-4688-a38a-ee10bbce473d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4kzvjv" Jul 06 19:15:05.385599 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:05.385555 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fc73ba95-f338-4688-a38a-ee10bbce473d-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4kzvjv\" (UID: \"fc73ba95-f338-4688-a38a-ee10bbce473d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4kzvjv" Jul 06 19:15:05.385599 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:05.385587 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fc73ba95-f338-4688-a38a-ee10bbce473d-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4kzvjv\" (UID: \"fc73ba95-f338-4688-a38a-ee10bbce473d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4kzvjv" Jul 06 19:15:05.385708 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:05.385623 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc73ba95-f338-4688-a38a-ee10bbce473d-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4kzvjv\" (UID: \"fc73ba95-f338-4688-a38a-ee10bbce473d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4kzvjv" Jul 06 19:15:05.385708 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:05.385648 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fc73ba95-f338-4688-a38a-ee10bbce473d-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4kzvjv\" (UID: \"fc73ba95-f338-4688-a38a-ee10bbce473d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4kzvjv" Jul 06 19:15:05.486481 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:05.486444 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fc73ba95-f338-4688-a38a-ee10bbce473d-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4kzvjv\" (UID: \"fc73ba95-f338-4688-a38a-ee10bbce473d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4kzvjv" Jul 06 19:15:05.486662 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:05.486493 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fc73ba95-f338-4688-a38a-ee10bbce473d-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4kzvjv\" (UID: \"fc73ba95-f338-4688-a38a-ee10bbce473d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4kzvjv" Jul 06 19:15:05.486662 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:05.486532 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-55znv\" (UniqueName: \"kubernetes.io/projected/fc73ba95-f338-4688-a38a-ee10bbce473d-kube-api-access-55znv\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4kzvjv\" (UID: \"fc73ba95-f338-4688-a38a-ee10bbce473d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4kzvjv" Jul 06 19:15:05.486662 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:05.486553 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fc73ba95-f338-4688-a38a-ee10bbce473d-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4kzvjv\" (UID: \"fc73ba95-f338-4688-a38a-ee10bbce473d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4kzvjv" Jul 06 19:15:05.486662 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:05.486588 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fc73ba95-f338-4688-a38a-ee10bbce473d-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4kzvjv\" (UID: \"fc73ba95-f338-4688-a38a-ee10bbce473d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4kzvjv" Jul 06 19:15:05.486662 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:05.486613 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fc73ba95-f338-4688-a38a-ee10bbce473d-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4kzvjv\" (UID: \"fc73ba95-f338-4688-a38a-ee10bbce473d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4kzvjv" Jul 06 19:15:05.486662 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:05.486643 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc73ba95-f338-4688-a38a-ee10bbce473d-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4kzvjv\" (UID: \"fc73ba95-f338-4688-a38a-ee10bbce473d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4kzvjv" Jul 06 19:15:05.486958 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:05.486666 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fc73ba95-f338-4688-a38a-ee10bbce473d-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4kzvjv\" (UID: \"fc73ba95-f338-4688-a38a-ee10bbce473d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4kzvjv" Jul 06 19:15:05.486958 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:05.486714 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc73ba95-f338-4688-a38a-ee10bbce473d-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4kzvjv\" (UID: \"fc73ba95-f338-4688-a38a-ee10bbce473d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4kzvjv" Jul 06 19:15:05.487115 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:05.487085 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fc73ba95-f338-4688-a38a-ee10bbce473d-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4kzvjv\" (UID: \"fc73ba95-f338-4688-a38a-ee10bbce473d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4kzvjv" Jul 06 19:15:05.487115 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:05.487100 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc73ba95-f338-4688-a38a-ee10bbce473d-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4kzvjv\" (UID: \"fc73ba95-f338-4688-a38a-ee10bbce473d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4kzvjv" Jul 06 19:15:05.487260 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:05.487166 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fc73ba95-f338-4688-a38a-ee10bbce473d-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4kzvjv\" (UID: \"fc73ba95-f338-4688-a38a-ee10bbce473d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4kzvjv" Jul 06 19:15:05.487310 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:05.487298 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fc73ba95-f338-4688-a38a-ee10bbce473d-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4kzvjv\" (UID: \"fc73ba95-f338-4688-a38a-ee10bbce473d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4kzvjv" Jul 06 19:15:05.487598 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:05.487530 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fc73ba95-f338-4688-a38a-ee10bbce473d-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4kzvjv\" (UID: \"fc73ba95-f338-4688-a38a-ee10bbce473d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4kzvjv" Jul 06 19:15:05.489152 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:05.489129 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fc73ba95-f338-4688-a38a-ee10bbce473d-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4kzvjv\" (UID: \"fc73ba95-f338-4688-a38a-ee10bbce473d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4kzvjv" Jul 06 19:15:05.489432 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:05.489416 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fc73ba95-f338-4688-a38a-ee10bbce473d-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4kzvjv\" (UID: \"fc73ba95-f338-4688-a38a-ee10bbce473d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4kzvjv" Jul 06 19:15:05.506440 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:05.506411 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fc73ba95-f338-4688-a38a-ee10bbce473d-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4kzvjv\" (UID: \"fc73ba95-f338-4688-a38a-ee10bbce473d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4kzvjv" Jul 06 19:15:05.526335 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:05.526304 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-55znv\" (UniqueName: \"kubernetes.io/projected/fc73ba95-f338-4688-a38a-ee10bbce473d-kube-api-access-55znv\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4kzvjv\" (UID: \"fc73ba95-f338-4688-a38a-ee10bbce473d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4kzvjv" Jul 06 19:15:05.578604 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:05.578571 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:15:05.591731 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:05.591703 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4kzvjv" Jul 06 19:15:05.647032 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:05.647009 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-tj6gd" Jul 06 19:15:05.726338 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:05.726305 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:15:05.733463 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:05.733437 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4kzvjv"] Jul 06 19:15:05.736353 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:15:05.736332 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc73ba95_f338_4688_a38a_ee10bbce473d.slice/crio-ec09aeb14ecc70783848262462ee5a8ccf613138f0e36b7d41ab21f2568ffdb1 WatchSource:0}: Error finding container ec09aeb14ecc70783848262462ee5a8ccf613138f0e36b7d41ab21f2568ffdb1: Status 404 returned error can't find the container with id ec09aeb14ecc70783848262462ee5a8ccf613138f0e36b7d41ab21f2568ffdb1 Jul 06 19:15:06.654088 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:06.654039 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4kzvjv" event={"ID":"fc73ba95-f338-4688-a38a-ee10bbce473d","Type":"ContainerStarted","Data":"ec09aeb14ecc70783848262462ee5a8ccf613138f0e36b7d41ab21f2568ffdb1"} Jul 06 19:15:07.229460 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:07.229427 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:15:07.290664 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:07.290609 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-wkh6n"] Jul 06 19:15:07.294167 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:07.294146 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-wkh6n" Jul 06 19:15:07.297887 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:07.297867 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 06 19:15:07.297988 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:07.297870 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-t7fnh\"" Jul 06 19:15:07.314385 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:07.314361 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-wkh6n"] Jul 06 19:15:07.339625 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:07.339589 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:15:07.403495 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:07.403459 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/abe694ae-dad6-43a4-a8e0-767db8e50929-cert\") pod \"kserve-controller-manager-d6f4b65b8-wkh6n\" (UID: \"abe694ae-dad6-43a4-a8e0-767db8e50929\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-wkh6n" Jul 06 19:15:07.403673 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:07.403518 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c7drk\" (UniqueName: \"kubernetes.io/projected/abe694ae-dad6-43a4-a8e0-767db8e50929-kube-api-access-c7drk\") pod \"kserve-controller-manager-d6f4b65b8-wkh6n\" (UID: \"abe694ae-dad6-43a4-a8e0-767db8e50929\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-wkh6n" Jul 06 19:15:07.427594 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:07.427561 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:15:07.504985 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:07.504897 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/abe694ae-dad6-43a4-a8e0-767db8e50929-cert\") pod \"kserve-controller-manager-d6f4b65b8-wkh6n\" (UID: \"abe694ae-dad6-43a4-a8e0-767db8e50929\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-wkh6n" Jul 06 19:15:07.504985 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:07.504954 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c7drk\" (UniqueName: \"kubernetes.io/projected/abe694ae-dad6-43a4-a8e0-767db8e50929-kube-api-access-c7drk\") pod \"kserve-controller-manager-d6f4b65b8-wkh6n\" (UID: \"abe694ae-dad6-43a4-a8e0-767db8e50929\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-wkh6n" Jul 06 19:15:07.507762 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:07.507736 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/abe694ae-dad6-43a4-a8e0-767db8e50929-cert\") pod \"kserve-controller-manager-d6f4b65b8-wkh6n\" (UID: \"abe694ae-dad6-43a4-a8e0-767db8e50929\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-wkh6n" Jul 06 19:15:07.515526 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:07.515497 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c7drk\" (UniqueName: \"kubernetes.io/projected/abe694ae-dad6-43a4-a8e0-767db8e50929-kube-api-access-c7drk\") pod \"kserve-controller-manager-d6f4b65b8-wkh6n\" (UID: \"abe694ae-dad6-43a4-a8e0-767db8e50929\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-wkh6n" Jul 06 19:15:07.538536 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:07.538497 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:15:07.608406 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:07.608355 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-wkh6n" Jul 06 19:15:08.490142 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:08.490104 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 19:15:08.490602 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:08.490192 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 19:15:08.490602 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:08.490238 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 19:15:08.491564 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:08.491540 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:15:08.560885 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:08.560864 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-wkh6n"] Jul 06 19:15:08.563273 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:15:08.563240 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podabe694ae_dad6_43a4_a8e0_767db8e50929.slice/crio-808c8fa507c6569ee653294d4dd50e84b13e79feb182426c44247ca020174030 WatchSource:0}: Error finding container 808c8fa507c6569ee653294d4dd50e84b13e79feb182426c44247ca020174030: Status 404 returned error can't find the container with id 808c8fa507c6569ee653294d4dd50e84b13e79feb182426c44247ca020174030 Jul 06 19:15:08.563499 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:08.563474 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:15:08.623216 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:08.623186 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:15:08.665410 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:08.665363 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4kzvjv" event={"ID":"fc73ba95-f338-4688-a38a-ee10bbce473d","Type":"ContainerStarted","Data":"279536fe545a51f7beb2347d4b5da1ea40f89784d0a48bfddc8cf07664729315"} Jul 06 19:15:08.666532 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:08.666510 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-wkh6n" event={"ID":"abe694ae-dad6-43a4-a8e0-767db8e50929","Type":"ContainerStarted","Data":"808c8fa507c6569ee653294d4dd50e84b13e79feb182426c44247ca020174030"} Jul 06 19:15:08.705326 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:08.705216 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4kzvjv" podStartSLOduration=0.953853441 podStartE2EDuration="3.705199985s" totalImagesPullingTime="2.751346544s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:15:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:15:08.702757531 +0000 UTC m=+334.010116478" watchObservedRunningTime="2026-07-06 19:15:08.705199985 +0000 UTC m=+334.012558932" Jul 06 19:15:08.787017 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:08.786988 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-tj6gd"] Jul 06 19:15:08.787213 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:08.787191 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-tj6gd" podUID="e2443d8a-b4df-4de7-a2f6-7ade12827727" containerName="discovery" containerID="cri-o://4edd0deef81e1bf2c4655572feeae84d49b90bb1384bf99d6c3bd84d936acdf1" gracePeriod=30 Jul 06 19:15:09.031619 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:09.031592 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-tj6gd" Jul 06 19:15:09.118139 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:09.118107 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e2443d8a-b4df-4de7-a2f6-7ade12827727-local-certs\") pod \"e2443d8a-b4df-4de7-a2f6-7ade12827727\" (UID: \"e2443d8a-b4df-4de7-a2f6-7ade12827727\") " Jul 06 19:15:09.118139 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:09.118145 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e2443d8a-b4df-4de7-a2f6-7ade12827727-istio-csr-dns-cert\") pod \"e2443d8a-b4df-4de7-a2f6-7ade12827727\" (UID: \"e2443d8a-b4df-4de7-a2f6-7ade12827727\") " Jul 06 19:15:09.118381 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:09.118162 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e2443d8a-b4df-4de7-a2f6-7ade12827727-cacerts\") pod \"e2443d8a-b4df-4de7-a2f6-7ade12827727\" (UID: \"e2443d8a-b4df-4de7-a2f6-7ade12827727\") " Jul 06 19:15:09.118381 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:09.118222 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e2443d8a-b4df-4de7-a2f6-7ade12827727-istio-csr-ca-configmap\") pod \"e2443d8a-b4df-4de7-a2f6-7ade12827727\" (UID: \"e2443d8a-b4df-4de7-a2f6-7ade12827727\") " Jul 06 19:15:09.118381 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:09.118260 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5j7mw\" (UniqueName: \"kubernetes.io/projected/e2443d8a-b4df-4de7-a2f6-7ade12827727-kube-api-access-5j7mw\") pod \"e2443d8a-b4df-4de7-a2f6-7ade12827727\" (UID: \"e2443d8a-b4df-4de7-a2f6-7ade12827727\") " Jul 06 19:15:09.118381 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:09.118303 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e2443d8a-b4df-4de7-a2f6-7ade12827727-istio-kubeconfig\") pod \"e2443d8a-b4df-4de7-a2f6-7ade12827727\" (UID: \"e2443d8a-b4df-4de7-a2f6-7ade12827727\") " Jul 06 19:15:09.118381 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:09.118350 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e2443d8a-b4df-4de7-a2f6-7ade12827727-istio-token\") pod \"e2443d8a-b4df-4de7-a2f6-7ade12827727\" (UID: \"e2443d8a-b4df-4de7-a2f6-7ade12827727\") " Jul 06 19:15:09.119008 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:09.118939 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e2443d8a-b4df-4de7-a2f6-7ade12827727-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "e2443d8a-b4df-4de7-a2f6-7ade12827727" (UID: "e2443d8a-b4df-4de7-a2f6-7ade12827727"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:15:09.121460 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:09.121394 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e2443d8a-b4df-4de7-a2f6-7ade12827727-cacerts" (OuterVolumeSpecName: "cacerts") pod "e2443d8a-b4df-4de7-a2f6-7ade12827727" (UID: "e2443d8a-b4df-4de7-a2f6-7ade12827727"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:15:09.121460 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:09.121402 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2443d8a-b4df-4de7-a2f6-7ade12827727-local-certs" (OuterVolumeSpecName: "local-certs") pod "e2443d8a-b4df-4de7-a2f6-7ade12827727" (UID: "e2443d8a-b4df-4de7-a2f6-7ade12827727"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:15:09.121633 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:09.121457 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e2443d8a-b4df-4de7-a2f6-7ade12827727-istio-token" (OuterVolumeSpecName: "istio-token") pod "e2443d8a-b4df-4de7-a2f6-7ade12827727" (UID: "e2443d8a-b4df-4de7-a2f6-7ade12827727"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:15:09.121633 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:09.121484 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e2443d8a-b4df-4de7-a2f6-7ade12827727-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "e2443d8a-b4df-4de7-a2f6-7ade12827727" (UID: "e2443d8a-b4df-4de7-a2f6-7ade12827727"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:15:09.121633 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:09.121497 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e2443d8a-b4df-4de7-a2f6-7ade12827727-kube-api-access-5j7mw" (OuterVolumeSpecName: "kube-api-access-5j7mw") pod "e2443d8a-b4df-4de7-a2f6-7ade12827727" (UID: "e2443d8a-b4df-4de7-a2f6-7ade12827727"). InnerVolumeSpecName "kube-api-access-5j7mw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:15:09.121756 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:09.121725 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e2443d8a-b4df-4de7-a2f6-7ade12827727-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "e2443d8a-b4df-4de7-a2f6-7ade12827727" (UID: "e2443d8a-b4df-4de7-a2f6-7ade12827727"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:15:09.219970 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:09.219872 2566 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e2443d8a-b4df-4de7-a2f6-7ade12827727-istio-kubeconfig\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:15:09.219970 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:09.219918 2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e2443d8a-b4df-4de7-a2f6-7ade12827727-istio-token\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:15:09.219970 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:09.219931 2566 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e2443d8a-b4df-4de7-a2f6-7ade12827727-local-certs\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:15:09.219970 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:09.219944 2566 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e2443d8a-b4df-4de7-a2f6-7ade12827727-istio-csr-dns-cert\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:15:09.219970 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:09.219957 2566 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e2443d8a-b4df-4de7-a2f6-7ade12827727-cacerts\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:15:09.219970 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:09.219972 2566 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e2443d8a-b4df-4de7-a2f6-7ade12827727-istio-csr-ca-configmap\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:15:09.220286 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:09.219984 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5j7mw\" (UniqueName: \"kubernetes.io/projected/e2443d8a-b4df-4de7-a2f6-7ade12827727-kube-api-access-5j7mw\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:15:09.592913 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:09.592880 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4kzvjv" Jul 06 19:15:09.597787 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:09.597762 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4kzvjv" Jul 06 19:15:09.672564 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:09.672524 2566 generic.go:356] "Generic (PLEG): container finished" podID="e2443d8a-b4df-4de7-a2f6-7ade12827727" containerID="4edd0deef81e1bf2c4655572feeae84d49b90bb1384bf99d6c3bd84d936acdf1" exitCode=0 Jul 06 19:15:09.672711 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:09.672582 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-tj6gd" Jul 06 19:15:09.672711 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:09.672603 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-tj6gd" event={"ID":"e2443d8a-b4df-4de7-a2f6-7ade12827727","Type":"ContainerDied","Data":"4edd0deef81e1bf2c4655572feeae84d49b90bb1384bf99d6c3bd84d936acdf1"} Jul 06 19:15:09.672711 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:09.672655 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-tj6gd" event={"ID":"e2443d8a-b4df-4de7-a2f6-7ade12827727","Type":"ContainerDied","Data":"0aaf89a634a8fa5ff5f9c1b9c9edb2814da07561d0df02fc8d5da76ab07416e3"} Jul 06 19:15:09.672711 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:09.672674 2566 scope.go:117] "RemoveContainer" containerID="4edd0deef81e1bf2c4655572feeae84d49b90bb1384bf99d6c3bd84d936acdf1" Jul 06 19:15:09.673047 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:09.673024 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4kzvjv" Jul 06 19:15:09.674298 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:09.674276 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4kzvjv" Jul 06 19:15:09.681762 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:09.681746 2566 scope.go:117] "RemoveContainer" containerID="4edd0deef81e1bf2c4655572feeae84d49b90bb1384bf99d6c3bd84d936acdf1" Jul 06 19:15:09.682011 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:15:09.681994 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4edd0deef81e1bf2c4655572feeae84d49b90bb1384bf99d6c3bd84d936acdf1\": container with ID starting with 4edd0deef81e1bf2c4655572feeae84d49b90bb1384bf99d6c3bd84d936acdf1 not found: ID does not exist" containerID="4edd0deef81e1bf2c4655572feeae84d49b90bb1384bf99d6c3bd84d936acdf1" Jul 06 19:15:09.682084 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:09.682021 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4edd0deef81e1bf2c4655572feeae84d49b90bb1384bf99d6c3bd84d936acdf1"} err="failed to get container status \"4edd0deef81e1bf2c4655572feeae84d49b90bb1384bf99d6c3bd84d936acdf1\": rpc error: code = NotFound desc = could not find container \"4edd0deef81e1bf2c4655572feeae84d49b90bb1384bf99d6c3bd84d936acdf1\": container with ID starting with 4edd0deef81e1bf2c4655572feeae84d49b90bb1384bf99d6c3bd84d936acdf1 not found: ID does not exist" Jul 06 19:15:09.711341 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:09.711306 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-tj6gd"] Jul 06 19:15:09.724565 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:09.724528 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-tj6gd"] Jul 06 19:15:11.236464 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:11.236432 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e2443d8a-b4df-4de7-a2f6-7ade12827727" path="/var/lib/kubelet/pods/e2443d8a-b4df-4de7-a2f6-7ade12827727/volumes" Jul 06 19:15:14.696365 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:14.696330 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-wkh6n" event={"ID":"abe694ae-dad6-43a4-a8e0-767db8e50929","Type":"ContainerStarted","Data":"3bc18cb682fb69d1c490aed6b987292d65b5cebf393ec67e77775ac61cd8450a"} Jul 06 19:15:14.696750 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:14.696438 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-wkh6n" Jul 06 19:15:14.737681 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:14.737626 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-wkh6n" podStartSLOduration=1.809310361 podStartE2EDuration="7.737578654s" totalImagesPullingTime="5.928268293s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:15:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:15:14.736342889 +0000 UTC m=+340.043701836" watchObservedRunningTime="2026-07-06 19:15:14.737578654 +0000 UTC m=+340.044937606" Jul 06 19:15:15.648322 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:15.648293 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-sdhp4" Jul 06 19:15:15.650283 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:15.650266 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-bl6x9" Jul 06 19:15:18.855486 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:18.855453 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:15:18.951142 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:18.951110 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:15:45.705896 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:45.705868 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-wkh6n" Jul 06 19:15:57.952362 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:57.952307 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:15:58.052157 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:58.052130 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:15:58.089405 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:58.089370 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-vbtx9"] Jul 06 19:15:58.089704 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:58.089692 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e2443d8a-b4df-4de7-a2f6-7ade12827727" containerName="discovery" Jul 06 19:15:58.089758 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:58.089707 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2443d8a-b4df-4de7-a2f6-7ade12827727" containerName="discovery" Jul 06 19:15:58.089800 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:58.089761 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="e2443d8a-b4df-4de7-a2f6-7ade12827727" containerName="discovery" Jul 06 19:15:58.091648 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:58.091626 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-vbtx9" Jul 06 19:15:58.094392 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:58.094372 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-844qm\"" Jul 06 19:15:58.095488 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:58.095473 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 06 19:15:58.095553 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:58.095489 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 06 19:15:58.101038 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:58.101011 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9g6sv\" (UniqueName: \"kubernetes.io/projected/d6d547c3-50f7-4a4a-a0e6-d27341de00d6-kube-api-access-9g6sv\") pod \"kuadrant-operator-catalog-vbtx9\" (UID: \"d6d547c3-50f7-4a4a-a0e6-d27341de00d6\") " pod="kuadrant-system/kuadrant-operator-catalog-vbtx9" Jul 06 19:15:58.103038 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:58.103009 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-vbtx9"] Jul 06 19:15:58.141327 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:58.141298 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:15:58.201590 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:58.201554 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9g6sv\" (UniqueName: \"kubernetes.io/projected/d6d547c3-50f7-4a4a-a0e6-d27341de00d6-kube-api-access-9g6sv\") pod \"kuadrant-operator-catalog-vbtx9\" (UID: \"d6d547c3-50f7-4a4a-a0e6-d27341de00d6\") " pod="kuadrant-system/kuadrant-operator-catalog-vbtx9" Jul 06 19:15:58.210180 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:58.210106 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9g6sv\" (UniqueName: \"kubernetes.io/projected/d6d547c3-50f7-4a4a-a0e6-d27341de00d6-kube-api-access-9g6sv\") pod \"kuadrant-operator-catalog-vbtx9\" (UID: \"d6d547c3-50f7-4a4a-a0e6-d27341de00d6\") " pod="kuadrant-system/kuadrant-operator-catalog-vbtx9" Jul 06 19:15:58.401969 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:58.401927 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-vbtx9" Jul 06 19:15:58.461479 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:58.461403 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-vbtx9"] Jul 06 19:15:58.535310 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:58.535287 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-vbtx9"] Jul 06 19:15:58.537455 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:15:58.537430 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6d547c3_50f7_4a4a_a0e6_d27341de00d6.slice/crio-ec4426fdcdfb8458768468a5561c8968148f0612ebb5c3835efe2d8f2376b905 WatchSource:0}: Error finding container ec4426fdcdfb8458768468a5561c8968148f0612ebb5c3835efe2d8f2376b905: Status 404 returned error can't find the container with id ec4426fdcdfb8458768468a5561c8968148f0612ebb5c3835efe2d8f2376b905 Jul 06 19:15:58.671060 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:58.671020 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-zfdvv"] Jul 06 19:15:58.674250 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:58.674227 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-zfdvv" Jul 06 19:15:58.681633 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:58.681601 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-zfdvv"] Jul 06 19:15:58.705494 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:58.705460 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j8wq7\" (UniqueName: \"kubernetes.io/projected/43e2d40f-c63e-4db9-b0f0-25cf4189f93d-kube-api-access-j8wq7\") pod \"kuadrant-operator-catalog-zfdvv\" (UID: \"43e2d40f-c63e-4db9-b0f0-25cf4189f93d\") " pod="kuadrant-system/kuadrant-operator-catalog-zfdvv" Jul 06 19:15:58.806925 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:58.806885 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j8wq7\" (UniqueName: \"kubernetes.io/projected/43e2d40f-c63e-4db9-b0f0-25cf4189f93d-kube-api-access-j8wq7\") pod \"kuadrant-operator-catalog-zfdvv\" (UID: \"43e2d40f-c63e-4db9-b0f0-25cf4189f93d\") " pod="kuadrant-system/kuadrant-operator-catalog-zfdvv" Jul 06 19:15:58.821324 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:58.821296 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j8wq7\" (UniqueName: \"kubernetes.io/projected/43e2d40f-c63e-4db9-b0f0-25cf4189f93d-kube-api-access-j8wq7\") pod \"kuadrant-operator-catalog-zfdvv\" (UID: \"43e2d40f-c63e-4db9-b0f0-25cf4189f93d\") " pod="kuadrant-system/kuadrant-operator-catalog-zfdvv" Jul 06 19:15:58.891098 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:58.891048 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-vbtx9" event={"ID":"d6d547c3-50f7-4a4a-a0e6-d27341de00d6","Type":"ContainerStarted","Data":"ec4426fdcdfb8458768468a5561c8968148f0612ebb5c3835efe2d8f2376b905"} Jul 06 19:15:58.934684 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:58.934625 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:15:58.983742 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:58.983712 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-zfdvv" Jul 06 19:15:59.115459 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:59.115428 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:15:59.161197 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:59.161167 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-zfdvv"] Jul 06 19:15:59.197144 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:15:59.197061 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43e2d40f_c63e_4db9_b0f0_25cf4189f93d.slice/crio-f51848d83523931def8c2946371dd3686c8f46dd4dd93a666bc16d5055de8c55 WatchSource:0}: Error finding container f51848d83523931def8c2946371dd3686c8f46dd4dd93a666bc16d5055de8c55: Status 404 returned error can't find the container with id f51848d83523931def8c2946371dd3686c8f46dd4dd93a666bc16d5055de8c55 Jul 06 19:15:59.899133 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:15:59.899102 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-zfdvv" event={"ID":"43e2d40f-c63e-4db9-b0f0-25cf4189f93d","Type":"ContainerStarted","Data":"f51848d83523931def8c2946371dd3686c8f46dd4dd93a666bc16d5055de8c55"} Jul 06 19:16:01.910147 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:01.910107 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-vbtx9" event={"ID":"d6d547c3-50f7-4a4a-a0e6-d27341de00d6","Type":"ContainerStarted","Data":"87221652d9c6d1bdd0392bf5c872b193319dc359b09e42d5a9f59f26a86610d8"} Jul 06 19:16:01.910588 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:01.910244 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-vbtx9" podUID="d6d547c3-50f7-4a4a-a0e6-d27341de00d6" containerName="registry-server" containerID="cri-o://87221652d9c6d1bdd0392bf5c872b193319dc359b09e42d5a9f59f26a86610d8" gracePeriod=2 Jul 06 19:16:01.911659 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:01.911627 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-zfdvv" event={"ID":"43e2d40f-c63e-4db9-b0f0-25cf4189f93d","Type":"ContainerStarted","Data":"755e8bcaab97c8e81f7ed0efbb902c82747b060bcbc6aa03e5e0c13d1f32b6d7"} Jul 06 19:16:01.926092 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:01.926026 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-vbtx9" podStartSLOduration=1.572508548 podStartE2EDuration="3.926011404s" totalImagesPullingTime="2.353502856s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:15:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:16:01.924351965 +0000 UTC m=+387.231710914" watchObservedRunningTime="2026-07-06 19:16:01.926011404 +0000 UTC m=+387.233370352" Jul 06 19:16:01.938727 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:01.938680 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-zfdvv" podStartSLOduration=2.248665692 podStartE2EDuration="3.938665187s" totalImagesPullingTime="1.689999495s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:15:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:16:01.937490916 +0000 UTC m=+387.244849863" watchObservedRunningTime="2026-07-06 19:16:01.938665187 +0000 UTC m=+387.246024143" Jul 06 19:16:02.146361 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:02.146338 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-vbtx9" Jul 06 19:16:02.237469 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:02.237392 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9g6sv\" (UniqueName: \"kubernetes.io/projected/d6d547c3-50f7-4a4a-a0e6-d27341de00d6-kube-api-access-9g6sv\") pod \"d6d547c3-50f7-4a4a-a0e6-d27341de00d6\" (UID: \"d6d547c3-50f7-4a4a-a0e6-d27341de00d6\") " Jul 06 19:16:02.239635 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:02.239607 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d6d547c3-50f7-4a4a-a0e6-d27341de00d6-kube-api-access-9g6sv" (OuterVolumeSpecName: "kube-api-access-9g6sv") pod "d6d547c3-50f7-4a4a-a0e6-d27341de00d6" (UID: "d6d547c3-50f7-4a4a-a0e6-d27341de00d6"). InnerVolumeSpecName "kube-api-access-9g6sv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:16:02.338222 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:02.338187 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9g6sv\" (UniqueName: \"kubernetes.io/projected/d6d547c3-50f7-4a4a-a0e6-d27341de00d6-kube-api-access-9g6sv\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:16:02.917464 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:02.917426 2566 generic.go:356] "Generic (PLEG): container finished" podID="d6d547c3-50f7-4a4a-a0e6-d27341de00d6" containerID="87221652d9c6d1bdd0392bf5c872b193319dc359b09e42d5a9f59f26a86610d8" exitCode=0 Jul 06 19:16:02.917919 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:02.917472 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-vbtx9" Jul 06 19:16:02.917919 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:02.917523 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-vbtx9" event={"ID":"d6d547c3-50f7-4a4a-a0e6-d27341de00d6","Type":"ContainerDied","Data":"87221652d9c6d1bdd0392bf5c872b193319dc359b09e42d5a9f59f26a86610d8"} Jul 06 19:16:02.917919 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:02.917583 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-vbtx9" event={"ID":"d6d547c3-50f7-4a4a-a0e6-d27341de00d6","Type":"ContainerDied","Data":"ec4426fdcdfb8458768468a5561c8968148f0612ebb5c3835efe2d8f2376b905"} Jul 06 19:16:02.917919 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:02.917618 2566 scope.go:117] "RemoveContainer" containerID="87221652d9c6d1bdd0392bf5c872b193319dc359b09e42d5a9f59f26a86610d8" Jul 06 19:16:02.926294 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:02.926269 2566 scope.go:117] "RemoveContainer" containerID="87221652d9c6d1bdd0392bf5c872b193319dc359b09e42d5a9f59f26a86610d8" Jul 06 19:16:02.926543 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:16:02.926524 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"87221652d9c6d1bdd0392bf5c872b193319dc359b09e42d5a9f59f26a86610d8\": container with ID starting with 87221652d9c6d1bdd0392bf5c872b193319dc359b09e42d5a9f59f26a86610d8 not found: ID does not exist" containerID="87221652d9c6d1bdd0392bf5c872b193319dc359b09e42d5a9f59f26a86610d8" Jul 06 19:16:02.926596 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:02.926552 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"87221652d9c6d1bdd0392bf5c872b193319dc359b09e42d5a9f59f26a86610d8"} err="failed to get container status \"87221652d9c6d1bdd0392bf5c872b193319dc359b09e42d5a9f59f26a86610d8\": rpc error: code = NotFound desc = could not find container \"87221652d9c6d1bdd0392bf5c872b193319dc359b09e42d5a9f59f26a86610d8\": container with ID starting with 87221652d9c6d1bdd0392bf5c872b193319dc359b09e42d5a9f59f26a86610d8 not found: ID does not exist" Jul 06 19:16:02.944083 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:02.944040 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-vbtx9"] Jul 06 19:16:02.948735 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:02.948711 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-vbtx9"] Jul 06 19:16:03.236732 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:03.236656 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d6d547c3-50f7-4a4a-a0e6-d27341de00d6" path="/var/lib/kubelet/pods/d6d547c3-50f7-4a4a-a0e6-d27341de00d6/volumes" Jul 06 19:16:08.984197 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:08.984164 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-zfdvv" Jul 06 19:16:08.984643 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:08.984220 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-zfdvv" Jul 06 19:16:09.007020 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:09.006995 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-zfdvv" Jul 06 19:16:09.971466 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:09.971441 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-zfdvv" Jul 06 19:16:13.898997 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:13.898965 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732l4dr"] Jul 06 19:16:13.899404 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:13.899298 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d6d547c3-50f7-4a4a-a0e6-d27341de00d6" containerName="registry-server" Jul 06 19:16:13.899404 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:13.899310 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="d6d547c3-50f7-4a4a-a0e6-d27341de00d6" containerName="registry-server" Jul 06 19:16:13.899404 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:13.899378 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="d6d547c3-50f7-4a4a-a0e6-d27341de00d6" containerName="registry-server" Jul 06 19:16:13.902557 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:13.902540 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732l4dr" Jul 06 19:16:13.904697 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:13.904676 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-bdxch\"" Jul 06 19:16:13.909906 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:13.909868 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732l4dr"] Jul 06 19:16:13.938051 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:13.938018 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/101e034c-ad87-4f3c-af45-de74d04a1c21-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732l4dr\" (UID: \"101e034c-ad87-4f3c-af45-de74d04a1c21\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732l4dr" Jul 06 19:16:13.938209 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:13.938060 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w92s5\" (UniqueName: \"kubernetes.io/projected/101e034c-ad87-4f3c-af45-de74d04a1c21-kube-api-access-w92s5\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732l4dr\" (UID: \"101e034c-ad87-4f3c-af45-de74d04a1c21\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732l4dr" Jul 06 19:16:13.938209 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:13.938150 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/101e034c-ad87-4f3c-af45-de74d04a1c21-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732l4dr\" (UID: \"101e034c-ad87-4f3c-af45-de74d04a1c21\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732l4dr" Jul 06 19:16:14.038633 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:14.038591 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w92s5\" (UniqueName: \"kubernetes.io/projected/101e034c-ad87-4f3c-af45-de74d04a1c21-kube-api-access-w92s5\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732l4dr\" (UID: \"101e034c-ad87-4f3c-af45-de74d04a1c21\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732l4dr" Jul 06 19:16:14.038835 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:14.038686 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/101e034c-ad87-4f3c-af45-de74d04a1c21-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732l4dr\" (UID: \"101e034c-ad87-4f3c-af45-de74d04a1c21\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732l4dr" Jul 06 19:16:14.038835 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:14.038728 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/101e034c-ad87-4f3c-af45-de74d04a1c21-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732l4dr\" (UID: \"101e034c-ad87-4f3c-af45-de74d04a1c21\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732l4dr" Jul 06 19:16:14.039113 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:14.039060 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/101e034c-ad87-4f3c-af45-de74d04a1c21-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732l4dr\" (UID: \"101e034c-ad87-4f3c-af45-de74d04a1c21\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732l4dr" Jul 06 19:16:14.039196 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:14.039144 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/101e034c-ad87-4f3c-af45-de74d04a1c21-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732l4dr\" (UID: \"101e034c-ad87-4f3c-af45-de74d04a1c21\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732l4dr" Jul 06 19:16:14.047322 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:14.047298 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w92s5\" (UniqueName: \"kubernetes.io/projected/101e034c-ad87-4f3c-af45-de74d04a1c21-kube-api-access-w92s5\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732l4dr\" (UID: \"101e034c-ad87-4f3c-af45-de74d04a1c21\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732l4dr" Jul 06 19:16:14.212204 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:14.212123 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732l4dr" Jul 06 19:16:14.343037 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:14.343005 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732l4dr"] Jul 06 19:16:14.346328 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:16:14.346305 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod101e034c_ad87_4f3c_af45_de74d04a1c21.slice/crio-b78675f2a9a15910de4bc1f135877e3d273ca588b6b93351f034cf642f5815b2 WatchSource:0}: Error finding container b78675f2a9a15910de4bc1f135877e3d273ca588b6b93351f034cf642f5815b2: Status 404 returned error can't find the container with id b78675f2a9a15910de4bc1f135877e3d273ca588b6b93351f034cf642f5815b2 Jul 06 19:16:14.701808 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:14.701773 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zcf4q"] Jul 06 19:16:14.704223 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:14.704206 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zcf4q" Jul 06 19:16:14.714546 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:14.714520 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zcf4q"] Jul 06 19:16:14.745097 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:14.745055 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t9vh6\" (UniqueName: \"kubernetes.io/projected/0d0869d7-b5a9-4689-93d2-90ed8ad52d52-kube-api-access-t9vh6\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zcf4q\" (UID: \"0d0869d7-b5a9-4689-93d2-90ed8ad52d52\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zcf4q" Jul 06 19:16:14.745214 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:14.745111 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0d0869d7-b5a9-4689-93d2-90ed8ad52d52-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zcf4q\" (UID: \"0d0869d7-b5a9-4689-93d2-90ed8ad52d52\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zcf4q" Jul 06 19:16:14.745214 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:14.745189 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0d0869d7-b5a9-4689-93d2-90ed8ad52d52-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zcf4q\" (UID: \"0d0869d7-b5a9-4689-93d2-90ed8ad52d52\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zcf4q" Jul 06 19:16:14.846372 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:14.846331 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t9vh6\" (UniqueName: \"kubernetes.io/projected/0d0869d7-b5a9-4689-93d2-90ed8ad52d52-kube-api-access-t9vh6\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zcf4q\" (UID: \"0d0869d7-b5a9-4689-93d2-90ed8ad52d52\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zcf4q" Jul 06 19:16:14.846552 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:14.846388 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0d0869d7-b5a9-4689-93d2-90ed8ad52d52-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zcf4q\" (UID: \"0d0869d7-b5a9-4689-93d2-90ed8ad52d52\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zcf4q" Jul 06 19:16:14.846552 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:14.846437 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0d0869d7-b5a9-4689-93d2-90ed8ad52d52-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zcf4q\" (UID: \"0d0869d7-b5a9-4689-93d2-90ed8ad52d52\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zcf4q" Jul 06 19:16:14.846857 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:14.846837 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0d0869d7-b5a9-4689-93d2-90ed8ad52d52-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zcf4q\" (UID: \"0d0869d7-b5a9-4689-93d2-90ed8ad52d52\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zcf4q" Jul 06 19:16:14.846905 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:14.846861 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0d0869d7-b5a9-4689-93d2-90ed8ad52d52-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zcf4q\" (UID: \"0d0869d7-b5a9-4689-93d2-90ed8ad52d52\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zcf4q" Jul 06 19:16:14.854671 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:14.854646 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t9vh6\" (UniqueName: \"kubernetes.io/projected/0d0869d7-b5a9-4689-93d2-90ed8ad52d52-kube-api-access-t9vh6\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zcf4q\" (UID: \"0d0869d7-b5a9-4689-93d2-90ed8ad52d52\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zcf4q" Jul 06 19:16:14.973272 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:14.973168 2566 generic.go:356] "Generic (PLEG): container finished" podID="101e034c-ad87-4f3c-af45-de74d04a1c21" containerID="86fec67a26f1fa905391b4da34a0d7c1cf521e985e1f041b65aaae851da6cbe9" exitCode=0 Jul 06 19:16:14.973272 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:14.973205 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732l4dr" event={"ID":"101e034c-ad87-4f3c-af45-de74d04a1c21","Type":"ContainerDied","Data":"86fec67a26f1fa905391b4da34a0d7c1cf521e985e1f041b65aaae851da6cbe9"} Jul 06 19:16:14.973272 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:14.973250 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732l4dr" event={"ID":"101e034c-ad87-4f3c-af45-de74d04a1c21","Type":"ContainerStarted","Data":"b78675f2a9a15910de4bc1f135877e3d273ca588b6b93351f034cf642f5815b2"} Jul 06 19:16:15.012524 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:15.012494 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zcf4q" Jul 06 19:16:15.103997 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:15.103964 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594zlb4"] Jul 06 19:16:15.107403 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:15.107382 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594zlb4" Jul 06 19:16:15.116327 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:15.116299 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594zlb4"] Jul 06 19:16:15.141630 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:15.141609 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zcf4q"] Jul 06 19:16:15.143858 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:16:15.143832 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d0869d7_b5a9_4689_93d2_90ed8ad52d52.slice/crio-2dd91ef6297f05b048cf68c19a7d14d5014ee4ad6e81a4e8635bebb131f10558 WatchSource:0}: Error finding container 2dd91ef6297f05b048cf68c19a7d14d5014ee4ad6e81a4e8635bebb131f10558: Status 404 returned error can't find the container with id 2dd91ef6297f05b048cf68c19a7d14d5014ee4ad6e81a4e8635bebb131f10558 Jul 06 19:16:15.148692 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:15.148607 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/403eaac6-75e0-459d-a251-a28e7e46c434-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594zlb4\" (UID: \"403eaac6-75e0-459d-a251-a28e7e46c434\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594zlb4" Jul 06 19:16:15.148692 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:15.148651 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sdntg\" (UniqueName: \"kubernetes.io/projected/403eaac6-75e0-459d-a251-a28e7e46c434-kube-api-access-sdntg\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594zlb4\" (UID: \"403eaac6-75e0-459d-a251-a28e7e46c434\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594zlb4" Jul 06 19:16:15.148763 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:15.148721 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/403eaac6-75e0-459d-a251-a28e7e46c434-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594zlb4\" (UID: \"403eaac6-75e0-459d-a251-a28e7e46c434\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594zlb4" Jul 06 19:16:15.249853 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:15.249827 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/403eaac6-75e0-459d-a251-a28e7e46c434-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594zlb4\" (UID: \"403eaac6-75e0-459d-a251-a28e7e46c434\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594zlb4" Jul 06 19:16:15.249940 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:15.249871 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/403eaac6-75e0-459d-a251-a28e7e46c434-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594zlb4\" (UID: \"403eaac6-75e0-459d-a251-a28e7e46c434\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594zlb4" Jul 06 19:16:15.249940 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:15.249910 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sdntg\" (UniqueName: \"kubernetes.io/projected/403eaac6-75e0-459d-a251-a28e7e46c434-kube-api-access-sdntg\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594zlb4\" (UID: \"403eaac6-75e0-459d-a251-a28e7e46c434\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594zlb4" Jul 06 19:16:15.250179 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:15.250161 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/403eaac6-75e0-459d-a251-a28e7e46c434-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594zlb4\" (UID: \"403eaac6-75e0-459d-a251-a28e7e46c434\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594zlb4" Jul 06 19:16:15.250239 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:15.250213 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/403eaac6-75e0-459d-a251-a28e7e46c434-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594zlb4\" (UID: \"403eaac6-75e0-459d-a251-a28e7e46c434\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594zlb4" Jul 06 19:16:15.257787 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:15.257763 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sdntg\" (UniqueName: \"kubernetes.io/projected/403eaac6-75e0-459d-a251-a28e7e46c434-kube-api-access-sdntg\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594zlb4\" (UID: \"403eaac6-75e0-459d-a251-a28e7e46c434\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594zlb4" Jul 06 19:16:15.418597 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:15.418547 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594zlb4" Jul 06 19:16:15.507360 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:15.507308 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mkghr"] Jul 06 19:16:15.510310 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:15.510288 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mkghr" Jul 06 19:16:15.520133 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:15.520102 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mkghr"] Jul 06 19:16:15.552667 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:15.552645 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594zlb4"] Jul 06 19:16:15.552742 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:15.552693 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b8af6bcb-5bcc-4465-85e0-778e48921c4c-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mkghr\" (UID: \"b8af6bcb-5bcc-4465-85e0-778e48921c4c\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mkghr" Jul 06 19:16:15.552856 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:15.552837 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b8af6bcb-5bcc-4465-85e0-778e48921c4c-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mkghr\" (UID: \"b8af6bcb-5bcc-4465-85e0-778e48921c4c\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mkghr" Jul 06 19:16:15.552917 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:15.552873 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fgctv\" (UniqueName: \"kubernetes.io/projected/b8af6bcb-5bcc-4465-85e0-778e48921c4c-kube-api-access-fgctv\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mkghr\" (UID: \"b8af6bcb-5bcc-4465-85e0-778e48921c4c\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mkghr" Jul 06 19:16:15.553787 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:16:15.553767 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod403eaac6_75e0_459d_a251_a28e7e46c434.slice/crio-12194fe367e2e042dacc017ac6d08469fa1c93ec1d05982d953428b907ec6a1c WatchSource:0}: Error finding container 12194fe367e2e042dacc017ac6d08469fa1c93ec1d05982d953428b907ec6a1c: Status 404 returned error can't find the container with id 12194fe367e2e042dacc017ac6d08469fa1c93ec1d05982d953428b907ec6a1c Jul 06 19:16:15.653901 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:15.653868 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b8af6bcb-5bcc-4465-85e0-778e48921c4c-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mkghr\" (UID: \"b8af6bcb-5bcc-4465-85e0-778e48921c4c\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mkghr" Jul 06 19:16:15.654047 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:15.653910 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fgctv\" (UniqueName: \"kubernetes.io/projected/b8af6bcb-5bcc-4465-85e0-778e48921c4c-kube-api-access-fgctv\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mkghr\" (UID: \"b8af6bcb-5bcc-4465-85e0-778e48921c4c\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mkghr" Jul 06 19:16:15.654047 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:15.653962 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b8af6bcb-5bcc-4465-85e0-778e48921c4c-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mkghr\" (UID: \"b8af6bcb-5bcc-4465-85e0-778e48921c4c\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mkghr" Jul 06 19:16:15.654298 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:15.654273 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b8af6bcb-5bcc-4465-85e0-778e48921c4c-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mkghr\" (UID: \"b8af6bcb-5bcc-4465-85e0-778e48921c4c\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mkghr" Jul 06 19:16:15.654342 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:15.654293 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b8af6bcb-5bcc-4465-85e0-778e48921c4c-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mkghr\" (UID: \"b8af6bcb-5bcc-4465-85e0-778e48921c4c\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mkghr" Jul 06 19:16:15.662086 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:15.662043 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fgctv\" (UniqueName: \"kubernetes.io/projected/b8af6bcb-5bcc-4465-85e0-778e48921c4c-kube-api-access-fgctv\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mkghr\" (UID: \"b8af6bcb-5bcc-4465-85e0-778e48921c4c\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mkghr" Jul 06 19:16:15.820848 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:15.820806 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mkghr" Jul 06 19:16:15.959000 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:15.958960 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mkghr"] Jul 06 19:16:15.960879 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:16:15.960847 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8af6bcb_5bcc_4465_85e0_778e48921c4c.slice/crio-a80aafaea973d4f9f4dfa592326caabc5f2693adfe1e6e50ed73755c75494d1f WatchSource:0}: Error finding container a80aafaea973d4f9f4dfa592326caabc5f2693adfe1e6e50ed73755c75494d1f: Status 404 returned error can't find the container with id a80aafaea973d4f9f4dfa592326caabc5f2693adfe1e6e50ed73755c75494d1f Jul 06 19:16:15.979948 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:15.979859 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mkghr" event={"ID":"b8af6bcb-5bcc-4465-85e0-778e48921c4c","Type":"ContainerStarted","Data":"a80aafaea973d4f9f4dfa592326caabc5f2693adfe1e6e50ed73755c75494d1f"} Jul 06 19:16:15.982152 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:15.982086 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732l4dr" event={"ID":"101e034c-ad87-4f3c-af45-de74d04a1c21","Type":"ContainerStarted","Data":"e99b42726ebd2be1db72cbb05534a2d9026520cab0dc006a8cf81183f9418ddf"} Jul 06 19:16:15.983658 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:15.983633 2566 generic.go:356] "Generic (PLEG): container finished" podID="403eaac6-75e0-459d-a251-a28e7e46c434" containerID="8030386bfc04b656f9ef72ed1b5077809257ced5c274cb1119566ecac5fdc583" exitCode=0 Jul 06 19:16:15.983741 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:15.983717 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594zlb4" event={"ID":"403eaac6-75e0-459d-a251-a28e7e46c434","Type":"ContainerDied","Data":"8030386bfc04b656f9ef72ed1b5077809257ced5c274cb1119566ecac5fdc583"} Jul 06 19:16:15.983815 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:15.983762 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594zlb4" event={"ID":"403eaac6-75e0-459d-a251-a28e7e46c434","Type":"ContainerStarted","Data":"12194fe367e2e042dacc017ac6d08469fa1c93ec1d05982d953428b907ec6a1c"} Jul 06 19:16:15.985378 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:15.985347 2566 generic.go:356] "Generic (PLEG): container finished" podID="0d0869d7-b5a9-4689-93d2-90ed8ad52d52" containerID="856be570faa75b66348937f2bc5192c6f9a49ec64535b50805a151944375865a" exitCode=0 Jul 06 19:16:15.985472 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:15.985382 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zcf4q" event={"ID":"0d0869d7-b5a9-4689-93d2-90ed8ad52d52","Type":"ContainerDied","Data":"856be570faa75b66348937f2bc5192c6f9a49ec64535b50805a151944375865a"} Jul 06 19:16:15.985472 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:15.985415 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zcf4q" event={"ID":"0d0869d7-b5a9-4689-93d2-90ed8ad52d52","Type":"ContainerStarted","Data":"2dd91ef6297f05b048cf68c19a7d14d5014ee4ad6e81a4e8635bebb131f10558"} Jul 06 19:16:16.991605 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:16.991576 2566 generic.go:356] "Generic (PLEG): container finished" podID="b8af6bcb-5bcc-4465-85e0-778e48921c4c" containerID="df5ccf24db258ee8561ad777c2c033fa8efcbf4c827c5b01cd236ce65993e886" exitCode=0 Jul 06 19:16:16.991976 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:16.991656 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mkghr" event={"ID":"b8af6bcb-5bcc-4465-85e0-778e48921c4c","Type":"ContainerDied","Data":"df5ccf24db258ee8561ad777c2c033fa8efcbf4c827c5b01cd236ce65993e886"} Jul 06 19:16:16.993618 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:16.993593 2566 generic.go:356] "Generic (PLEG): container finished" podID="101e034c-ad87-4f3c-af45-de74d04a1c21" containerID="e99b42726ebd2be1db72cbb05534a2d9026520cab0dc006a8cf81183f9418ddf" exitCode=0 Jul 06 19:16:16.993717 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:16.993672 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732l4dr" event={"ID":"101e034c-ad87-4f3c-af45-de74d04a1c21","Type":"ContainerDied","Data":"e99b42726ebd2be1db72cbb05534a2d9026520cab0dc006a8cf81183f9418ddf"} Jul 06 19:16:16.995630 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:16.995607 2566 generic.go:356] "Generic (PLEG): container finished" podID="403eaac6-75e0-459d-a251-a28e7e46c434" containerID="44bc54ac1bfed1608216e13fcbd77ff75e0997886dd1da1f52608111e1bcf1e4" exitCode=0 Jul 06 19:16:16.995710 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:16.995680 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594zlb4" event={"ID":"403eaac6-75e0-459d-a251-a28e7e46c434","Type":"ContainerDied","Data":"44bc54ac1bfed1608216e13fcbd77ff75e0997886dd1da1f52608111e1bcf1e4"} Jul 06 19:16:16.997581 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:16.997560 2566 generic.go:356] "Generic (PLEG): container finished" podID="0d0869d7-b5a9-4689-93d2-90ed8ad52d52" containerID="083452621d737e674dc73a8711eaa6b4589518c36cb65dd4d2cdcdc65109210e" exitCode=0 Jul 06 19:16:16.997649 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:16.997611 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zcf4q" event={"ID":"0d0869d7-b5a9-4689-93d2-90ed8ad52d52","Type":"ContainerDied","Data":"083452621d737e674dc73a8711eaa6b4589518c36cb65dd4d2cdcdc65109210e"} Jul 06 19:16:18.003919 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:18.003835 2566 generic.go:356] "Generic (PLEG): container finished" podID="101e034c-ad87-4f3c-af45-de74d04a1c21" containerID="28e91cd0b698cb3eb196c1521b4b18a2fd804f064a7088f8eee8a71df2db9d8c" exitCode=0 Jul 06 19:16:18.004354 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:18.003930 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732l4dr" event={"ID":"101e034c-ad87-4f3c-af45-de74d04a1c21","Type":"ContainerDied","Data":"28e91cd0b698cb3eb196c1521b4b18a2fd804f064a7088f8eee8a71df2db9d8c"} Jul 06 19:16:18.005982 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:18.005952 2566 generic.go:356] "Generic (PLEG): container finished" podID="403eaac6-75e0-459d-a251-a28e7e46c434" containerID="419909e5033046278c4c22d727848a7ae785adbf10868416eb7cb05b2dade97c" exitCode=0 Jul 06 19:16:18.006122 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:18.006036 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594zlb4" event={"ID":"403eaac6-75e0-459d-a251-a28e7e46c434","Type":"ContainerDied","Data":"419909e5033046278c4c22d727848a7ae785adbf10868416eb7cb05b2dade97c"} Jul 06 19:16:18.008005 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:18.007987 2566 generic.go:356] "Generic (PLEG): container finished" podID="0d0869d7-b5a9-4689-93d2-90ed8ad52d52" containerID="350c10f2ed49c7486587f45429b2ac8c9e0c5837a2f6d4599f1b9e7614af63ab" exitCode=0 Jul 06 19:16:18.008108 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:18.008059 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zcf4q" event={"ID":"0d0869d7-b5a9-4689-93d2-90ed8ad52d52","Type":"ContainerDied","Data":"350c10f2ed49c7486587f45429b2ac8c9e0c5837a2f6d4599f1b9e7614af63ab"} Jul 06 19:16:18.009737 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:18.009718 2566 generic.go:356] "Generic (PLEG): container finished" podID="b8af6bcb-5bcc-4465-85e0-778e48921c4c" containerID="7ea10f26ef948965c755f70375e9a4f3f37c08fdeed4edc6915821173f349038" exitCode=0 Jul 06 19:16:18.009813 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:18.009797 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mkghr" event={"ID":"b8af6bcb-5bcc-4465-85e0-778e48921c4c","Type":"ContainerDied","Data":"7ea10f26ef948965c755f70375e9a4f3f37c08fdeed4edc6915821173f349038"} Jul 06 19:16:19.018590 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:19.018559 2566 generic.go:356] "Generic (PLEG): container finished" podID="b8af6bcb-5bcc-4465-85e0-778e48921c4c" containerID="34a75af1dfc791935ec384e4845a8278b26684c55859355810b4b4be27da591d" exitCode=0 Jul 06 19:16:19.019000 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:19.018643 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mkghr" event={"ID":"b8af6bcb-5bcc-4465-85e0-778e48921c4c","Type":"ContainerDied","Data":"34a75af1dfc791935ec384e4845a8278b26684c55859355810b4b4be27da591d"} Jul 06 19:16:19.164251 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:19.164225 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zcf4q" Jul 06 19:16:19.207821 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:19.207799 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732l4dr" Jul 06 19:16:19.210844 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:19.210822 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594zlb4" Jul 06 19:16:19.296741 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:19.296713 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0d0869d7-b5a9-4689-93d2-90ed8ad52d52-util\") pod \"0d0869d7-b5a9-4689-93d2-90ed8ad52d52\" (UID: \"0d0869d7-b5a9-4689-93d2-90ed8ad52d52\") " Jul 06 19:16:19.296929 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:19.296780 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sdntg\" (UniqueName: \"kubernetes.io/projected/403eaac6-75e0-459d-a251-a28e7e46c434-kube-api-access-sdntg\") pod \"403eaac6-75e0-459d-a251-a28e7e46c434\" (UID: \"403eaac6-75e0-459d-a251-a28e7e46c434\") " Jul 06 19:16:19.296929 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:19.296801 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t9vh6\" (UniqueName: \"kubernetes.io/projected/0d0869d7-b5a9-4689-93d2-90ed8ad52d52-kube-api-access-t9vh6\") pod \"0d0869d7-b5a9-4689-93d2-90ed8ad52d52\" (UID: \"0d0869d7-b5a9-4689-93d2-90ed8ad52d52\") " Jul 06 19:16:19.296929 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:19.296895 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/403eaac6-75e0-459d-a251-a28e7e46c434-util\") pod \"403eaac6-75e0-459d-a251-a28e7e46c434\" (UID: \"403eaac6-75e0-459d-a251-a28e7e46c434\") " Jul 06 19:16:19.297116 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:19.296943 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/101e034c-ad87-4f3c-af45-de74d04a1c21-util\") pod \"101e034c-ad87-4f3c-af45-de74d04a1c21\" (UID: \"101e034c-ad87-4f3c-af45-de74d04a1c21\") " Jul 06 19:16:19.297116 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:19.296984 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w92s5\" (UniqueName: \"kubernetes.io/projected/101e034c-ad87-4f3c-af45-de74d04a1c21-kube-api-access-w92s5\") pod \"101e034c-ad87-4f3c-af45-de74d04a1c21\" (UID: \"101e034c-ad87-4f3c-af45-de74d04a1c21\") " Jul 06 19:16:19.297116 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:19.297018 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/403eaac6-75e0-459d-a251-a28e7e46c434-bundle\") pod \"403eaac6-75e0-459d-a251-a28e7e46c434\" (UID: \"403eaac6-75e0-459d-a251-a28e7e46c434\") " Jul 06 19:16:19.297258 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:19.297121 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0d0869d7-b5a9-4689-93d2-90ed8ad52d52-bundle\") pod \"0d0869d7-b5a9-4689-93d2-90ed8ad52d52\" (UID: \"0d0869d7-b5a9-4689-93d2-90ed8ad52d52\") " Jul 06 19:16:19.297258 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:19.297152 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/101e034c-ad87-4f3c-af45-de74d04a1c21-bundle\") pod \"101e034c-ad87-4f3c-af45-de74d04a1c21\" (UID: \"101e034c-ad87-4f3c-af45-de74d04a1c21\") " Jul 06 19:16:19.297837 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:19.297524 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/403eaac6-75e0-459d-a251-a28e7e46c434-bundle" (OuterVolumeSpecName: "bundle") pod "403eaac6-75e0-459d-a251-a28e7e46c434" (UID: "403eaac6-75e0-459d-a251-a28e7e46c434"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:16:19.298039 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:19.298018 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d0869d7-b5a9-4689-93d2-90ed8ad52d52-bundle" (OuterVolumeSpecName: "bundle") pod "0d0869d7-b5a9-4689-93d2-90ed8ad52d52" (UID: "0d0869d7-b5a9-4689-93d2-90ed8ad52d52"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:16:19.298203 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:19.298178 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/101e034c-ad87-4f3c-af45-de74d04a1c21-bundle" (OuterVolumeSpecName: "bundle") pod "101e034c-ad87-4f3c-af45-de74d04a1c21" (UID: "101e034c-ad87-4f3c-af45-de74d04a1c21"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:16:19.299615 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:19.299593 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/403eaac6-75e0-459d-a251-a28e7e46c434-kube-api-access-sdntg" (OuterVolumeSpecName: "kube-api-access-sdntg") pod "403eaac6-75e0-459d-a251-a28e7e46c434" (UID: "403eaac6-75e0-459d-a251-a28e7e46c434"). InnerVolumeSpecName "kube-api-access-sdntg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:16:19.299988 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:19.299948 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/101e034c-ad87-4f3c-af45-de74d04a1c21-kube-api-access-w92s5" (OuterVolumeSpecName: "kube-api-access-w92s5") pod "101e034c-ad87-4f3c-af45-de74d04a1c21" (UID: "101e034c-ad87-4f3c-af45-de74d04a1c21"). InnerVolumeSpecName "kube-api-access-w92s5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:16:19.300076 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:19.300004 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0d0869d7-b5a9-4689-93d2-90ed8ad52d52-kube-api-access-t9vh6" (OuterVolumeSpecName: "kube-api-access-t9vh6") pod "0d0869d7-b5a9-4689-93d2-90ed8ad52d52" (UID: "0d0869d7-b5a9-4689-93d2-90ed8ad52d52"). InnerVolumeSpecName "kube-api-access-t9vh6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:16:19.303980 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:19.303949 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/403eaac6-75e0-459d-a251-a28e7e46c434-util" (OuterVolumeSpecName: "util") pod "403eaac6-75e0-459d-a251-a28e7e46c434" (UID: "403eaac6-75e0-459d-a251-a28e7e46c434"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:16:19.305589 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:19.305564 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/101e034c-ad87-4f3c-af45-de74d04a1c21-util" (OuterVolumeSpecName: "util") pod "101e034c-ad87-4f3c-af45-de74d04a1c21" (UID: "101e034c-ad87-4f3c-af45-de74d04a1c21"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:16:19.306421 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:19.306404 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d0869d7-b5a9-4689-93d2-90ed8ad52d52-util" (OuterVolumeSpecName: "util") pod "0d0869d7-b5a9-4689-93d2-90ed8ad52d52" (UID: "0d0869d7-b5a9-4689-93d2-90ed8ad52d52"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:16:19.397839 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:19.397804 2566 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0d0869d7-b5a9-4689-93d2-90ed8ad52d52-bundle\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:16:19.397839 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:19.397834 2566 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/101e034c-ad87-4f3c-af45-de74d04a1c21-bundle\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:16:19.397839 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:19.397842 2566 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0d0869d7-b5a9-4689-93d2-90ed8ad52d52-util\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:16:19.398108 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:19.397851 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sdntg\" (UniqueName: \"kubernetes.io/projected/403eaac6-75e0-459d-a251-a28e7e46c434-kube-api-access-sdntg\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:16:19.398108 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:19.397861 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t9vh6\" (UniqueName: \"kubernetes.io/projected/0d0869d7-b5a9-4689-93d2-90ed8ad52d52-kube-api-access-t9vh6\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:16:19.398108 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:19.397870 2566 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/403eaac6-75e0-459d-a251-a28e7e46c434-util\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:16:19.398108 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:19.397877 2566 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/101e034c-ad87-4f3c-af45-de74d04a1c21-util\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:16:19.398108 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:19.397886 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w92s5\" (UniqueName: \"kubernetes.io/projected/101e034c-ad87-4f3c-af45-de74d04a1c21-kube-api-access-w92s5\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:16:19.398108 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:19.397893 2566 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/403eaac6-75e0-459d-a251-a28e7e46c434-bundle\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:16:20.025532 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:20.025497 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732l4dr" event={"ID":"101e034c-ad87-4f3c-af45-de74d04a1c21","Type":"ContainerDied","Data":"b78675f2a9a15910de4bc1f135877e3d273ca588b6b93351f034cf642f5815b2"} Jul 06 19:16:20.025928 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:20.025548 2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b78675f2a9a15910de4bc1f135877e3d273ca588b6b93351f034cf642f5815b2" Jul 06 19:16:20.025928 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:20.025513 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732l4dr" Jul 06 19:16:20.027422 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:20.027402 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594zlb4" Jul 06 19:16:20.027541 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:20.027420 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594zlb4" event={"ID":"403eaac6-75e0-459d-a251-a28e7e46c434","Type":"ContainerDied","Data":"12194fe367e2e042dacc017ac6d08469fa1c93ec1d05982d953428b907ec6a1c"} Jul 06 19:16:20.027541 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:20.027464 2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="12194fe367e2e042dacc017ac6d08469fa1c93ec1d05982d953428b907ec6a1c" Jul 06 19:16:20.029340 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:20.029322 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zcf4q" Jul 06 19:16:20.029465 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:20.029357 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zcf4q" event={"ID":"0d0869d7-b5a9-4689-93d2-90ed8ad52d52","Type":"ContainerDied","Data":"2dd91ef6297f05b048cf68c19a7d14d5014ee4ad6e81a4e8635bebb131f10558"} Jul 06 19:16:20.029465 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:20.029393 2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2dd91ef6297f05b048cf68c19a7d14d5014ee4ad6e81a4e8635bebb131f10558" Jul 06 19:16:20.157477 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:20.157456 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mkghr" Jul 06 19:16:20.307229 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:20.307204 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b8af6bcb-5bcc-4465-85e0-778e48921c4c-util\") pod \"b8af6bcb-5bcc-4465-85e0-778e48921c4c\" (UID: \"b8af6bcb-5bcc-4465-85e0-778e48921c4c\") " Jul 06 19:16:20.307413 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:20.307266 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b8af6bcb-5bcc-4465-85e0-778e48921c4c-bundle\") pod \"b8af6bcb-5bcc-4465-85e0-778e48921c4c\" (UID: \"b8af6bcb-5bcc-4465-85e0-778e48921c4c\") " Jul 06 19:16:20.307413 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:20.307342 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fgctv\" (UniqueName: \"kubernetes.io/projected/b8af6bcb-5bcc-4465-85e0-778e48921c4c-kube-api-access-fgctv\") pod \"b8af6bcb-5bcc-4465-85e0-778e48921c4c\" (UID: \"b8af6bcb-5bcc-4465-85e0-778e48921c4c\") " Jul 06 19:16:20.307833 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:20.307801 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b8af6bcb-5bcc-4465-85e0-778e48921c4c-bundle" (OuterVolumeSpecName: "bundle") pod "b8af6bcb-5bcc-4465-85e0-778e48921c4c" (UID: "b8af6bcb-5bcc-4465-85e0-778e48921c4c"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:16:20.309519 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:20.309495 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b8af6bcb-5bcc-4465-85e0-778e48921c4c-kube-api-access-fgctv" (OuterVolumeSpecName: "kube-api-access-fgctv") pod "b8af6bcb-5bcc-4465-85e0-778e48921c4c" (UID: "b8af6bcb-5bcc-4465-85e0-778e48921c4c"). InnerVolumeSpecName "kube-api-access-fgctv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:16:20.315274 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:20.315249 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b8af6bcb-5bcc-4465-85e0-778e48921c4c-util" (OuterVolumeSpecName: "util") pod "b8af6bcb-5bcc-4465-85e0-778e48921c4c" (UID: "b8af6bcb-5bcc-4465-85e0-778e48921c4c"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:16:20.408921 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:20.408879 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fgctv\" (UniqueName: \"kubernetes.io/projected/b8af6bcb-5bcc-4465-85e0-778e48921c4c-kube-api-access-fgctv\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:16:20.408921 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:20.408918 2566 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b8af6bcb-5bcc-4465-85e0-778e48921c4c-util\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:16:20.408921 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:20.408928 2566 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b8af6bcb-5bcc-4465-85e0-778e48921c4c-bundle\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:16:21.036058 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:21.036025 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mkghr" event={"ID":"b8af6bcb-5bcc-4465-85e0-778e48921c4c","Type":"ContainerDied","Data":"a80aafaea973d4f9f4dfa592326caabc5f2693adfe1e6e50ed73755c75494d1f"} Jul 06 19:16:21.036437 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:21.036087 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mkghr" Jul 06 19:16:21.036437 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:21.036090 2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a80aafaea973d4f9f4dfa592326caabc5f2693adfe1e6e50ed73755c75494d1f" Jul 06 19:16:27.102282 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:27.102247 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:16:30.940660 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:30.940626 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:16:32.047341 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:32.047304 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mkk8r"] Jul 06 19:16:32.047727 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:32.047609 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0d0869d7-b5a9-4689-93d2-90ed8ad52d52" containerName="pull" Jul 06 19:16:32.047727 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:32.047619 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d0869d7-b5a9-4689-93d2-90ed8ad52d52" containerName="pull" Jul 06 19:16:32.047727 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:32.047630 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="101e034c-ad87-4f3c-af45-de74d04a1c21" containerName="pull" Jul 06 19:16:32.047727 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:32.047635 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="101e034c-ad87-4f3c-af45-de74d04a1c21" containerName="pull" Jul 06 19:16:32.047727 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:32.047642 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="403eaac6-75e0-459d-a251-a28e7e46c434" containerName="util" Jul 06 19:16:32.047727 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:32.047646 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="403eaac6-75e0-459d-a251-a28e7e46c434" containerName="util" Jul 06 19:16:32.047727 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:32.047651 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b8af6bcb-5bcc-4465-85e0-778e48921c4c" containerName="util" Jul 06 19:16:32.047727 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:32.047656 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="b8af6bcb-5bcc-4465-85e0-778e48921c4c" containerName="util" Jul 06 19:16:32.047727 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:32.047663 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="101e034c-ad87-4f3c-af45-de74d04a1c21" containerName="extract" Jul 06 19:16:32.047727 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:32.047669 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="101e034c-ad87-4f3c-af45-de74d04a1c21" containerName="extract" Jul 06 19:16:32.047727 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:32.047674 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b8af6bcb-5bcc-4465-85e0-778e48921c4c" containerName="extract" Jul 06 19:16:32.047727 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:32.047679 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="b8af6bcb-5bcc-4465-85e0-778e48921c4c" containerName="extract" Jul 06 19:16:32.047727 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:32.047688 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="403eaac6-75e0-459d-a251-a28e7e46c434" containerName="extract" Jul 06 19:16:32.047727 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:32.047692 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="403eaac6-75e0-459d-a251-a28e7e46c434" containerName="extract" Jul 06 19:16:32.047727 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:32.047701 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="101e034c-ad87-4f3c-af45-de74d04a1c21" containerName="util" Jul 06 19:16:32.047727 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:32.047707 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="101e034c-ad87-4f3c-af45-de74d04a1c21" containerName="util" Jul 06 19:16:32.047727 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:32.047716 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0d0869d7-b5a9-4689-93d2-90ed8ad52d52" containerName="extract" Jul 06 19:16:32.047727 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:32.047720 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d0869d7-b5a9-4689-93d2-90ed8ad52d52" containerName="extract" Jul 06 19:16:32.047727 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:32.047725 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b8af6bcb-5bcc-4465-85e0-778e48921c4c" containerName="pull" Jul 06 19:16:32.047727 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:32.047730 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="b8af6bcb-5bcc-4465-85e0-778e48921c4c" containerName="pull" Jul 06 19:16:32.047727 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:32.047736 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0d0869d7-b5a9-4689-93d2-90ed8ad52d52" containerName="util" Jul 06 19:16:32.047727 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:32.047740 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d0869d7-b5a9-4689-93d2-90ed8ad52d52" containerName="util" Jul 06 19:16:32.047727 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:32.047746 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="403eaac6-75e0-459d-a251-a28e7e46c434" containerName="pull" Jul 06 19:16:32.048493 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:32.047750 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="403eaac6-75e0-459d-a251-a28e7e46c434" containerName="pull" Jul 06 19:16:32.048493 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:32.047797 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="0d0869d7-b5a9-4689-93d2-90ed8ad52d52" containerName="extract" Jul 06 19:16:32.048493 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:32.047805 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="403eaac6-75e0-459d-a251-a28e7e46c434" containerName="extract" Jul 06 19:16:32.048493 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:32.047810 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="101e034c-ad87-4f3c-af45-de74d04a1c21" containerName="extract" Jul 06 19:16:32.048493 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:32.047817 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="b8af6bcb-5bcc-4465-85e0-778e48921c4c" containerName="extract" Jul 06 19:16:32.050681 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:32.050665 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mkk8r" Jul 06 19:16:32.053005 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:32.052988 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-4n949\"" Jul 06 19:16:32.065362 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:32.065340 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mkk8r"] Jul 06 19:16:32.218127 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:32.218091 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4f53da6e-5c46-44b5-b784-87ba7e3e1fff-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-mkk8r\" (UID: \"4f53da6e-5c46-44b5-b784-87ba7e3e1fff\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mkk8r" Jul 06 19:16:32.218314 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:32.218157 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bbzf9\" (UniqueName: \"kubernetes.io/projected/4f53da6e-5c46-44b5-b784-87ba7e3e1fff-kube-api-access-bbzf9\") pod \"kuadrant-operator-controller-manager-8684c9ff49-mkk8r\" (UID: \"4f53da6e-5c46-44b5-b784-87ba7e3e1fff\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mkk8r" Jul 06 19:16:32.319018 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:32.318933 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bbzf9\" (UniqueName: \"kubernetes.io/projected/4f53da6e-5c46-44b5-b784-87ba7e3e1fff-kube-api-access-bbzf9\") pod \"kuadrant-operator-controller-manager-8684c9ff49-mkk8r\" (UID: \"4f53da6e-5c46-44b5-b784-87ba7e3e1fff\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mkk8r" Jul 06 19:16:32.319018 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:32.318985 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4f53da6e-5c46-44b5-b784-87ba7e3e1fff-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-mkk8r\" (UID: \"4f53da6e-5c46-44b5-b784-87ba7e3e1fff\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mkk8r" Jul 06 19:16:32.319420 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:32.319401 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4f53da6e-5c46-44b5-b784-87ba7e3e1fff-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-mkk8r\" (UID: \"4f53da6e-5c46-44b5-b784-87ba7e3e1fff\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mkk8r" Jul 06 19:16:32.328041 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:32.328014 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bbzf9\" (UniqueName: \"kubernetes.io/projected/4f53da6e-5c46-44b5-b784-87ba7e3e1fff-kube-api-access-bbzf9\") pod \"kuadrant-operator-controller-manager-8684c9ff49-mkk8r\" (UID: \"4f53da6e-5c46-44b5-b784-87ba7e3e1fff\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mkk8r" Jul 06 19:16:32.360632 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:32.360599 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mkk8r" Jul 06 19:16:32.499188 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:32.499165 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mkk8r"] Jul 06 19:16:32.500773 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:16:32.500744 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f53da6e_5c46_44b5_b784_87ba7e3e1fff.slice/crio-25cd4631d8aaee404ba1ab750e26a9499f1aa073ca3a21972021897403c62354 WatchSource:0}: Error finding container 25cd4631d8aaee404ba1ab750e26a9499f1aa073ca3a21972021897403c62354: Status 404 returned error can't find the container with id 25cd4631d8aaee404ba1ab750e26a9499f1aa073ca3a21972021897403c62354 Jul 06 19:16:32.719556 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:32.719464 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:16:33.096310 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:33.096273 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mkk8r" event={"ID":"4f53da6e-5c46-44b5-b784-87ba7e3e1fff","Type":"ContainerStarted","Data":"25cd4631d8aaee404ba1ab750e26a9499f1aa073ca3a21972021897403c62354"} Jul 06 19:16:34.020245 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:34.020207 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-5j2ll"] Jul 06 19:16:34.022450 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:34.022435 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-5j2ll" Jul 06 19:16:34.025156 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:34.025136 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 06 19:16:34.025699 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:34.025683 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-pfggs\"" Jul 06 19:16:34.036691 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:34.036668 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-5j2ll"] Jul 06 19:16:34.134823 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:34.134787 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7dc78\" (UniqueName: \"kubernetes.io/projected/f2e760ff-630e-4095-ae84-8377ff92111c-kube-api-access-7dc78\") pod \"dns-operator-controller-manager-864dd78c4c-5j2ll\" (UID: \"f2e760ff-630e-4095-ae84-8377ff92111c\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-5j2ll" Jul 06 19:16:34.235954 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:34.235921 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7dc78\" (UniqueName: \"kubernetes.io/projected/f2e760ff-630e-4095-ae84-8377ff92111c-kube-api-access-7dc78\") pod \"dns-operator-controller-manager-864dd78c4c-5j2ll\" (UID: \"f2e760ff-630e-4095-ae84-8377ff92111c\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-5j2ll" Jul 06 19:16:34.246371 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:34.246339 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7dc78\" (UniqueName: \"kubernetes.io/projected/f2e760ff-630e-4095-ae84-8377ff92111c-kube-api-access-7dc78\") pod \"dns-operator-controller-manager-864dd78c4c-5j2ll\" (UID: \"f2e760ff-630e-4095-ae84-8377ff92111c\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-5j2ll" Jul 06 19:16:34.306673 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:34.306637 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:16:34.333323 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:34.333292 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-5j2ll" Jul 06 19:16:34.471700 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:34.471674 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-5j2ll"] Jul 06 19:16:34.473034 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:16:34.473002 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2e760ff_630e_4095_ae84_8377ff92111c.slice/crio-1c92ad61a00b2ac971166edf46060ee467f5eb314ce3630c0294ca4ce1283de2 WatchSource:0}: Error finding container 1c92ad61a00b2ac971166edf46060ee467f5eb314ce3630c0294ca4ce1283de2: Status 404 returned error can't find the container with id 1c92ad61a00b2ac971166edf46060ee467f5eb314ce3630c0294ca4ce1283de2 Jul 06 19:16:34.586927 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:34.586835 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:16:34.710443 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:34.710412 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:16:35.108172 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:35.108139 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-5j2ll" event={"ID":"f2e760ff-630e-4095-ae84-8377ff92111c","Type":"ContainerStarted","Data":"1c92ad61a00b2ac971166edf46060ee467f5eb314ce3630c0294ca4ce1283de2"} Jul 06 19:16:36.290961 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:36.290938 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 06 19:16:37.121542 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:37.121508 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-5j2ll" event={"ID":"f2e760ff-630e-4095-ae84-8377ff92111c","Type":"ContainerStarted","Data":"e6d0d5d0df339e4020fc0f76a30c873efd919b4427ad7fd9a73262f2b1db8fb7"} Jul 06 19:16:37.121682 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:37.121645 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-5j2ll" Jul 06 19:16:37.137769 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:37.137717 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-5j2ll" podStartSLOduration=2.324275572 podStartE2EDuration="4.137702232s" totalImagesPullingTime="1.81342666s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:16:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:16:37.137113283 +0000 UTC m=+422.444472225" watchObservedRunningTime="2026-07-06 19:16:37.137702232 +0000 UTC m=+422.445061179" Jul 06 19:16:37.309998 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:37.309960 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-rddw8"] Jul 06 19:16:37.313387 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:37.313368 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-rddw8" Jul 06 19:16:37.315745 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:37.315724 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-cmgsp\"" Jul 06 19:16:37.322768 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:37.322747 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-rddw8"] Jul 06 19:16:37.361165 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:37.361134 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cwq7f\" (UniqueName: \"kubernetes.io/projected/bf9d0112-eb11-4996-8fca-e6a781f22e14-kube-api-access-cwq7f\") pod \"authorino-operator-75cc8d5cb7-rddw8\" (UID: \"bf9d0112-eb11-4996-8fca-e6a781f22e14\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-rddw8" Jul 06 19:16:37.462007 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:37.461934 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cwq7f\" (UniqueName: \"kubernetes.io/projected/bf9d0112-eb11-4996-8fca-e6a781f22e14-kube-api-access-cwq7f\") pod \"authorino-operator-75cc8d5cb7-rddw8\" (UID: \"bf9d0112-eb11-4996-8fca-e6a781f22e14\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-rddw8" Jul 06 19:16:37.475429 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:37.475408 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cwq7f\" (UniqueName: \"kubernetes.io/projected/bf9d0112-eb11-4996-8fca-e6a781f22e14-kube-api-access-cwq7f\") pod \"authorino-operator-75cc8d5cb7-rddw8\" (UID: \"bf9d0112-eb11-4996-8fca-e6a781f22e14\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-rddw8" Jul 06 19:16:37.623373 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:37.623345 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-rddw8" Jul 06 19:16:37.765684 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:37.765661 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-rddw8"] Jul 06 19:16:37.767465 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:16:37.767431 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf9d0112_eb11_4996_8fca_e6a781f22e14.slice/crio-42db4cf0d2536b624a846239e437d0ec3a6c7c7df0b664b18e90bb98d1efdd0f WatchSource:0}: Error finding container 42db4cf0d2536b624a846239e437d0ec3a6c7c7df0b664b18e90bb98d1efdd0f: Status 404 returned error can't find the container with id 42db4cf0d2536b624a846239e437d0ec3a6c7c7df0b664b18e90bb98d1efdd0f Jul 06 19:16:37.918185 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:37.918150 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:16:38.128258 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:38.128216 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-rddw8" event={"ID":"bf9d0112-eb11-4996-8fca-e6a781f22e14","Type":"ContainerStarted","Data":"42db4cf0d2536b624a846239e437d0ec3a6c7c7df0b664b18e90bb98d1efdd0f"} Jul 06 19:16:38.163493 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:38.163461 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:16:38.270805 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:38.270775 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:16:40.054676 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:40.054636 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cr4hd"] Jul 06 19:16:40.058492 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:40.058471 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cr4hd" Jul 06 19:16:40.060902 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:40.060883 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-6645b\"" Jul 06 19:16:40.065255 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:40.065229 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cr4hd"] Jul 06 19:16:40.084144 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:40.084116 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bh8sd\" (UniqueName: \"kubernetes.io/projected/ab85183c-b2e1-4682-859e-a9c2213e9c2a-kube-api-access-bh8sd\") pod \"limitador-operator-controller-manager-859d7b78bd-cr4hd\" (UID: \"ab85183c-b2e1-4682-859e-a9c2213e9c2a\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cr4hd" Jul 06 19:16:40.184552 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:40.184508 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bh8sd\" (UniqueName: \"kubernetes.io/projected/ab85183c-b2e1-4682-859e-a9c2213e9c2a-kube-api-access-bh8sd\") pod \"limitador-operator-controller-manager-859d7b78bd-cr4hd\" (UID: \"ab85183c-b2e1-4682-859e-a9c2213e9c2a\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cr4hd" Jul 06 19:16:40.192990 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:40.192952 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bh8sd\" (UniqueName: \"kubernetes.io/projected/ab85183c-b2e1-4682-859e-a9c2213e9c2a-kube-api-access-bh8sd\") pod \"limitador-operator-controller-manager-859d7b78bd-cr4hd\" (UID: \"ab85183c-b2e1-4682-859e-a9c2213e9c2a\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cr4hd" Jul 06 19:16:40.371151 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:40.371052 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cr4hd" Jul 06 19:16:41.150720 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:41.150690 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-rddw8" event={"ID":"bf9d0112-eb11-4996-8fca-e6a781f22e14","Type":"ContainerStarted","Data":"8ed935d63e0caacd58a4117aaa56fb3b17af04c6607a325c7aac514076b96477"} Jul 06 19:16:41.150997 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:41.150981 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-rddw8" Jul 06 19:16:41.151506 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:41.151474 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cr4hd"] Jul 06 19:16:41.153214 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:16:41.153185 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab85183c_b2e1_4682_859e_a9c2213e9c2a.slice/crio-a3bb9d20152497039885ad4b73490343974af9fdb1e41af5b504e67b70b99c6b WatchSource:0}: Error finding container a3bb9d20152497039885ad4b73490343974af9fdb1e41af5b504e67b70b99c6b: Status 404 returned error can't find the container with id a3bb9d20152497039885ad4b73490343974af9fdb1e41af5b504e67b70b99c6b Jul 06 19:16:41.153332 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:41.153228 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mkk8r" event={"ID":"4f53da6e-5c46-44b5-b784-87ba7e3e1fff","Type":"ContainerStarted","Data":"0be64186370a90a31643ef2870240799fb3b6ebdb86ffc6448bd8cbe7e0ce987"} Jul 06 19:16:41.153397 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:41.153376 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mkk8r" Jul 06 19:16:41.159212 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:41.159192 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:16:41.170942 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:41.170903 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-rddw8" podStartSLOduration=0.866949314 podStartE2EDuration="4.170890253s" totalImagesPullingTime="3.303940939s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:16:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:16:41.168783476 +0000 UTC m=+426.476142424" watchObservedRunningTime="2026-07-06 19:16:41.170890253 +0000 UTC m=+426.478249199" Jul 06 19:16:41.188866 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:41.188814 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mkk8r" podStartSLOduration=0.665437179 podStartE2EDuration="9.188800054s" totalImagesPullingTime="8.523362875s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:16:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:16:41.187537741 +0000 UTC m=+426.494896691" watchObservedRunningTime="2026-07-06 19:16:41.188800054 +0000 UTC m=+426.496159063" Jul 06 19:16:42.160345 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:42.160305 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cr4hd" event={"ID":"ab85183c-b2e1-4682-859e-a9c2213e9c2a","Type":"ContainerStarted","Data":"a3bb9d20152497039885ad4b73490343974af9fdb1e41af5b504e67b70b99c6b"} Jul 06 19:16:42.314940 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:42.314821 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:16:42.415914 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:42.415838 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:16:43.169030 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:43.169000 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cr4hd" event={"ID":"ab85183c-b2e1-4682-859e-a9c2213e9c2a","Type":"ContainerStarted","Data":"37b6ae77faa8c24fb594e88801f1a864278eca8c45d9f8d31ef81cb931abdc52"} Jul 06 19:16:43.169457 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:43.169113 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cr4hd" Jul 06 19:16:43.190113 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:43.190044 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cr4hd" podStartSLOduration=1.926068377 podStartE2EDuration="3.190028371s" totalImagesPullingTime="1.263959994s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:16:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:16:43.188703461 +0000 UTC m=+428.496062409" watchObservedRunningTime="2026-07-06 19:16:43.190028371 +0000 UTC m=+428.497387319" Jul 06 19:16:43.935462 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:43.935422 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:16:43.936224 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:43.936148 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-q2m67"] Jul 06 19:16:43.940351 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:43.940325 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-q2m67"] Jul 06 19:16:43.940488 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:43.940426 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-q2m67" Jul 06 19:16:43.942578 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:43.942555 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jul 06 19:16:43.942712 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:43.942575 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-bdxch\"" Jul 06 19:16:43.942790 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:43.942748 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jul 06 19:16:44.022658 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:44.022617 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/5ef34edc-9e6f-42ed-a3f1-ac7b914b2921-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-q2m67\" (UID: \"5ef34edc-9e6f-42ed-a3f1-ac7b914b2921\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-q2m67" Jul 06 19:16:44.022847 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:44.022674 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/5ef34edc-9e6f-42ed-a3f1-ac7b914b2921-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-q2m67\" (UID: \"5ef34edc-9e6f-42ed-a3f1-ac7b914b2921\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-q2m67" Jul 06 19:16:44.022847 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:44.022734 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pdpcp\" (UniqueName: \"kubernetes.io/projected/5ef34edc-9e6f-42ed-a3f1-ac7b914b2921-kube-api-access-pdpcp\") pod \"kuadrant-console-plugin-66b5774985-q2m67\" (UID: \"5ef34edc-9e6f-42ed-a3f1-ac7b914b2921\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-q2m67" Jul 06 19:16:44.030860 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:44.030832 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:16:44.123619 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:44.123571 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/5ef34edc-9e6f-42ed-a3f1-ac7b914b2921-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-q2m67\" (UID: \"5ef34edc-9e6f-42ed-a3f1-ac7b914b2921\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-q2m67" Jul 06 19:16:44.123776 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:44.123645 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/5ef34edc-9e6f-42ed-a3f1-ac7b914b2921-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-q2m67\" (UID: \"5ef34edc-9e6f-42ed-a3f1-ac7b914b2921\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-q2m67" Jul 06 19:16:44.123776 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:44.123672 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pdpcp\" (UniqueName: \"kubernetes.io/projected/5ef34edc-9e6f-42ed-a3f1-ac7b914b2921-kube-api-access-pdpcp\") pod \"kuadrant-console-plugin-66b5774985-q2m67\" (UID: \"5ef34edc-9e6f-42ed-a3f1-ac7b914b2921\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-q2m67" Jul 06 19:16:44.124400 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:44.124373 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/5ef34edc-9e6f-42ed-a3f1-ac7b914b2921-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-q2m67\" (UID: \"5ef34edc-9e6f-42ed-a3f1-ac7b914b2921\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-q2m67" Jul 06 19:16:44.126278 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:44.126248 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/5ef34edc-9e6f-42ed-a3f1-ac7b914b2921-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-q2m67\" (UID: \"5ef34edc-9e6f-42ed-a3f1-ac7b914b2921\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-q2m67" Jul 06 19:16:44.131905 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:44.131873 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pdpcp\" (UniqueName: \"kubernetes.io/projected/5ef34edc-9e6f-42ed-a3f1-ac7b914b2921-kube-api-access-pdpcp\") pod \"kuadrant-console-plugin-66b5774985-q2m67\" (UID: \"5ef34edc-9e6f-42ed-a3f1-ac7b914b2921\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-q2m67" Jul 06 19:16:44.251187 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:44.251106 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-q2m67" Jul 06 19:16:44.381182 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:44.381153 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-q2m67"] Jul 06 19:16:44.382640 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:16:44.382614 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5ef34edc_9e6f_42ed_a3f1_ac7b914b2921.slice/crio-8159baf688505443039de18ba503ac8b46814f3ca0d26e5c0433af712093e6e4 WatchSource:0}: Error finding container 8159baf688505443039de18ba503ac8b46814f3ca0d26e5c0433af712093e6e4: Status 404 returned error can't find the container with id 8159baf688505443039de18ba503ac8b46814f3ca0d26e5c0433af712093e6e4 Jul 06 19:16:45.181697 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:45.181662 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-q2m67" event={"ID":"5ef34edc-9e6f-42ed-a3f1-ac7b914b2921","Type":"ContainerStarted","Data":"8159baf688505443039de18ba503ac8b46814f3ca0d26e5c0433af712093e6e4"} Jul 06 19:16:45.186621 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:45.186594 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:16:48.131763 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:48.131729 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-5j2ll" Jul 06 19:16:50.116492 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:50.116449 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:16:50.217526 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:50.217488 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:16:52.163079 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:52.163050 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mkk8r" Jul 06 19:16:52.163458 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:52.163132 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-rddw8" Jul 06 19:16:54.029113 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:54.029062 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mkk8r"] Jul 06 19:16:54.029537 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:54.029259 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mkk8r" podUID="4f53da6e-5c46-44b5-b784-87ba7e3e1fff" containerName="manager" containerID="cri-o://0be64186370a90a31643ef2870240799fb3b6ebdb86ffc6448bd8cbe7e0ce987" gracePeriod=2 Jul 06 19:16:54.039899 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:54.039872 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mkk8r"] Jul 06 19:16:54.045145 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:16:54.045108 2566 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"4f53da6e-5c46-44b5-b784-87ba7e3e1fff\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-8684c9ff49-mkk8r\": pods \"kuadrant-operator-controller-manager-8684c9ff49-mkk8r\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mkk8r" Jul 06 19:16:54.055787 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:54.055760 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cr4hd"] Jul 06 19:16:54.055984 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:54.055953 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cr4hd" podUID="ab85183c-b2e1-4682-859e-a9c2213e9c2a" containerName="manager" containerID="cri-o://37b6ae77faa8c24fb594e88801f1a864278eca8c45d9f8d31ef81cb931abdc52" gracePeriod=2 Jul 06 19:16:54.058415 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:54.058387 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cr4hd" Jul 06 19:16:54.061449 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:54.061423 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8bxsc"] Jul 06 19:16:54.061783 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:54.061764 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f53da6e-5c46-44b5-b784-87ba7e3e1fff" containerName="manager" Jul 06 19:16:54.061783 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:54.061781 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f53da6e-5c46-44b5-b784-87ba7e3e1fff" containerName="manager" Jul 06 19:16:54.061939 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:54.061893 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f53da6e-5c46-44b5-b784-87ba7e3e1fff" containerName="manager" Jul 06 19:16:54.065177 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:54.065123 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8bxsc" Jul 06 19:16:54.068837 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:54.068809 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cr4hd"] Jul 06 19:16:54.081531 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:16:54.081491 2566 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"ab85183c-b2e1-4682-859e-a9c2213e9c2a\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"limitador-operator-controller-manager-859d7b78bd-cr4hd\": pods \"limitador-operator-controller-manager-859d7b78bd-cr4hd\" not found" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cr4hd" Jul 06 19:16:54.084351 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:54.084326 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8bxsc"] Jul 06 19:16:54.086500 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:16:54.086459 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-mkk8r\" is forbidden: User \"system:node:ip-10-0-138-35.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-35.ec2.internal' and this object" podUID="4f53da6e-5c46-44b5-b784-87ba7e3e1fff" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mkk8r" Jul 06 19:16:54.089321 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:54.089296 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lxjln"] Jul 06 19:16:54.089759 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:16:54.089723 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-cr4hd\" is forbidden: User \"system:node:ip-10-0-138-35.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-35.ec2.internal' and this object" podUID="ab85183c-b2e1-4682-859e-a9c2213e9c2a" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cr4hd" Jul 06 19:16:54.089981 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:54.089811 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab85183c-b2e1-4682-859e-a9c2213e9c2a" containerName="manager" Jul 06 19:16:54.090094 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:54.089985 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab85183c-b2e1-4682-859e-a9c2213e9c2a" containerName="manager" Jul 06 19:16:54.090153 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:54.090138 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="ab85183c-b2e1-4682-859e-a9c2213e9c2a" containerName="manager" Jul 06 19:16:54.092642 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:16:54.092611 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-mkk8r\" is forbidden: User \"system:node:ip-10-0-138-35.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-35.ec2.internal' and this object" podUID="4f53da6e-5c46-44b5-b784-87ba7e3e1fff" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mkk8r" Jul 06 19:16:54.093972 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:54.093902 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lxjln" Jul 06 19:16:54.108707 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:54.108678 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lxjln"] Jul 06 19:16:54.115731 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:54.115537 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e4494ed9-70e8-4a38-bdd1-6aeecf9b8f95-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-8bxsc\" (UID: \"e4494ed9-70e8-4a38-bdd1-6aeecf9b8f95\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8bxsc" Jul 06 19:16:54.115731 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:54.115625 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j6687\" (UniqueName: \"kubernetes.io/projected/e4494ed9-70e8-4a38-bdd1-6aeecf9b8f95-kube-api-access-j6687\") pod \"kuadrant-operator-controller-manager-8684c9ff49-8bxsc\" (UID: \"e4494ed9-70e8-4a38-bdd1-6aeecf9b8f95\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8bxsc" Jul 06 19:16:54.123689 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:16:54.123652 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-cr4hd\" is forbidden: User \"system:node:ip-10-0-138-35.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-35.ec2.internal' and this object" podUID="ab85183c-b2e1-4682-859e-a9c2213e9c2a" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cr4hd" Jul 06 19:16:54.144896 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:16:54.144851 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-mkk8r\" is forbidden: User \"system:node:ip-10-0-138-35.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-35.ec2.internal' and this object" podUID="4f53da6e-5c46-44b5-b784-87ba7e3e1fff" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mkk8r" Jul 06 19:16:54.217233 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:54.217186 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j6687\" (UniqueName: \"kubernetes.io/projected/e4494ed9-70e8-4a38-bdd1-6aeecf9b8f95-kube-api-access-j6687\") pod \"kuadrant-operator-controller-manager-8684c9ff49-8bxsc\" (UID: \"e4494ed9-70e8-4a38-bdd1-6aeecf9b8f95\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8bxsc" Jul 06 19:16:54.217393 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:54.217312 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e4494ed9-70e8-4a38-bdd1-6aeecf9b8f95-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-8bxsc\" (UID: \"e4494ed9-70e8-4a38-bdd1-6aeecf9b8f95\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8bxsc" Jul 06 19:16:54.217393 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:54.217349 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p8jvq\" (UniqueName: \"kubernetes.io/projected/75cfa14e-1d0b-490c-ac55-15c3715e8adb-kube-api-access-p8jvq\") pod \"limitador-operator-controller-manager-859d7b78bd-lxjln\" (UID: \"75cfa14e-1d0b-490c-ac55-15c3715e8adb\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lxjln" Jul 06 19:16:54.217858 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:54.217826 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e4494ed9-70e8-4a38-bdd1-6aeecf9b8f95-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-8bxsc\" (UID: \"e4494ed9-70e8-4a38-bdd1-6aeecf9b8f95\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8bxsc" Jul 06 19:16:54.227562 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:54.227510 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j6687\" (UniqueName: \"kubernetes.io/projected/e4494ed9-70e8-4a38-bdd1-6aeecf9b8f95-kube-api-access-j6687\") pod \"kuadrant-operator-controller-manager-8684c9ff49-8bxsc\" (UID: \"e4494ed9-70e8-4a38-bdd1-6aeecf9b8f95\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8bxsc" Jul 06 19:16:54.258947 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:54.258908 2566 generic.go:356] "Generic (PLEG): container finished" podID="ab85183c-b2e1-4682-859e-a9c2213e9c2a" containerID="37b6ae77faa8c24fb594e88801f1a864278eca8c45d9f8d31ef81cb931abdc52" exitCode=0 Jul 06 19:16:54.260866 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:54.260839 2566 generic.go:356] "Generic (PLEG): container finished" podID="4f53da6e-5c46-44b5-b784-87ba7e3e1fff" containerID="0be64186370a90a31643ef2870240799fb3b6ebdb86ffc6448bd8cbe7e0ce987" exitCode=0 Jul 06 19:16:54.304594 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:54.304572 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mkk8r" Jul 06 19:16:54.306397 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:16:54.306371 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-cr4hd\" is forbidden: User \"system:node:ip-10-0-138-35.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-35.ec2.internal' and this object" podUID="ab85183c-b2e1-4682-859e-a9c2213e9c2a" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cr4hd" Jul 06 19:16:54.306779 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:54.306760 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cr4hd" Jul 06 19:16:54.307850 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:16:54.307829 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-mkk8r\" is forbidden: User \"system:node:ip-10-0-138-35.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-35.ec2.internal' and this object" podUID="4f53da6e-5c46-44b5-b784-87ba7e3e1fff" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mkk8r" Jul 06 19:16:54.309387 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:16:54.309365 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-cr4hd\" is forbidden: User \"system:node:ip-10-0-138-35.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-35.ec2.internal' and this object" podUID="ab85183c-b2e1-4682-859e-a9c2213e9c2a" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cr4hd" Jul 06 19:16:54.310799 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:16:54.310769 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-mkk8r\" is forbidden: User \"system:node:ip-10-0-138-35.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-35.ec2.internal' and this object" podUID="4f53da6e-5c46-44b5-b784-87ba7e3e1fff" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mkk8r" Jul 06 19:16:54.318276 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:54.318252 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p8jvq\" (UniqueName: \"kubernetes.io/projected/75cfa14e-1d0b-490c-ac55-15c3715e8adb-kube-api-access-p8jvq\") pod \"limitador-operator-controller-manager-859d7b78bd-lxjln\" (UID: \"75cfa14e-1d0b-490c-ac55-15c3715e8adb\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lxjln" Jul 06 19:16:54.333959 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:54.333931 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p8jvq\" (UniqueName: \"kubernetes.io/projected/75cfa14e-1d0b-490c-ac55-15c3715e8adb-kube-api-access-p8jvq\") pod \"limitador-operator-controller-manager-859d7b78bd-lxjln\" (UID: \"75cfa14e-1d0b-490c-ac55-15c3715e8adb\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lxjln" Jul 06 19:16:54.418903 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:54.418856 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bbzf9\" (UniqueName: \"kubernetes.io/projected/4f53da6e-5c46-44b5-b784-87ba7e3e1fff-kube-api-access-bbzf9\") pod \"4f53da6e-5c46-44b5-b784-87ba7e3e1fff\" (UID: \"4f53da6e-5c46-44b5-b784-87ba7e3e1fff\") " Jul 06 19:16:54.419123 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:54.418961 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4f53da6e-5c46-44b5-b784-87ba7e3e1fff-extensions-socket-volume\") pod \"4f53da6e-5c46-44b5-b784-87ba7e3e1fff\" (UID: \"4f53da6e-5c46-44b5-b784-87ba7e3e1fff\") " Jul 06 19:16:54.419123 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:54.418993 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bh8sd\" (UniqueName: \"kubernetes.io/projected/ab85183c-b2e1-4682-859e-a9c2213e9c2a-kube-api-access-bh8sd\") pod \"ab85183c-b2e1-4682-859e-a9c2213e9c2a\" (UID: \"ab85183c-b2e1-4682-859e-a9c2213e9c2a\") " Jul 06 19:16:54.419567 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:54.419541 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f53da6e-5c46-44b5-b784-87ba7e3e1fff-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "4f53da6e-5c46-44b5-b784-87ba7e3e1fff" (UID: "4f53da6e-5c46-44b5-b784-87ba7e3e1fff"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:16:54.421173 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:54.421147 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4f53da6e-5c46-44b5-b784-87ba7e3e1fff-kube-api-access-bbzf9" (OuterVolumeSpecName: "kube-api-access-bbzf9") pod "4f53da6e-5c46-44b5-b784-87ba7e3e1fff" (UID: "4f53da6e-5c46-44b5-b784-87ba7e3e1fff"). InnerVolumeSpecName "kube-api-access-bbzf9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:16:54.421275 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:54.421254 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ab85183c-b2e1-4682-859e-a9c2213e9c2a-kube-api-access-bh8sd" (OuterVolumeSpecName: "kube-api-access-bh8sd") pod "ab85183c-b2e1-4682-859e-a9c2213e9c2a" (UID: "ab85183c-b2e1-4682-859e-a9c2213e9c2a"). InnerVolumeSpecName "kube-api-access-bh8sd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:16:54.463321 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:54.463283 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8bxsc" Jul 06 19:16:54.470673 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:54.470652 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lxjln" Jul 06 19:16:54.520282 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:54.520246 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bh8sd\" (UniqueName: \"kubernetes.io/projected/ab85183c-b2e1-4682-859e-a9c2213e9c2a-kube-api-access-bh8sd\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:16:54.520282 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:54.520276 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bbzf9\" (UniqueName: \"kubernetes.io/projected/4f53da6e-5c46-44b5-b784-87ba7e3e1fff-kube-api-access-bbzf9\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:16:54.520282 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:54.520286 2566 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4f53da6e-5c46-44b5-b784-87ba7e3e1fff-extensions-socket-volume\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:16:54.632038 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:54.632015 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8bxsc"] Jul 06 19:16:54.634213 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:16:54.634183 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4494ed9_70e8_4a38_bdd1_6aeecf9b8f95.slice/crio-d1e8e5d33dfee127807603a49b61be416cbb8c6e62c4c0d50637cd826edb89c5 WatchSource:0}: Error finding container d1e8e5d33dfee127807603a49b61be416cbb8c6e62c4c0d50637cd826edb89c5: Status 404 returned error can't find the container with id d1e8e5d33dfee127807603a49b61be416cbb8c6e62c4c0d50637cd826edb89c5 Jul 06 19:16:54.646824 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:54.646794 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lxjln"] Jul 06 19:16:55.235730 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:16:55.235643 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-cr4hd\" is forbidden: User \"system:node:ip-10-0-138-35.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-35.ec2.internal' and this object" podUID="ab85183c-b2e1-4682-859e-a9c2213e9c2a" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cr4hd" Jul 06 19:16:55.237214 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:16:55.237177 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-mkk8r\" is forbidden: User \"system:node:ip-10-0-138-35.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-35.ec2.internal' and this object" podUID="4f53da6e-5c46-44b5-b784-87ba7e3e1fff" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mkk8r" Jul 06 19:16:55.238549 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:55.238520 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4f53da6e-5c46-44b5-b784-87ba7e3e1fff" path="/var/lib/kubelet/pods/4f53da6e-5c46-44b5-b784-87ba7e3e1fff/volumes" Jul 06 19:16:55.238972 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:55.238952 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ab85183c-b2e1-4682-859e-a9c2213e9c2a" path="/var/lib/kubelet/pods/ab85183c-b2e1-4682-859e-a9c2213e9c2a/volumes" Jul 06 19:16:55.268092 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:55.268039 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cr4hd" Jul 06 19:16:55.268092 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:55.268042 2566 scope.go:117] "RemoveContainer" containerID="37b6ae77faa8c24fb594e88801f1a864278eca8c45d9f8d31ef81cb931abdc52" Jul 06 19:16:55.270271 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:55.270236 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8bxsc" event={"ID":"e4494ed9-70e8-4a38-bdd1-6aeecf9b8f95","Type":"ContainerStarted","Data":"d1e8e5d33dfee127807603a49b61be416cbb8c6e62c4c0d50637cd826edb89c5"} Jul 06 19:16:55.272100 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:55.272060 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mkk8r" Jul 06 19:16:55.928196 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:55.928160 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:16:56.028554 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:56.028509 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:16:58.922429 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:58.922372 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:16:59.043623 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:16:59.043582 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:17:02.715635 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:02.715601 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:17:02.815997 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:02.815969 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:17:06.755783 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:06.755749 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:17:06.815626 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:06.815588 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:17:08.675465 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:17:08.675437 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75cfa14e_1d0b_490c_ac55_15c3715e8adb.slice/crio-ce66362a5f0d634f5a4036e7999341803bb8382eb43ac2730a53ea3b4a121cfe WatchSource:0}: Error finding container ce66362a5f0d634f5a4036e7999341803bb8382eb43ac2730a53ea3b4a121cfe: Status 404 returned error can't find the container with id ce66362a5f0d634f5a4036e7999341803bb8382eb43ac2730a53ea3b4a121cfe Jul 06 19:17:08.677864 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:08.677821 2566 scope.go:117] "RemoveContainer" containerID="0be64186370a90a31643ef2870240799fb3b6ebdb86ffc6448bd8cbe7e0ce987" Jul 06 19:17:08.710257 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:08.710232 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:17:09.024492 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:09.024396 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:17:09.124915 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:09.124858 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:17:09.361613 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:09.361575 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-q2m67" event={"ID":"5ef34edc-9e6f-42ed-a3f1-ac7b914b2921","Type":"ContainerStarted","Data":"2dcc18b92b55a793eaf301838f8753cb6038f637685e009e7eda5a5cff0c10ca"} Jul 06 19:17:09.363196 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:09.363172 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8bxsc" event={"ID":"e4494ed9-70e8-4a38-bdd1-6aeecf9b8f95","Type":"ContainerStarted","Data":"23242c9af84341094fd038a9350595999303ff9bc821c94dacfeab143de2adf2"} Jul 06 19:17:09.363329 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:09.363221 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8bxsc" Jul 06 19:17:09.364762 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:09.364734 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lxjln" event={"ID":"75cfa14e-1d0b-490c-ac55-15c3715e8adb","Type":"ContainerStarted","Data":"e53a7b526c85167524db789146f1788095481adfb6408fd24065e482cb7bc826"} Jul 06 19:17:09.364888 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:09.364787 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lxjln" event={"ID":"75cfa14e-1d0b-490c-ac55-15c3715e8adb","Type":"ContainerStarted","Data":"ce66362a5f0d634f5a4036e7999341803bb8382eb43ac2730a53ea3b4a121cfe"} Jul 06 19:17:09.364888 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:09.364844 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lxjln" Jul 06 19:17:09.377738 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:09.377688 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-q2m67" podStartSLOduration=2.010554495 podStartE2EDuration="26.377674008s" totalImagesPullingTime="24.367119513s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:16:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:17:09.376927191 +0000 UTC m=+454.684286141" watchObservedRunningTime="2026-07-06 19:17:09.377674008 +0000 UTC m=+454.685032956" Jul 06 19:17:09.398981 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:09.398928 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8bxsc" podStartSLOduration=15.398910601 podStartE2EDuration="15.398910601s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:16:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:17:09.397019471 +0000 UTC m=+454.704378426" watchObservedRunningTime="2026-07-06 19:17:09.398910601 +0000 UTC m=+454.706269607" Jul 06 19:17:09.414219 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:09.414167 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lxjln" podStartSLOduration=15.414150676 podStartE2EDuration="15.414150676s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:16:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:17:09.413205351 +0000 UTC m=+454.720564303" watchObservedRunningTime="2026-07-06 19:17:09.414150676 +0000 UTC m=+454.721509623" Jul 06 19:17:11.748696 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:11.748656 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8bxsc"] Jul 06 19:17:11.749152 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:11.748842 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8bxsc" podUID="e4494ed9-70e8-4a38-bdd1-6aeecf9b8f95" containerName="manager" containerID="cri-o://23242c9af84341094fd038a9350595999303ff9bc821c94dacfeab143de2adf2" gracePeriod=10 Jul 06 19:17:12.241898 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:12.241862 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rvpvz"] Jul 06 19:17:12.272715 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:12.272684 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rvpvz"] Jul 06 19:17:12.272932 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:12.272786 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rvpvz" Jul 06 19:17:12.383184 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:12.383146 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/62641703-00a9-4b2e-9d0d-d4a5a1540cfd-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-rvpvz\" (UID: \"62641703-00a9-4b2e-9d0d-d4a5a1540cfd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rvpvz" Jul 06 19:17:12.383366 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:12.383303 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6wl84\" (UniqueName: \"kubernetes.io/projected/62641703-00a9-4b2e-9d0d-d4a5a1540cfd-kube-api-access-6wl84\") pod \"kuadrant-operator-controller-manager-86878c769f-rvpvz\" (UID: \"62641703-00a9-4b2e-9d0d-d4a5a1540cfd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rvpvz" Jul 06 19:17:12.384199 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:12.384171 2566 generic.go:356] "Generic (PLEG): container finished" podID="e4494ed9-70e8-4a38-bdd1-6aeecf9b8f95" containerID="23242c9af84341094fd038a9350595999303ff9bc821c94dacfeab143de2adf2" exitCode=0 Jul 06 19:17:12.384305 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:12.384230 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8bxsc" event={"ID":"e4494ed9-70e8-4a38-bdd1-6aeecf9b8f95","Type":"ContainerDied","Data":"23242c9af84341094fd038a9350595999303ff9bc821c94dacfeab143de2adf2"} Jul 06 19:17:12.484658 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:12.484618 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6wl84\" (UniqueName: \"kubernetes.io/projected/62641703-00a9-4b2e-9d0d-d4a5a1540cfd-kube-api-access-6wl84\") pod \"kuadrant-operator-controller-manager-86878c769f-rvpvz\" (UID: \"62641703-00a9-4b2e-9d0d-d4a5a1540cfd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rvpvz" Jul 06 19:17:12.484821 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:12.484669 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/62641703-00a9-4b2e-9d0d-d4a5a1540cfd-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-rvpvz\" (UID: \"62641703-00a9-4b2e-9d0d-d4a5a1540cfd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rvpvz" Jul 06 19:17:12.485006 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:12.484988 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/62641703-00a9-4b2e-9d0d-d4a5a1540cfd-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-rvpvz\" (UID: \"62641703-00a9-4b2e-9d0d-d4a5a1540cfd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rvpvz" Jul 06 19:17:12.493530 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:12.493453 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6wl84\" (UniqueName: \"kubernetes.io/projected/62641703-00a9-4b2e-9d0d-d4a5a1540cfd-kube-api-access-6wl84\") pod \"kuadrant-operator-controller-manager-86878c769f-rvpvz\" (UID: \"62641703-00a9-4b2e-9d0d-d4a5a1540cfd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rvpvz" Jul 06 19:17:12.538931 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:12.538899 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8bxsc" Jul 06 19:17:12.583895 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:12.583859 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rvpvz" Jul 06 19:17:12.685847 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:12.685624 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j6687\" (UniqueName: \"kubernetes.io/projected/e4494ed9-70e8-4a38-bdd1-6aeecf9b8f95-kube-api-access-j6687\") pod \"e4494ed9-70e8-4a38-bdd1-6aeecf9b8f95\" (UID: \"e4494ed9-70e8-4a38-bdd1-6aeecf9b8f95\") " Jul 06 19:17:12.685847 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:12.685765 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e4494ed9-70e8-4a38-bdd1-6aeecf9b8f95-extensions-socket-volume\") pod \"e4494ed9-70e8-4a38-bdd1-6aeecf9b8f95\" (UID: \"e4494ed9-70e8-4a38-bdd1-6aeecf9b8f95\") " Jul 06 19:17:12.686436 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:12.686389 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4494ed9-70e8-4a38-bdd1-6aeecf9b8f95-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "e4494ed9-70e8-4a38-bdd1-6aeecf9b8f95" (UID: "e4494ed9-70e8-4a38-bdd1-6aeecf9b8f95"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:17:12.688748 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:12.688714 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e4494ed9-70e8-4a38-bdd1-6aeecf9b8f95-kube-api-access-j6687" (OuterVolumeSpecName: "kube-api-access-j6687") pod "e4494ed9-70e8-4a38-bdd1-6aeecf9b8f95" (UID: "e4494ed9-70e8-4a38-bdd1-6aeecf9b8f95"). InnerVolumeSpecName "kube-api-access-j6687". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:17:12.778422 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:12.778393 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rvpvz"] Jul 06 19:17:12.781165 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:17:12.781130 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62641703_00a9_4b2e_9d0d_d4a5a1540cfd.slice/crio-607b0de5fefee39121c9fb10bf0a703076daa0a1072011e053d09977bdc3c299 WatchSource:0}: Error finding container 607b0de5fefee39121c9fb10bf0a703076daa0a1072011e053d09977bdc3c299: Status 404 returned error can't find the container with id 607b0de5fefee39121c9fb10bf0a703076daa0a1072011e053d09977bdc3c299 Jul 06 19:17:12.786789 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:12.786766 2566 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e4494ed9-70e8-4a38-bdd1-6aeecf9b8f95-extensions-socket-volume\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:17:12.786789 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:12.786789 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j6687\" (UniqueName: \"kubernetes.io/projected/e4494ed9-70e8-4a38-bdd1-6aeecf9b8f95-kube-api-access-j6687\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:17:13.391848 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:13.391823 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8bxsc" Jul 06 19:17:13.392052 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:13.391864 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8bxsc" event={"ID":"e4494ed9-70e8-4a38-bdd1-6aeecf9b8f95","Type":"ContainerDied","Data":"d1e8e5d33dfee127807603a49b61be416cbb8c6e62c4c0d50637cd826edb89c5"} Jul 06 19:17:13.392052 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:13.391932 2566 scope.go:117] "RemoveContainer" containerID="23242c9af84341094fd038a9350595999303ff9bc821c94dacfeab143de2adf2" Jul 06 19:17:13.393826 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:13.393800 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rvpvz" event={"ID":"62641703-00a9-4b2e-9d0d-d4a5a1540cfd","Type":"ContainerStarted","Data":"7453e9d11490e652b5504b1b6b4572595ce835f3f182f4ab364a7dedc0e52050"} Jul 06 19:17:13.393965 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:13.393847 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rvpvz" event={"ID":"62641703-00a9-4b2e-9d0d-d4a5a1540cfd","Type":"ContainerStarted","Data":"607b0de5fefee39121c9fb10bf0a703076daa0a1072011e053d09977bdc3c299"} Jul 06 19:17:13.393965 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:13.393954 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rvpvz" Jul 06 19:17:13.417957 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:13.417910 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rvpvz" podStartSLOduration=1.417891258 podStartE2EDuration="1.417891258s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:17:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:17:13.416580211 +0000 UTC m=+458.723939160" watchObservedRunningTime="2026-07-06 19:17:13.417891258 +0000 UTC m=+458.725250269" Jul 06 19:17:13.430596 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:13.430561 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8bxsc"] Jul 06 19:17:13.436426 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:13.436389 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8bxsc"] Jul 06 19:17:15.237980 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:15.237945 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e4494ed9-70e8-4a38-bdd1-6aeecf9b8f95" path="/var/lib/kubelet/pods/e4494ed9-70e8-4a38-bdd1-6aeecf9b8f95/volumes" Jul 06 19:17:15.589721 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:15.589688 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:17:15.690151 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:15.690118 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:17:18.591895 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:18.591866 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:17:18.689507 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:18.689473 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:17:20.373785 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:20.373755 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-lxjln" Jul 06 19:17:22.805377 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:22.805338 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:17:22.906155 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:22.906125 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:17:24.402239 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:24.402203 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rvpvz" Jul 06 19:17:28.451944 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:28.451907 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t9fgz"] Jul 06 19:17:28.452493 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:28.452471 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e4494ed9-70e8-4a38-bdd1-6aeecf9b8f95" containerName="manager" Jul 06 19:17:28.452586 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:28.452497 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="e4494ed9-70e8-4a38-bdd1-6aeecf9b8f95" containerName="manager" Jul 06 19:17:28.452641 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:28.452613 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="e4494ed9-70e8-4a38-bdd1-6aeecf9b8f95" containerName="manager" Jul 06 19:17:28.454765 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:28.454743 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t9fgz" Jul 06 19:17:28.457054 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:28.457032 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-m97f7\"" Jul 06 19:17:28.468230 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:28.468199 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t9fgz"] Jul 06 19:17:28.515544 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:28.515510 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:17:28.592325 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:28.592294 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:17:28.625281 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:28.625247 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0af75ee7-3c24-4d81-9764-64b52e4f3623-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-t9fgz\" (UID: \"0af75ee7-3c24-4d81-9764-64b52e4f3623\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t9fgz" Jul 06 19:17:28.625692 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:28.625288 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0af75ee7-3c24-4d81-9764-64b52e4f3623-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-t9fgz\" (UID: \"0af75ee7-3c24-4d81-9764-64b52e4f3623\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t9fgz" Jul 06 19:17:28.625692 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:28.625318 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0af75ee7-3c24-4d81-9764-64b52e4f3623-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-t9fgz\" (UID: \"0af75ee7-3c24-4d81-9764-64b52e4f3623\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t9fgz" Jul 06 19:17:28.625692 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:28.625387 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0af75ee7-3c24-4d81-9764-64b52e4f3623-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-t9fgz\" (UID: \"0af75ee7-3c24-4d81-9764-64b52e4f3623\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t9fgz" Jul 06 19:17:28.625692 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:28.625421 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fpvgf\" (UniqueName: \"kubernetes.io/projected/0af75ee7-3c24-4d81-9764-64b52e4f3623-kube-api-access-fpvgf\") pod \"maas-default-gateway-openshift-default-56654dcf7-t9fgz\" (UID: \"0af75ee7-3c24-4d81-9764-64b52e4f3623\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t9fgz" Jul 06 19:17:28.625692 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:28.625464 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0af75ee7-3c24-4d81-9764-64b52e4f3623-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-t9fgz\" (UID: \"0af75ee7-3c24-4d81-9764-64b52e4f3623\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t9fgz" Jul 06 19:17:28.625692 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:28.625485 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0af75ee7-3c24-4d81-9764-64b52e4f3623-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-t9fgz\" (UID: \"0af75ee7-3c24-4d81-9764-64b52e4f3623\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t9fgz" Jul 06 19:17:28.625692 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:28.625544 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0af75ee7-3c24-4d81-9764-64b52e4f3623-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-t9fgz\" (UID: \"0af75ee7-3c24-4d81-9764-64b52e4f3623\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t9fgz" Jul 06 19:17:28.625692 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:28.625561 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0af75ee7-3c24-4d81-9764-64b52e4f3623-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-t9fgz\" (UID: \"0af75ee7-3c24-4d81-9764-64b52e4f3623\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t9fgz" Jul 06 19:17:28.726059 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:28.725959 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0af75ee7-3c24-4d81-9764-64b52e4f3623-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-t9fgz\" (UID: \"0af75ee7-3c24-4d81-9764-64b52e4f3623\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t9fgz" Jul 06 19:17:28.726059 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:28.726001 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0af75ee7-3c24-4d81-9764-64b52e4f3623-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-t9fgz\" (UID: \"0af75ee7-3c24-4d81-9764-64b52e4f3623\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t9fgz" Jul 06 19:17:28.726059 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:28.726044 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0af75ee7-3c24-4d81-9764-64b52e4f3623-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-t9fgz\" (UID: \"0af75ee7-3c24-4d81-9764-64b52e4f3623\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t9fgz" Jul 06 19:17:28.726385 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:28.726105 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0af75ee7-3c24-4d81-9764-64b52e4f3623-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-t9fgz\" (UID: \"0af75ee7-3c24-4d81-9764-64b52e4f3623\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t9fgz" Jul 06 19:17:28.726385 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:28.726155 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0af75ee7-3c24-4d81-9764-64b52e4f3623-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-t9fgz\" (UID: \"0af75ee7-3c24-4d81-9764-64b52e4f3623\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t9fgz" Jul 06 19:17:28.726385 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:28.726195 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0af75ee7-3c24-4d81-9764-64b52e4f3623-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-t9fgz\" (UID: \"0af75ee7-3c24-4d81-9764-64b52e4f3623\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t9fgz" Jul 06 19:17:28.726385 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:28.726220 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fpvgf\" (UniqueName: \"kubernetes.io/projected/0af75ee7-3c24-4d81-9764-64b52e4f3623-kube-api-access-fpvgf\") pod \"maas-default-gateway-openshift-default-56654dcf7-t9fgz\" (UID: \"0af75ee7-3c24-4d81-9764-64b52e4f3623\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t9fgz" Jul 06 19:17:28.726385 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:28.726263 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0af75ee7-3c24-4d81-9764-64b52e4f3623-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-t9fgz\" (UID: \"0af75ee7-3c24-4d81-9764-64b52e4f3623\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t9fgz" Jul 06 19:17:28.726385 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:28.726296 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0af75ee7-3c24-4d81-9764-64b52e4f3623-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-t9fgz\" (UID: \"0af75ee7-3c24-4d81-9764-64b52e4f3623\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t9fgz" Jul 06 19:17:28.726734 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:28.726407 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0af75ee7-3c24-4d81-9764-64b52e4f3623-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-t9fgz\" (UID: \"0af75ee7-3c24-4d81-9764-64b52e4f3623\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t9fgz" Jul 06 19:17:28.726734 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:28.726509 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0af75ee7-3c24-4d81-9764-64b52e4f3623-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-t9fgz\" (UID: \"0af75ee7-3c24-4d81-9764-64b52e4f3623\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t9fgz" Jul 06 19:17:28.726734 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:28.726688 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0af75ee7-3c24-4d81-9764-64b52e4f3623-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-t9fgz\" (UID: \"0af75ee7-3c24-4d81-9764-64b52e4f3623\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t9fgz" Jul 06 19:17:28.726854 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:28.726793 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0af75ee7-3c24-4d81-9764-64b52e4f3623-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-t9fgz\" (UID: \"0af75ee7-3c24-4d81-9764-64b52e4f3623\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t9fgz" Jul 06 19:17:28.726897 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:28.726869 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0af75ee7-3c24-4d81-9764-64b52e4f3623-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-t9fgz\" (UID: \"0af75ee7-3c24-4d81-9764-64b52e4f3623\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t9fgz" Jul 06 19:17:28.729156 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:28.729126 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0af75ee7-3c24-4d81-9764-64b52e4f3623-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-t9fgz\" (UID: \"0af75ee7-3c24-4d81-9764-64b52e4f3623\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t9fgz" Jul 06 19:17:28.729325 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:28.729307 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0af75ee7-3c24-4d81-9764-64b52e4f3623-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-t9fgz\" (UID: \"0af75ee7-3c24-4d81-9764-64b52e4f3623\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t9fgz" Jul 06 19:17:28.734119 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:28.734087 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fpvgf\" (UniqueName: \"kubernetes.io/projected/0af75ee7-3c24-4d81-9764-64b52e4f3623-kube-api-access-fpvgf\") pod \"maas-default-gateway-openshift-default-56654dcf7-t9fgz\" (UID: \"0af75ee7-3c24-4d81-9764-64b52e4f3623\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t9fgz" Jul 06 19:17:28.734441 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:28.734419 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0af75ee7-3c24-4d81-9764-64b52e4f3623-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-t9fgz\" (UID: \"0af75ee7-3c24-4d81-9764-64b52e4f3623\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t9fgz" Jul 06 19:17:28.765542 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:28.765479 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t9fgz" Jul 06 19:17:28.907431 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:28.907399 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t9fgz"] Jul 06 19:17:28.909417 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:17:28.909390 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0af75ee7_3c24_4d81_9764_64b52e4f3623.slice/crio-1cfcabe869b986ab5696a267f6229a3114687ff3acb34aefa45f11a47aa9b5d0 WatchSource:0}: Error finding container 1cfcabe869b986ab5696a267f6229a3114687ff3acb34aefa45f11a47aa9b5d0: Status 404 returned error can't find the container with id 1cfcabe869b986ab5696a267f6229a3114687ff3acb34aefa45f11a47aa9b5d0 Jul 06 19:17:28.912105 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:28.912055 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 19:17:28.912179 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:28.912141 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 19:17:28.912179 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:28.912170 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 19:17:29.487174 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:29.487139 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t9fgz" event={"ID":"0af75ee7-3c24-4d81-9764-64b52e4f3623","Type":"ContainerStarted","Data":"9757c4a82fed1f284e65920af54fc83000d923ef5c019c6e1788f5ca2fb82a6f"} Jul 06 19:17:29.487571 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:29.487190 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t9fgz" event={"ID":"0af75ee7-3c24-4d81-9764-64b52e4f3623","Type":"ContainerStarted","Data":"1cfcabe869b986ab5696a267f6229a3114687ff3acb34aefa45f11a47aa9b5d0"} Jul 06 19:17:29.506826 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:29.506766 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t9fgz" podStartSLOduration=1.506748815 podStartE2EDuration="1.506748815s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:17:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:17:29.505397014 +0000 UTC m=+474.812755956" watchObservedRunningTime="2026-07-06 19:17:29.506748815 +0000 UTC m=+474.814107765" Jul 06 19:17:29.766462 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:29.766373 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t9fgz" Jul 06 19:17:29.771258 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:29.771233 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t9fgz" Jul 06 19:17:29.986922 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:29.986890 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:17:30.039187 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:30.039159 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:17:30.493591 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:30.493496 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t9fgz" Jul 06 19:17:30.494498 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:30.494477 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t9fgz" Jul 06 19:17:35.283916 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:35.283881 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:17:36.003798 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:36.003766 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:17:36.238548 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:36.238519 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:17:36.373720 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:36.373688 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:17:36.483090 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:36.483045 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:17:36.601852 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:36.601819 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:17:37.102385 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:37.102348 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:17:37.376598 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:37.376506 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:17:38.998569 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:38.998529 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:17:39.101929 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:39.101895 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:17:43.789758 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:43.789725 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:17:43.900701 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:43.900673 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:17:43.961661 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:43.961622 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:17:44.592725 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:44.592688 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-ggz2l"] Jul 06 19:17:44.615602 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:44.615564 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:17:44.619250 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:44.619224 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-ggz2l"] Jul 06 19:17:44.619377 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:44.619319 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-ggz2l" Jul 06 19:17:44.621499 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:44.621476 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-7q8l6\"" Jul 06 19:17:44.661096 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:44.661024 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z92xs\" (UniqueName: \"kubernetes.io/projected/fc68ec21-b4cc-46b3-a233-8b8b82d272e5-kube-api-access-z92xs\") pod \"authorino-58cd955675-ggz2l\" (UID: \"fc68ec21-b4cc-46b3-a233-8b8b82d272e5\") " pod="kuadrant-system/authorino-58cd955675-ggz2l" Jul 06 19:17:44.715200 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:44.715165 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:17:44.762689 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:44.762651 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z92xs\" (UniqueName: \"kubernetes.io/projected/fc68ec21-b4cc-46b3-a233-8b8b82d272e5-kube-api-access-z92xs\") pod \"authorino-58cd955675-ggz2l\" (UID: \"fc68ec21-b4cc-46b3-a233-8b8b82d272e5\") " pod="kuadrant-system/authorino-58cd955675-ggz2l" Jul 06 19:17:44.772170 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:44.772138 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z92xs\" (UniqueName: \"kubernetes.io/projected/fc68ec21-b4cc-46b3-a233-8b8b82d272e5-kube-api-access-z92xs\") pod \"authorino-58cd955675-ggz2l\" (UID: \"fc68ec21-b4cc-46b3-a233-8b8b82d272e5\") " pod="kuadrant-system/authorino-58cd955675-ggz2l" Jul 06 19:17:44.776180 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:44.776154 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-tfdpx"] Jul 06 19:17:44.778671 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:44.778651 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-tfdpx" Jul 06 19:17:44.786491 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:44.786449 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-tfdpx"] Jul 06 19:17:44.863498 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:44.863414 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vh5g6\" (UniqueName: \"kubernetes.io/projected/fb73cf0a-47dc-4258-be1f-1f922d1c3439-kube-api-access-vh5g6\") pod \"authorino-75d556bd77-tfdpx\" (UID: \"fb73cf0a-47dc-4258-be1f-1f922d1c3439\") " pod="kuadrant-system/authorino-75d556bd77-tfdpx" Jul 06 19:17:44.929287 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:44.929252 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-ggz2l" Jul 06 19:17:44.963984 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:44.963943 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vh5g6\" (UniqueName: \"kubernetes.io/projected/fb73cf0a-47dc-4258-be1f-1f922d1c3439-kube-api-access-vh5g6\") pod \"authorino-75d556bd77-tfdpx\" (UID: \"fb73cf0a-47dc-4258-be1f-1f922d1c3439\") " pod="kuadrant-system/authorino-75d556bd77-tfdpx" Jul 06 19:17:44.972574 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:44.972546 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vh5g6\" (UniqueName: \"kubernetes.io/projected/fb73cf0a-47dc-4258-be1f-1f922d1c3439-kube-api-access-vh5g6\") pod \"authorino-75d556bd77-tfdpx\" (UID: \"fb73cf0a-47dc-4258-be1f-1f922d1c3439\") " pod="kuadrant-system/authorino-75d556bd77-tfdpx" Jul 06 19:17:45.068856 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:45.068829 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-ggz2l"] Jul 06 19:17:45.071027 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:17:45.070999 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfc68ec21_b4cc_46b3_a233_8b8b82d272e5.slice/crio-ad0e426d7e1d559e75b525ffe1eba1e278f09f7fea6a0d8b8c7c7b779ecadf45 WatchSource:0}: Error finding container ad0e426d7e1d559e75b525ffe1eba1e278f09f7fea6a0d8b8c7c7b779ecadf45: Status 404 returned error can't find the container with id ad0e426d7e1d559e75b525ffe1eba1e278f09f7fea6a0d8b8c7c7b779ecadf45 Jul 06 19:17:45.090034 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:45.090004 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-tfdpx" Jul 06 19:17:45.233779 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:17:45.233748 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfb73cf0a_47dc_4258_be1f_1f922d1c3439.slice/crio-19fdb656f38101726e8fdc33255be96ffadf79485f1fc1a993a29bbdb75408a3 WatchSource:0}: Error finding container 19fdb656f38101726e8fdc33255be96ffadf79485f1fc1a993a29bbdb75408a3: Status 404 returned error can't find the container with id 19fdb656f38101726e8fdc33255be96ffadf79485f1fc1a993a29bbdb75408a3 Jul 06 19:17:45.237369 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:45.237346 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-tfdpx"] Jul 06 19:17:45.572773 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:45.572730 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-tfdpx" event={"ID":"fb73cf0a-47dc-4258-be1f-1f922d1c3439","Type":"ContainerStarted","Data":"19fdb656f38101726e8fdc33255be96ffadf79485f1fc1a993a29bbdb75408a3"} Jul 06 19:17:45.573942 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:45.573914 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-ggz2l" event={"ID":"fc68ec21-b4cc-46b3-a233-8b8b82d272e5","Type":"ContainerStarted","Data":"ad0e426d7e1d559e75b525ffe1eba1e278f09f7fea6a0d8b8c7c7b779ecadf45"} Jul 06 19:17:47.133438 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:47.133403 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:17:47.234466 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:47.234425 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:17:49.601709 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:49.601671 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-tfdpx" event={"ID":"fb73cf0a-47dc-4258-be1f-1f922d1c3439","Type":"ContainerStarted","Data":"47d2511acaaff2544ce93566791d00e03e22f148062cb3987146fbb03ab3b35b"} Jul 06 19:17:49.603117 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:49.603090 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-ggz2l" event={"ID":"fc68ec21-b4cc-46b3-a233-8b8b82d272e5","Type":"ContainerStarted","Data":"75dbd9d378da72133b3a6abfeecfb2670c43894f47fbe696fee5c6bbbb1a945e"} Jul 06 19:17:49.615812 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:49.615758 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-tfdpx" podStartSLOduration=1.7897625289999999 podStartE2EDuration="5.615744119s" totalImagesPullingTime="3.82598159s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:17:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:17:49.615141073 +0000 UTC m=+494.922500022" watchObservedRunningTime="2026-07-06 19:17:49.615744119 +0000 UTC m=+494.923103069" Jul 06 19:17:49.630751 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:49.630693 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-ggz2l" podStartSLOduration=1.630436123 podStartE2EDuration="5.630673576s" totalImagesPullingTime="4.000237453s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:17:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:17:49.628476239 +0000 UTC m=+494.935835189" watchObservedRunningTime="2026-07-06 19:17:49.630673576 +0000 UTC m=+494.938032524" Jul 06 19:17:49.654693 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:49.654621 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-ggz2l"] Jul 06 19:17:51.614963 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:51.614924 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-ggz2l" podUID="fc68ec21-b4cc-46b3-a233-8b8b82d272e5" containerName="authorino" containerID="cri-o://75dbd9d378da72133b3a6abfeecfb2670c43894f47fbe696fee5c6bbbb1a945e" gracePeriod=30 Jul 06 19:17:51.872466 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:51.872401 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-ggz2l" Jul 06 19:17:51.931058 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:51.931012 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z92xs\" (UniqueName: \"kubernetes.io/projected/fc68ec21-b4cc-46b3-a233-8b8b82d272e5-kube-api-access-z92xs\") pod \"fc68ec21-b4cc-46b3-a233-8b8b82d272e5\" (UID: \"fc68ec21-b4cc-46b3-a233-8b8b82d272e5\") " Jul 06 19:17:51.933346 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:51.933320 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc68ec21-b4cc-46b3-a233-8b8b82d272e5-kube-api-access-z92xs" (OuterVolumeSpecName: "kube-api-access-z92xs") pod "fc68ec21-b4cc-46b3-a233-8b8b82d272e5" (UID: "fc68ec21-b4cc-46b3-a233-8b8b82d272e5"). InnerVolumeSpecName "kube-api-access-z92xs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:17:52.031735 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:52.031695 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z92xs\" (UniqueName: \"kubernetes.io/projected/fc68ec21-b4cc-46b3-a233-8b8b82d272e5-kube-api-access-z92xs\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:17:52.622159 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:52.622122 2566 generic.go:356] "Generic (PLEG): container finished" podID="fc68ec21-b4cc-46b3-a233-8b8b82d272e5" containerID="75dbd9d378da72133b3a6abfeecfb2670c43894f47fbe696fee5c6bbbb1a945e" exitCode=0 Jul 06 19:17:52.622565 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:52.622164 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-ggz2l" Jul 06 19:17:52.622565 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:52.622194 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-ggz2l" event={"ID":"fc68ec21-b4cc-46b3-a233-8b8b82d272e5","Type":"ContainerDied","Data":"75dbd9d378da72133b3a6abfeecfb2670c43894f47fbe696fee5c6bbbb1a945e"} Jul 06 19:17:52.622565 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:52.622225 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-ggz2l" event={"ID":"fc68ec21-b4cc-46b3-a233-8b8b82d272e5","Type":"ContainerDied","Data":"ad0e426d7e1d559e75b525ffe1eba1e278f09f7fea6a0d8b8c7c7b779ecadf45"} Jul 06 19:17:52.622565 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:52.622243 2566 scope.go:117] "RemoveContainer" containerID="75dbd9d378da72133b3a6abfeecfb2670c43894f47fbe696fee5c6bbbb1a945e" Jul 06 19:17:52.632132 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:52.632108 2566 scope.go:117] "RemoveContainer" containerID="75dbd9d378da72133b3a6abfeecfb2670c43894f47fbe696fee5c6bbbb1a945e" Jul 06 19:17:52.632418 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:17:52.632398 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"75dbd9d378da72133b3a6abfeecfb2670c43894f47fbe696fee5c6bbbb1a945e\": container with ID starting with 75dbd9d378da72133b3a6abfeecfb2670c43894f47fbe696fee5c6bbbb1a945e not found: ID does not exist" containerID="75dbd9d378da72133b3a6abfeecfb2670c43894f47fbe696fee5c6bbbb1a945e" Jul 06 19:17:52.632489 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:52.632431 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"75dbd9d378da72133b3a6abfeecfb2670c43894f47fbe696fee5c6bbbb1a945e"} err="failed to get container status \"75dbd9d378da72133b3a6abfeecfb2670c43894f47fbe696fee5c6bbbb1a945e\": rpc error: code = NotFound desc = could not find container \"75dbd9d378da72133b3a6abfeecfb2670c43894f47fbe696fee5c6bbbb1a945e\": container with ID starting with 75dbd9d378da72133b3a6abfeecfb2670c43894f47fbe696fee5c6bbbb1a945e not found: ID does not exist" Jul 06 19:17:52.642301 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:52.642276 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-ggz2l"] Jul 06 19:17:52.648610 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:52.648585 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-ggz2l"] Jul 06 19:17:53.236779 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:17:53.236742 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fc68ec21-b4cc-46b3-a233-8b8b82d272e5" path="/var/lib/kubelet/pods/fc68ec21-b4cc-46b3-a233-8b8b82d272e5/volumes" Jul 06 19:18:02.451867 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:18:02.451807 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:18:15.816631 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:18:15.816591 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:18:16.183672 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:18:16.183584 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:18:16.321000 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:18:16.320962 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-574449bdd-xdwr9"] Jul 06 19:18:16.321457 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:18:16.321438 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fc68ec21-b4cc-46b3-a233-8b8b82d272e5" containerName="authorino" Jul 06 19:18:16.321457 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:18:16.321456 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="fc68ec21-b4cc-46b3-a233-8b8b82d272e5" containerName="authorino" Jul 06 19:18:16.321643 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:18:16.321549 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="fc68ec21-b4cc-46b3-a233-8b8b82d272e5" containerName="authorino" Jul 06 19:18:16.324718 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:18:16.324700 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-574449bdd-xdwr9" Jul 06 19:18:16.334737 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:18:16.334709 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-574449bdd-xdwr9"] Jul 06 19:18:16.406177 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:18:16.406140 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-574449bdd-xdwr9"] Jul 06 19:18:16.406428 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:18:16.406408 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-vc6dw], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-574449bdd-xdwr9" podUID="1b1951f6-ea14-4376-aa45-0802aa661db9" Jul 06 19:18:16.447189 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:18:16.447101 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vc6dw\" (UniqueName: \"kubernetes.io/projected/1b1951f6-ea14-4376-aa45-0802aa661db9-kube-api-access-vc6dw\") pod \"authorino-574449bdd-xdwr9\" (UID: \"1b1951f6-ea14-4376-aa45-0802aa661db9\") " pod="kuadrant-system/authorino-574449bdd-xdwr9" Jul 06 19:18:16.547687 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:18:16.547646 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vc6dw\" (UniqueName: \"kubernetes.io/projected/1b1951f6-ea14-4376-aa45-0802aa661db9-kube-api-access-vc6dw\") pod \"authorino-574449bdd-xdwr9\" (UID: \"1b1951f6-ea14-4376-aa45-0802aa661db9\") " pod="kuadrant-system/authorino-574449bdd-xdwr9" Jul 06 19:18:16.557234 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:18:16.557189 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vc6dw\" (UniqueName: \"kubernetes.io/projected/1b1951f6-ea14-4376-aa45-0802aa661db9-kube-api-access-vc6dw\") pod \"authorino-574449bdd-xdwr9\" (UID: \"1b1951f6-ea14-4376-aa45-0802aa661db9\") " pod="kuadrant-system/authorino-574449bdd-xdwr9" Jul 06 19:18:16.754229 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:18:16.754142 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-574449bdd-xdwr9" Jul 06 19:18:16.764020 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:18:16.763996 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-574449bdd-xdwr9" Jul 06 19:18:16.850110 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:18:16.850054 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vc6dw\" (UniqueName: \"kubernetes.io/projected/1b1951f6-ea14-4376-aa45-0802aa661db9-kube-api-access-vc6dw\") pod \"1b1951f6-ea14-4376-aa45-0802aa661db9\" (UID: \"1b1951f6-ea14-4376-aa45-0802aa661db9\") " Jul 06 19:18:16.852346 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:18:16.852319 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b1951f6-ea14-4376-aa45-0802aa661db9-kube-api-access-vc6dw" (OuterVolumeSpecName: "kube-api-access-vc6dw") pod "1b1951f6-ea14-4376-aa45-0802aa661db9" (UID: "1b1951f6-ea14-4376-aa45-0802aa661db9"). InnerVolumeSpecName "kube-api-access-vc6dw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:18:16.950942 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:18:16.950901 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vc6dw\" (UniqueName: \"kubernetes.io/projected/1b1951f6-ea14-4376-aa45-0802aa661db9-kube-api-access-vc6dw\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:18:17.759737 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:18:17.759703 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-574449bdd-xdwr9" Jul 06 19:18:17.786195 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:18:17.786160 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-574449bdd-xdwr9"] Jul 06 19:18:17.791417 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:18:17.791385 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-574449bdd-xdwr9"] Jul 06 19:18:19.236724 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:18:19.236695 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1b1951f6-ea14-4376-aa45-0802aa661db9" path="/var/lib/kubelet/pods/1b1951f6-ea14-4376-aa45-0802aa661db9/volumes" Jul 06 19:18:20.532715 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:18:20.532665 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-tfdpx"] Jul 06 19:18:20.533246 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:18:20.532925 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-tfdpx" podUID="fb73cf0a-47dc-4258-be1f-1f922d1c3439" containerName="authorino" containerID="cri-o://47d2511acaaff2544ce93566791d00e03e22f148062cb3987146fbb03ab3b35b" gracePeriod=30 Jul 06 19:18:20.777805 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:18:20.777769 2566 generic.go:356] "Generic (PLEG): container finished" podID="fb73cf0a-47dc-4258-be1f-1f922d1c3439" containerID="47d2511acaaff2544ce93566791d00e03e22f148062cb3987146fbb03ab3b35b" exitCode=0 Jul 06 19:18:20.777937 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:18:20.777838 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-tfdpx" event={"ID":"fb73cf0a-47dc-4258-be1f-1f922d1c3439","Type":"ContainerDied","Data":"47d2511acaaff2544ce93566791d00e03e22f148062cb3987146fbb03ab3b35b"} Jul 06 19:18:20.797777 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:18:20.797752 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-tfdpx" Jul 06 19:18:20.987328 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:18:20.987288 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vh5g6\" (UniqueName: \"kubernetes.io/projected/fb73cf0a-47dc-4258-be1f-1f922d1c3439-kube-api-access-vh5g6\") pod \"fb73cf0a-47dc-4258-be1f-1f922d1c3439\" (UID: \"fb73cf0a-47dc-4258-be1f-1f922d1c3439\") " Jul 06 19:18:20.989523 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:18:20.989494 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb73cf0a-47dc-4258-be1f-1f922d1c3439-kube-api-access-vh5g6" (OuterVolumeSpecName: "kube-api-access-vh5g6") pod "fb73cf0a-47dc-4258-be1f-1f922d1c3439" (UID: "fb73cf0a-47dc-4258-be1f-1f922d1c3439"). InnerVolumeSpecName "kube-api-access-vh5g6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:18:21.088302 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:18:21.088217 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vh5g6\" (UniqueName: \"kubernetes.io/projected/fb73cf0a-47dc-4258-be1f-1f922d1c3439-kube-api-access-vh5g6\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:18:21.784908 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:18:21.784873 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-tfdpx" Jul 06 19:18:21.785398 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:18:21.784909 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-tfdpx" event={"ID":"fb73cf0a-47dc-4258-be1f-1f922d1c3439","Type":"ContainerDied","Data":"19fdb656f38101726e8fdc33255be96ffadf79485f1fc1a993a29bbdb75408a3"} Jul 06 19:18:21.785398 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:18:21.784967 2566 scope.go:117] "RemoveContainer" containerID="47d2511acaaff2544ce93566791d00e03e22f148062cb3987146fbb03ab3b35b" Jul 06 19:18:21.800865 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:18:21.800838 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-tfdpx"] Jul 06 19:18:21.803369 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:18:21.803350 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-tfdpx"] Jul 06 19:18:23.239438 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:18:23.239404 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fb73cf0a-47dc-4258-be1f-1f922d1c3439" path="/var/lib/kubelet/pods/fb73cf0a-47dc-4258-be1f-1f922d1c3439/volumes" Jul 06 19:18:24.810104 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:18:24.810052 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:18:24.823637 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:18:24.823598 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7874c5c4fc-82xrq"] Jul 06 19:18:24.823966 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:18:24.823954 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb73cf0a-47dc-4258-be1f-1f922d1c3439" containerName="authorino" Jul 06 19:18:24.824018 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:18:24.823969 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb73cf0a-47dc-4258-be1f-1f922d1c3439" containerName="authorino" Jul 06 19:18:24.824054 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:18:24.824034 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="fb73cf0a-47dc-4258-be1f-1f922d1c3439" containerName="authorino" Jul 06 19:18:24.827765 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:18:24.827739 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7874c5c4fc-82xrq" Jul 06 19:18:24.832729 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:18:24.832705 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 06 19:18:24.833160 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:18:24.833141 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 06 19:18:24.833732 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:18:24.833709 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-lm8jz\"" Jul 06 19:18:24.841338 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:18:24.841307 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7874c5c4fc-82xrq"] Jul 06 19:18:24.907996 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:18:24.907961 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:18:24.996026 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:18:24.995988 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:18:25.024378 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:18:25.024333 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rdrf5\" (UniqueName: \"kubernetes.io/projected/bd08b6c8-6edd-4dc9-8c67-8a06964aa6fe-kube-api-access-rdrf5\") pod \"maas-controller-7874c5c4fc-82xrq\" (UID: \"bd08b6c8-6edd-4dc9-8c67-8a06964aa6fe\") " pod="opendatahub/maas-controller-7874c5c4fc-82xrq" Jul 06 19:18:25.024598 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:18:25.024414 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bd08b6c8-6edd-4dc9-8c67-8a06964aa6fe-webhook-cert\") pod \"maas-controller-7874c5c4fc-82xrq\" (UID: \"bd08b6c8-6edd-4dc9-8c67-8a06964aa6fe\") " pod="opendatahub/maas-controller-7874c5c4fc-82xrq" Jul 06 19:18:25.125225 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:18:25.125126 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bd08b6c8-6edd-4dc9-8c67-8a06964aa6fe-webhook-cert\") pod \"maas-controller-7874c5c4fc-82xrq\" (UID: \"bd08b6c8-6edd-4dc9-8c67-8a06964aa6fe\") " pod="opendatahub/maas-controller-7874c5c4fc-82xrq" Jul 06 19:18:25.125225 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:18:25.125220 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rdrf5\" (UniqueName: \"kubernetes.io/projected/bd08b6c8-6edd-4dc9-8c67-8a06964aa6fe-kube-api-access-rdrf5\") pod \"maas-controller-7874c5c4fc-82xrq\" (UID: \"bd08b6c8-6edd-4dc9-8c67-8a06964aa6fe\") " pod="opendatahub/maas-controller-7874c5c4fc-82xrq" Jul 06 19:18:25.125416 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:18:25.125289 2566 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found Jul 06 19:18:25.125416 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:18:25.125364 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bd08b6c8-6edd-4dc9-8c67-8a06964aa6fe-webhook-cert podName:bd08b6c8-6edd-4dc9-8c67-8a06964aa6fe nodeName:}" failed. No retries permitted until 2026-07-06 19:18:25.625341978 +0000 UTC m=+530.932700915 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/bd08b6c8-6edd-4dc9-8c67-8a06964aa6fe-webhook-cert") pod "maas-controller-7874c5c4fc-82xrq" (UID: "bd08b6c8-6edd-4dc9-8c67-8a06964aa6fe") : secret "maas-controller-webhook-cert" not found Jul 06 19:18:25.137950 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:18:25.137917 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rdrf5\" (UniqueName: \"kubernetes.io/projected/bd08b6c8-6edd-4dc9-8c67-8a06964aa6fe-kube-api-access-rdrf5\") pod \"maas-controller-7874c5c4fc-82xrq\" (UID: \"bd08b6c8-6edd-4dc9-8c67-8a06964aa6fe\") " pod="opendatahub/maas-controller-7874c5c4fc-82xrq" Jul 06 19:18:25.175245 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:18:25.175177 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:18:25.628909 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:18:25.628863 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bd08b6c8-6edd-4dc9-8c67-8a06964aa6fe-webhook-cert\") pod \"maas-controller-7874c5c4fc-82xrq\" (UID: \"bd08b6c8-6edd-4dc9-8c67-8a06964aa6fe\") " pod="opendatahub/maas-controller-7874c5c4fc-82xrq" Jul 06 19:18:25.631587 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:18:25.631549 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bd08b6c8-6edd-4dc9-8c67-8a06964aa6fe-webhook-cert\") pod \"maas-controller-7874c5c4fc-82xrq\" (UID: \"bd08b6c8-6edd-4dc9-8c67-8a06964aa6fe\") " pod="opendatahub/maas-controller-7874c5c4fc-82xrq" Jul 06 19:18:25.739397 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:18:25.739347 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7874c5c4fc-82xrq" Jul 06 19:18:25.889325 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:18:25.889293 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7874c5c4fc-82xrq"] Jul 06 19:18:25.891577 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:18:25.891550 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd08b6c8_6edd_4dc9_8c67_8a06964aa6fe.slice/crio-2988b34eef9db42aca4a3056a29c413fc4b1f7e89f0e2a8705453c38c7bf1817 WatchSource:0}: Error finding container 2988b34eef9db42aca4a3056a29c413fc4b1f7e89f0e2a8705453c38c7bf1817: Status 404 returned error can't find the container with id 2988b34eef9db42aca4a3056a29c413fc4b1f7e89f0e2a8705453c38c7bf1817 Jul 06 19:18:26.820330 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:18:26.820292 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7874c5c4fc-82xrq" event={"ID":"bd08b6c8-6edd-4dc9-8c67-8a06964aa6fe","Type":"ContainerStarted","Data":"2988b34eef9db42aca4a3056a29c413fc4b1f7e89f0e2a8705453c38c7bf1817"} Jul 06 19:18:27.832860 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:18:27.832810 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7874c5c4fc-82xrq" event={"ID":"bd08b6c8-6edd-4dc9-8c67-8a06964aa6fe","Type":"ContainerStarted","Data":"2a386f03e500ea940f9e235f33c132e1fa56e0515ebf1d11482ff3bbf5a5def0"} Jul 06 19:18:27.833263 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:18:27.832904 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7874c5c4fc-82xrq" Jul 06 19:18:27.848678 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:18:27.848625 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7874c5c4fc-82xrq" podStartSLOduration=2.038488949 podStartE2EDuration="3.848608284s" totalImagesPullingTime="1.810119335s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:18:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:18:27.847658197 +0000 UTC m=+533.155017153" watchObservedRunningTime="2026-07-06 19:18:27.848608284 +0000 UTC m=+533.155967233" Jul 06 19:18:27.925285 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:18:27.925251 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:18:28.027542 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:18:28.027450 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:18:38.844639 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:18:38.844596 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7874c5c4fc-82xrq" Jul 06 19:18:39.585216 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:18:39.585177 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7874c5c4fc-82xrq"] Jul 06 19:18:39.585440 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:18:39.585419 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-7874c5c4fc-82xrq" podUID="bd08b6c8-6edd-4dc9-8c67-8a06964aa6fe" containerName="manager" containerID="cri-o://2a386f03e500ea940f9e235f33c132e1fa56e0515ebf1d11482ff3bbf5a5def0" gracePeriod=30 Jul 06 19:18:39.843332 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:18:39.843258 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7874c5c4fc-82xrq" Jul 06 19:18:39.905242 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:18:39.905210 2566 generic.go:356] "Generic (PLEG): container finished" podID="bd08b6c8-6edd-4dc9-8c67-8a06964aa6fe" containerID="2a386f03e500ea940f9e235f33c132e1fa56e0515ebf1d11482ff3bbf5a5def0" exitCode=0 Jul 06 19:18:39.905685 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:18:39.905257 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7874c5c4fc-82xrq" Jul 06 19:18:39.905685 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:18:39.905302 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7874c5c4fc-82xrq" event={"ID":"bd08b6c8-6edd-4dc9-8c67-8a06964aa6fe","Type":"ContainerDied","Data":"2a386f03e500ea940f9e235f33c132e1fa56e0515ebf1d11482ff3bbf5a5def0"} Jul 06 19:18:39.905685 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:18:39.905372 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7874c5c4fc-82xrq" event={"ID":"bd08b6c8-6edd-4dc9-8c67-8a06964aa6fe","Type":"ContainerDied","Data":"2988b34eef9db42aca4a3056a29c413fc4b1f7e89f0e2a8705453c38c7bf1817"} Jul 06 19:18:39.905685 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:18:39.905407 2566 scope.go:117] "RemoveContainer" containerID="2a386f03e500ea940f9e235f33c132e1fa56e0515ebf1d11482ff3bbf5a5def0" Jul 06 19:18:39.914577 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:18:39.914553 2566 scope.go:117] "RemoveContainer" containerID="2a386f03e500ea940f9e235f33c132e1fa56e0515ebf1d11482ff3bbf5a5def0" Jul 06 19:18:39.914927 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:18:39.914903 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2a386f03e500ea940f9e235f33c132e1fa56e0515ebf1d11482ff3bbf5a5def0\": container with ID starting with 2a386f03e500ea940f9e235f33c132e1fa56e0515ebf1d11482ff3bbf5a5def0 not found: ID does not exist" containerID="2a386f03e500ea940f9e235f33c132e1fa56e0515ebf1d11482ff3bbf5a5def0" Jul 06 19:18:39.914978 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:18:39.914938 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a386f03e500ea940f9e235f33c132e1fa56e0515ebf1d11482ff3bbf5a5def0"} err="failed to get container status \"2a386f03e500ea940f9e235f33c132e1fa56e0515ebf1d11482ff3bbf5a5def0\": rpc error: code = NotFound desc = could not find container \"2a386f03e500ea940f9e235f33c132e1fa56e0515ebf1d11482ff3bbf5a5def0\": container with ID starting with 2a386f03e500ea940f9e235f33c132e1fa56e0515ebf1d11482ff3bbf5a5def0 not found: ID does not exist" Jul 06 19:18:39.937342 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:18:39.937299 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rdrf5\" (UniqueName: \"kubernetes.io/projected/bd08b6c8-6edd-4dc9-8c67-8a06964aa6fe-kube-api-access-rdrf5\") pod \"bd08b6c8-6edd-4dc9-8c67-8a06964aa6fe\" (UID: \"bd08b6c8-6edd-4dc9-8c67-8a06964aa6fe\") " Jul 06 19:18:39.937342 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:18:39.937348 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bd08b6c8-6edd-4dc9-8c67-8a06964aa6fe-webhook-cert\") pod \"bd08b6c8-6edd-4dc9-8c67-8a06964aa6fe\" (UID: \"bd08b6c8-6edd-4dc9-8c67-8a06964aa6fe\") " Jul 06 19:18:39.939729 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:18:39.939695 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd08b6c8-6edd-4dc9-8c67-8a06964aa6fe-kube-api-access-rdrf5" (OuterVolumeSpecName: "kube-api-access-rdrf5") pod "bd08b6c8-6edd-4dc9-8c67-8a06964aa6fe" (UID: "bd08b6c8-6edd-4dc9-8c67-8a06964aa6fe"). InnerVolumeSpecName "kube-api-access-rdrf5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:18:39.939729 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:18:39.939705 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bd08b6c8-6edd-4dc9-8c67-8a06964aa6fe-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "bd08b6c8-6edd-4dc9-8c67-8a06964aa6fe" (UID: "bd08b6c8-6edd-4dc9-8c67-8a06964aa6fe"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:18:40.038579 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:18:40.038534 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rdrf5\" (UniqueName: \"kubernetes.io/projected/bd08b6c8-6edd-4dc9-8c67-8a06964aa6fe-kube-api-access-rdrf5\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:18:40.038579 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:18:40.038567 2566 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bd08b6c8-6edd-4dc9-8c67-8a06964aa6fe-webhook-cert\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:18:40.228305 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:18:40.228273 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7874c5c4fc-82xrq"] Jul 06 19:18:40.231675 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:18:40.231647 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7874c5c4fc-82xrq"] Jul 06 19:18:41.236845 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:18:41.236811 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bd08b6c8-6edd-4dc9-8c67-8a06964aa6fe" path="/var/lib/kubelet/pods/bd08b6c8-6edd-4dc9-8c67-8a06964aa6fe/volumes" Jul 06 19:18:57.364459 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:18:57.364419 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:18:57.773035 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:18:57.772947 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:18:57.840578 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:18:57.840541 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-5d68c7bd7c-4ltxv"] Jul 06 19:18:57.841129 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:18:57.841110 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd08b6c8-6edd-4dc9-8c67-8a06964aa6fe" containerName="manager" Jul 06 19:18:57.841129 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:18:57.841129 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd08b6c8-6edd-4dc9-8c67-8a06964aa6fe" containerName="manager" Jul 06 19:18:57.841296 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:18:57.841222 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd08b6c8-6edd-4dc9-8c67-8a06964aa6fe" containerName="manager" Jul 06 19:18:57.844525 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:18:57.844506 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-5d68c7bd7c-4ltxv" Jul 06 19:18:57.846607 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:18:57.846578 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-serving-cert\"" Jul 06 19:18:57.846719 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:18:57.846610 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-kjghq\"" Jul 06 19:18:57.853915 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:18:57.853857 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-5d68c7bd7c-4ltxv"] Jul 06 19:18:57.892255 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:18:57.892217 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f6e6c625-f78a-4051-bf1f-10953d59dba1-openshift-service-ca\") pod \"maas-api-5d68c7bd7c-4ltxv\" (UID: \"f6e6c625-f78a-4051-bf1f-10953d59dba1\") " pod="opendatahub/maas-api-5d68c7bd7c-4ltxv" Jul 06 19:18:57.892444 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:18:57.892343 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2shxh\" (UniqueName: \"kubernetes.io/projected/f6e6c625-f78a-4051-bf1f-10953d59dba1-kube-api-access-2shxh\") pod \"maas-api-5d68c7bd7c-4ltxv\" (UID: \"f6e6c625-f78a-4051-bf1f-10953d59dba1\") " pod="opendatahub/maas-api-5d68c7bd7c-4ltxv" Jul 06 19:18:57.892444 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:18:57.892409 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f6e6c625-f78a-4051-bf1f-10953d59dba1-maas-api-tls\") pod \"maas-api-5d68c7bd7c-4ltxv\" (UID: \"f6e6c625-f78a-4051-bf1f-10953d59dba1\") " pod="opendatahub/maas-api-5d68c7bd7c-4ltxv" Jul 06 19:18:57.936389 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:18:57.936357 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:18:57.993718 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:18:57.993673 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2shxh\" (UniqueName: \"kubernetes.io/projected/f6e6c625-f78a-4051-bf1f-10953d59dba1-kube-api-access-2shxh\") pod \"maas-api-5d68c7bd7c-4ltxv\" (UID: \"f6e6c625-f78a-4051-bf1f-10953d59dba1\") " pod="opendatahub/maas-api-5d68c7bd7c-4ltxv" Jul 06 19:18:57.993932 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:18:57.993745 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f6e6c625-f78a-4051-bf1f-10953d59dba1-maas-api-tls\") pod \"maas-api-5d68c7bd7c-4ltxv\" (UID: \"f6e6c625-f78a-4051-bf1f-10953d59dba1\") " pod="opendatahub/maas-api-5d68c7bd7c-4ltxv" Jul 06 19:18:57.993932 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:18:57.993775 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f6e6c625-f78a-4051-bf1f-10953d59dba1-openshift-service-ca\") pod \"maas-api-5d68c7bd7c-4ltxv\" (UID: \"f6e6c625-f78a-4051-bf1f-10953d59dba1\") " pod="opendatahub/maas-api-5d68c7bd7c-4ltxv" Jul 06 19:18:57.994560 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:18:57.994529 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f6e6c625-f78a-4051-bf1f-10953d59dba1-openshift-service-ca\") pod \"maas-api-5d68c7bd7c-4ltxv\" (UID: \"f6e6c625-f78a-4051-bf1f-10953d59dba1\") " pod="opendatahub/maas-api-5d68c7bd7c-4ltxv" Jul 06 19:18:57.997135 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:18:57.997088 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f6e6c625-f78a-4051-bf1f-10953d59dba1-maas-api-tls\") pod \"maas-api-5d68c7bd7c-4ltxv\" (UID: \"f6e6c625-f78a-4051-bf1f-10953d59dba1\") " pod="opendatahub/maas-api-5d68c7bd7c-4ltxv" Jul 06 19:18:58.002279 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:18:58.002255 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2shxh\" (UniqueName: \"kubernetes.io/projected/f6e6c625-f78a-4051-bf1f-10953d59dba1-kube-api-access-2shxh\") pod \"maas-api-5d68c7bd7c-4ltxv\" (UID: \"f6e6c625-f78a-4051-bf1f-10953d59dba1\") " pod="opendatahub/maas-api-5d68c7bd7c-4ltxv" Jul 06 19:18:58.156175 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:18:58.156140 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-5d68c7bd7c-4ltxv" Jul 06 19:18:58.299237 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:18:58.299210 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-5d68c7bd7c-4ltxv"] Jul 06 19:18:58.301462 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:18:58.301433 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6e6c625_f78a_4051_bf1f_10953d59dba1.slice/crio-f5aa4ea26d7292defdbe672d903ece21fd58633ca2d148bdef3e53b6f5f4eec0 WatchSource:0}: Error finding container f5aa4ea26d7292defdbe672d903ece21fd58633ca2d148bdef3e53b6f5f4eec0: Status 404 returned error can't find the container with id f5aa4ea26d7292defdbe672d903ece21fd58633ca2d148bdef3e53b6f5f4eec0 Jul 06 19:18:58.302812 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:18:58.302792 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:18:58.402909 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:18:58.402876 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:18:58.434655 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:18:58.434569 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-vt6xf"] Jul 06 19:18:58.439213 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:18:58.439192 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-vt6xf" Jul 06 19:18:58.441422 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:18:58.441401 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 06 19:18:58.441540 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:18:58.441423 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-jnzld\"" Jul 06 19:18:58.451921 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:18:58.451892 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-vt6xf"] Jul 06 19:18:58.498180 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:18:58.498146 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1e8eaff3-cc54-4648-b0c6-cb184ba29ff8-plugins-config-volume\") pod \"payload-processing-58f86c7798-vt6xf\" (UID: \"1e8eaff3-cc54-4648-b0c6-cb184ba29ff8\") " pod="openshift-ingress/payload-processing-58f86c7798-vt6xf" Jul 06 19:18:58.498317 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:18:58.498209 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ljfpk\" (UniqueName: \"kubernetes.io/projected/1e8eaff3-cc54-4648-b0c6-cb184ba29ff8-kube-api-access-ljfpk\") pod \"payload-processing-58f86c7798-vt6xf\" (UID: \"1e8eaff3-cc54-4648-b0c6-cb184ba29ff8\") " pod="openshift-ingress/payload-processing-58f86c7798-vt6xf" Jul 06 19:18:58.599278 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:18:58.599246 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ljfpk\" (UniqueName: \"kubernetes.io/projected/1e8eaff3-cc54-4648-b0c6-cb184ba29ff8-kube-api-access-ljfpk\") pod \"payload-processing-58f86c7798-vt6xf\" (UID: \"1e8eaff3-cc54-4648-b0c6-cb184ba29ff8\") " pod="openshift-ingress/payload-processing-58f86c7798-vt6xf" Jul 06 19:18:58.599445 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:18:58.599311 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1e8eaff3-cc54-4648-b0c6-cb184ba29ff8-plugins-config-volume\") pod \"payload-processing-58f86c7798-vt6xf\" (UID: \"1e8eaff3-cc54-4648-b0c6-cb184ba29ff8\") " pod="openshift-ingress/payload-processing-58f86c7798-vt6xf" Jul 06 19:18:58.599955 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:18:58.599937 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1e8eaff3-cc54-4648-b0c6-cb184ba29ff8-plugins-config-volume\") pod \"payload-processing-58f86c7798-vt6xf\" (UID: \"1e8eaff3-cc54-4648-b0c6-cb184ba29ff8\") " pod="openshift-ingress/payload-processing-58f86c7798-vt6xf" Jul 06 19:18:58.607560 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:18:58.607537 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ljfpk\" (UniqueName: \"kubernetes.io/projected/1e8eaff3-cc54-4648-b0c6-cb184ba29ff8-kube-api-access-ljfpk\") pod \"payload-processing-58f86c7798-vt6xf\" (UID: \"1e8eaff3-cc54-4648-b0c6-cb184ba29ff8\") " pod="openshift-ingress/payload-processing-58f86c7798-vt6xf" Jul 06 19:18:58.750368 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:18:58.750282 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-vt6xf" Jul 06 19:18:58.896035 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:18:58.892448 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-vt6xf"] Jul 06 19:18:58.896678 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:18:58.896651 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e8eaff3_cc54_4648_b0c6_cb184ba29ff8.slice/crio-1920f6c6ee4dd75e060a2a3e280219cdd2fed0a340617ed8262ddeb7acb0f21a WatchSource:0}: Error finding container 1920f6c6ee4dd75e060a2a3e280219cdd2fed0a340617ed8262ddeb7acb0f21a: Status 404 returned error can't find the container with id 1920f6c6ee4dd75e060a2a3e280219cdd2fed0a340617ed8262ddeb7acb0f21a Jul 06 19:18:59.020839 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:18:59.020751 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-5d68c7bd7c-4ltxv" event={"ID":"f6e6c625-f78a-4051-bf1f-10953d59dba1","Type":"ContainerStarted","Data":"f5aa4ea26d7292defdbe672d903ece21fd58633ca2d148bdef3e53b6f5f4eec0"} Jul 06 19:18:59.022268 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:18:59.022235 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-vt6xf" event={"ID":"1e8eaff3-cc54-4648-b0c6-cb184ba29ff8","Type":"ContainerStarted","Data":"1920f6c6ee4dd75e060a2a3e280219cdd2fed0a340617ed8262ddeb7acb0f21a"} Jul 06 19:19:01.038913 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:19:01.038865 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-vt6xf" event={"ID":"1e8eaff3-cc54-4648-b0c6-cb184ba29ff8","Type":"ContainerStarted","Data":"a1e63aeb1851f971faf238a56540eaed3cfb63639291543df54adc9ea1e60c48"} Jul 06 19:19:01.039795 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:19:01.039767 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-58f86c7798-vt6xf" Jul 06 19:19:01.041645 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:19:01.041609 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-5d68c7bd7c-4ltxv" event={"ID":"f6e6c625-f78a-4051-bf1f-10953d59dba1","Type":"ContainerStarted","Data":"f682fcef81d3d6c60e7b3c156ffc252e11e0b1fc09fc144ed6eb5a7e1681b30b"} Jul 06 19:19:01.042232 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:19:01.042207 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-5d68c7bd7c-4ltxv" Jul 06 19:19:01.054849 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:19:01.054748 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-58f86c7798-vt6xf" podStartSLOduration=1.178995496 podStartE2EDuration="3.054726933s" totalImagesPullingTime="1.875731437s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:18:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:19:01.054242622 +0000 UTC m=+566.361601571" watchObservedRunningTime="2026-07-06 19:19:01.054726933 +0000 UTC m=+566.362085882" Jul 06 19:19:01.074287 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:19:01.074219 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-5d68c7bd7c-4ltxv" podStartSLOduration=1.604976017 podStartE2EDuration="4.074201133s" totalImagesPullingTime="2.469225116s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:18:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:19:01.071651793 +0000 UTC m=+566.379010743" watchObservedRunningTime="2026-07-06 19:19:01.074201133 +0000 UTC m=+566.381560082" Jul 06 19:19:07.082126 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:19:07.082089 2566 generic.go:356] "Generic (PLEG): container finished" podID="0af75ee7-3c24-4d81-9764-64b52e4f3623" containerID="9757c4a82fed1f284e65920af54fc83000d923ef5c019c6e1788f5ca2fb82a6f" exitCode=0 Jul 06 19:19:07.082533 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:19:07.082151 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t9fgz" event={"ID":"0af75ee7-3c24-4d81-9764-64b52e4f3623","Type":"ContainerDied","Data":"9757c4a82fed1f284e65920af54fc83000d923ef5c019c6e1788f5ca2fb82a6f"} Jul 06 19:19:07.082578 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:19:07.082557 2566 scope.go:117] "RemoveContainer" containerID="9757c4a82fed1f284e65920af54fc83000d923ef5c019c6e1788f5ca2fb82a6f" Jul 06 19:19:07.083464 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:19:07.083433 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 19:19:07.083554 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:19:07.083502 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 19:19:07.083554 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:19:07.083535 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 19:19:08.063585 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:19:08.063553 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-5d68c7bd7c-4ltxv" Jul 06 19:19:08.090821 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:19:08.090777 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t9fgz" event={"ID":"0af75ee7-3c24-4d81-9764-64b52e4f3623","Type":"ContainerStarted","Data":"5a057e3d8579152b69218ec5ba293442020982d7d3704d4ad08dba3f1d4b910a"} Jul 06 19:19:08.765969 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:19:08.765933 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t9fgz" Jul 06 19:19:08.766182 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:19:08.766149 2566 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-t9fgz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Jul 06 19:19:08.766236 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:19:08.766195 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t9fgz" podUID="0af75ee7-3c24-4d81-9764-64b52e4f3623" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Jul 06 19:19:09.017204 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:19:09.017126 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:19:09.119990 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:19:09.119954 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:19:09.766761 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:19:09.766724 2566 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-t9fgz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Jul 06 19:19:09.766994 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:19:09.766796 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t9fgz" podUID="0af75ee7-3c24-4d81-9764-64b52e4f3623" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Jul 06 19:19:10.766211 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:19:10.766176 2566 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-t9fgz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Jul 06 19:19:10.766587 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:19:10.766266 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t9fgz" podUID="0af75ee7-3c24-4d81-9764-64b52e4f3623" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Jul 06 19:19:11.767007 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:19:11.766968 2566 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-t9fgz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Jul 06 19:19:11.767433 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:19:11.767030 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t9fgz" podUID="0af75ee7-3c24-4d81-9764-64b52e4f3623" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Jul 06 19:19:12.769656 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:19:12.769622 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t9fgz" Jul 06 19:19:12.770038 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:19:12.769906 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t9fgz" Jul 06 19:19:12.770814 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:19:12.770793 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t9fgz" Jul 06 19:19:13.060742 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:19:13.060716 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-58f86c7798-vt6xf" Jul 06 19:19:14.476145 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:19:14.476110 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bqfjf"] Jul 06 19:19:14.478870 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:19:14.478850 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bqfjf" Jul 06 19:19:14.481420 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:19:14.481392 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 06 19:19:14.481543 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:19:14.481392 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-mwzf5\"" Jul 06 19:19:14.482176 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:19:14.482154 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jul 06 19:19:14.482268 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:19:14.482201 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 06 19:19:14.490086 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:19:14.490042 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bqfjf"] Jul 06 19:19:14.516271 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:19:14.516243 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:19:14.649917 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:19:14.649881 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8eb3f370-ef60-4437-884e-967f9f8b27f9-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bqfjf\" (UID: \"8eb3f370-ef60-4437-884e-967f9f8b27f9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bqfjf" Jul 06 19:19:14.650135 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:19:14.649993 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8eb3f370-ef60-4437-884e-967f9f8b27f9-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bqfjf\" (UID: \"8eb3f370-ef60-4437-884e-967f9f8b27f9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bqfjf" Jul 06 19:19:14.650135 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:19:14.650019 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8eb3f370-ef60-4437-884e-967f9f8b27f9-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bqfjf\" (UID: \"8eb3f370-ef60-4437-884e-967f9f8b27f9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bqfjf" Jul 06 19:19:14.650135 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:19:14.650039 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8eb3f370-ef60-4437-884e-967f9f8b27f9-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bqfjf\" (UID: \"8eb3f370-ef60-4437-884e-967f9f8b27f9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bqfjf" Jul 06 19:19:14.650135 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:19:14.650109 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8eb3f370-ef60-4437-884e-967f9f8b27f9-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bqfjf\" (UID: \"8eb3f370-ef60-4437-884e-967f9f8b27f9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bqfjf" Jul 06 19:19:14.650306 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:19:14.650145 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nwqwt\" (UniqueName: \"kubernetes.io/projected/8eb3f370-ef60-4437-884e-967f9f8b27f9-kube-api-access-nwqwt\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bqfjf\" (UID: \"8eb3f370-ef60-4437-884e-967f9f8b27f9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bqfjf" Jul 06 19:19:14.751344 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:19:14.751242 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8eb3f370-ef60-4437-884e-967f9f8b27f9-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bqfjf\" (UID: \"8eb3f370-ef60-4437-884e-967f9f8b27f9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bqfjf" Jul 06 19:19:14.751344 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:19:14.751285 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8eb3f370-ef60-4437-884e-967f9f8b27f9-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bqfjf\" (UID: \"8eb3f370-ef60-4437-884e-967f9f8b27f9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bqfjf" Jul 06 19:19:14.751344 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:19:14.751307 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8eb3f370-ef60-4437-884e-967f9f8b27f9-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bqfjf\" (UID: \"8eb3f370-ef60-4437-884e-967f9f8b27f9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bqfjf" Jul 06 19:19:14.751344 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:19:14.751343 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8eb3f370-ef60-4437-884e-967f9f8b27f9-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bqfjf\" (UID: \"8eb3f370-ef60-4437-884e-967f9f8b27f9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bqfjf" Jul 06 19:19:14.751675 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:19:14.751377 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nwqwt\" (UniqueName: \"kubernetes.io/projected/8eb3f370-ef60-4437-884e-967f9f8b27f9-kube-api-access-nwqwt\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bqfjf\" (UID: \"8eb3f370-ef60-4437-884e-967f9f8b27f9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bqfjf" Jul 06 19:19:14.751675 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:19:14.751449 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8eb3f370-ef60-4437-884e-967f9f8b27f9-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bqfjf\" (UID: \"8eb3f370-ef60-4437-884e-967f9f8b27f9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bqfjf" Jul 06 19:19:14.751815 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:19:14.751787 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8eb3f370-ef60-4437-884e-967f9f8b27f9-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bqfjf\" (UID: \"8eb3f370-ef60-4437-884e-967f9f8b27f9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bqfjf" Jul 06 19:19:14.751872 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:19:14.751822 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8eb3f370-ef60-4437-884e-967f9f8b27f9-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bqfjf\" (UID: \"8eb3f370-ef60-4437-884e-967f9f8b27f9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bqfjf" Jul 06 19:19:14.751872 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:19:14.751857 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8eb3f370-ef60-4437-884e-967f9f8b27f9-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bqfjf\" (UID: \"8eb3f370-ef60-4437-884e-967f9f8b27f9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bqfjf" Jul 06 19:19:14.753854 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:19:14.753836 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8eb3f370-ef60-4437-884e-967f9f8b27f9-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bqfjf\" (UID: \"8eb3f370-ef60-4437-884e-967f9f8b27f9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bqfjf" Jul 06 19:19:14.753929 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:19:14.753914 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8eb3f370-ef60-4437-884e-967f9f8b27f9-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bqfjf\" (UID: \"8eb3f370-ef60-4437-884e-967f9f8b27f9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bqfjf" Jul 06 19:19:14.760289 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:19:14.760269 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nwqwt\" (UniqueName: \"kubernetes.io/projected/8eb3f370-ef60-4437-884e-967f9f8b27f9-kube-api-access-nwqwt\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bqfjf\" (UID: \"8eb3f370-ef60-4437-884e-967f9f8b27f9\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bqfjf" Jul 06 19:19:14.789910 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:19:14.789876 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bqfjf" Jul 06 19:19:14.929822 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:19:14.929793 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bqfjf"] Jul 06 19:19:14.936263 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:19:14.931945 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8eb3f370_ef60_4437_884e_967f9f8b27f9.slice/crio-4c70a6021a8b422568526053e15aa03043f422939b2aea61c6966890acdad30c WatchSource:0}: Error finding container 4c70a6021a8b422568526053e15aa03043f422939b2aea61c6966890acdad30c: Status 404 returned error can't find the container with id 4c70a6021a8b422568526053e15aa03043f422939b2aea61c6966890acdad30c Jul 06 19:19:15.131761 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:19:15.131725 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bqfjf" event={"ID":"8eb3f370-ef60-4437-884e-967f9f8b27f9","Type":"ContainerStarted","Data":"4c70a6021a8b422568526053e15aa03043f422939b2aea61c6966890acdad30c"} Jul 06 19:19:16.140132 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:19:16.140094 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bqfjf" event={"ID":"8eb3f370-ef60-4437-884e-967f9f8b27f9","Type":"ContainerStarted","Data":"bcb5b4cea86f0075242480d2c96fe5850abf4d26e84aa9ef767fb040f4704fd1"} Jul 06 19:19:16.159206 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:19:16.159142 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bqfjf" podStartSLOduration=1.124517402 podStartE2EDuration="2.159124232s" totalImagesPullingTime="1.03460683s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:19:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:19:16.156872314 +0000 UTC m=+581.464231267" watchObservedRunningTime="2026-07-06 19:19:16.159124232 +0000 UTC m=+581.466483542" Jul 06 19:19:23.201256 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:19:23.201219 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:19:24.790925 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:19:24.790894 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bqfjf" Jul 06 19:19:24.791328 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:19:24.791044 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bqfjf" Jul 06 19:19:24.803509 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:19:24.803482 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bqfjf" Jul 06 19:19:25.202120 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:19:25.202026 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bqfjf" Jul 06 19:19:36.933026 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:19:36.932991 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:19:49.293462 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:19:49.293425 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:19:57.259051 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:19:57.258929 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:20:07.104292 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:20:07.104250 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:20:09.642889 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:20:09.642836 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:20:09.696695 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:20:09.696659 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:20:33.688610 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:20:33.688560 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-sl787"] Jul 06 19:20:33.691280 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:20:33.691259 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-sl787" Jul 06 19:20:33.693425 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:20:33.693406 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jul 06 19:20:33.704144 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:20:33.704122 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-sl787"] Jul 06 19:20:33.722717 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:20:33.722532 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:20:33.741736 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:20:33.741701 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7f7af955-921e-4eb0-99f5-60af3cc477ec-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-sl787\" (UID: \"7f7af955-921e-4eb0-99f5-60af3cc477ec\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-sl787" Jul 06 19:20:33.741887 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:20:33.741748 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dcpf4\" (UniqueName: \"kubernetes.io/projected/7f7af955-921e-4eb0-99f5-60af3cc477ec-kube-api-access-dcpf4\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-sl787\" (UID: \"7f7af955-921e-4eb0-99f5-60af3cc477ec\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-sl787" Jul 06 19:20:33.741887 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:20:33.741777 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7f7af955-921e-4eb0-99f5-60af3cc477ec-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-sl787\" (UID: \"7f7af955-921e-4eb0-99f5-60af3cc477ec\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-sl787" Jul 06 19:20:33.741887 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:20:33.741799 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7f7af955-921e-4eb0-99f5-60af3cc477ec-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-sl787\" (UID: \"7f7af955-921e-4eb0-99f5-60af3cc477ec\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-sl787" Jul 06 19:20:33.742021 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:20:33.741896 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7f7af955-921e-4eb0-99f5-60af3cc477ec-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-sl787\" (UID: \"7f7af955-921e-4eb0-99f5-60af3cc477ec\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-sl787" Jul 06 19:20:33.742021 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:20:33.741941 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7f7af955-921e-4eb0-99f5-60af3cc477ec-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-sl787\" (UID: \"7f7af955-921e-4eb0-99f5-60af3cc477ec\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-sl787" Jul 06 19:20:33.843399 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:20:33.843359 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7f7af955-921e-4eb0-99f5-60af3cc477ec-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-sl787\" (UID: \"7f7af955-921e-4eb0-99f5-60af3cc477ec\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-sl787" Jul 06 19:20:33.843399 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:20:33.843406 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dcpf4\" (UniqueName: \"kubernetes.io/projected/7f7af955-921e-4eb0-99f5-60af3cc477ec-kube-api-access-dcpf4\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-sl787\" (UID: \"7f7af955-921e-4eb0-99f5-60af3cc477ec\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-sl787" Jul 06 19:20:33.843399 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:20:33.843424 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7f7af955-921e-4eb0-99f5-60af3cc477ec-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-sl787\" (UID: \"7f7af955-921e-4eb0-99f5-60af3cc477ec\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-sl787" Jul 06 19:20:33.843399 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:20:33.843441 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7f7af955-921e-4eb0-99f5-60af3cc477ec-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-sl787\" (UID: \"7f7af955-921e-4eb0-99f5-60af3cc477ec\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-sl787" Jul 06 19:20:33.843399 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:20:33.843473 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7f7af955-921e-4eb0-99f5-60af3cc477ec-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-sl787\" (UID: \"7f7af955-921e-4eb0-99f5-60af3cc477ec\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-sl787" Jul 06 19:20:33.845059 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:20:33.843514 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7f7af955-921e-4eb0-99f5-60af3cc477ec-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-sl787\" (UID: \"7f7af955-921e-4eb0-99f5-60af3cc477ec\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-sl787" Jul 06 19:20:33.845059 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:20:33.843823 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7f7af955-921e-4eb0-99f5-60af3cc477ec-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-sl787\" (UID: \"7f7af955-921e-4eb0-99f5-60af3cc477ec\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-sl787" Jul 06 19:20:33.845059 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:20:33.843906 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7f7af955-921e-4eb0-99f5-60af3cc477ec-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-sl787\" (UID: \"7f7af955-921e-4eb0-99f5-60af3cc477ec\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-sl787" Jul 06 19:20:33.845059 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:20:33.844008 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7f7af955-921e-4eb0-99f5-60af3cc477ec-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-sl787\" (UID: \"7f7af955-921e-4eb0-99f5-60af3cc477ec\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-sl787" Jul 06 19:20:33.846368 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:20:33.846341 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7f7af955-921e-4eb0-99f5-60af3cc477ec-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-sl787\" (UID: \"7f7af955-921e-4eb0-99f5-60af3cc477ec\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-sl787" Jul 06 19:20:33.846527 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:20:33.846507 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7f7af955-921e-4eb0-99f5-60af3cc477ec-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-sl787\" (UID: \"7f7af955-921e-4eb0-99f5-60af3cc477ec\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-sl787" Jul 06 19:20:33.852654 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:20:33.852623 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dcpf4\" (UniqueName: \"kubernetes.io/projected/7f7af955-921e-4eb0-99f5-60af3cc477ec-kube-api-access-dcpf4\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-sl787\" (UID: \"7f7af955-921e-4eb0-99f5-60af3cc477ec\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-sl787" Jul 06 19:20:34.001248 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:20:34.001155 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-sl787" Jul 06 19:20:34.138912 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:20:34.138882 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-sl787"] Jul 06 19:20:34.140086 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:20:34.140047 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7f7af955_921e_4eb0_99f5_60af3cc477ec.slice/crio-062502bbba2fa10355ac52ff6c94a60d5a86b942836fb7aa064b786b43b71710 WatchSource:0}: Error finding container 062502bbba2fa10355ac52ff6c94a60d5a86b942836fb7aa064b786b43b71710: Status 404 returned error can't find the container with id 062502bbba2fa10355ac52ff6c94a60d5a86b942836fb7aa064b786b43b71710 Jul 06 19:20:34.141438 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:20:34.141424 2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 19:20:34.571857 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:20:34.571822 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-sl787" event={"ID":"7f7af955-921e-4eb0-99f5-60af3cc477ec","Type":"ContainerStarted","Data":"364d941d5ec729b13aaf3a8a5918a7139c88df1fb4e26aba54628349487c0afc"} Jul 06 19:20:34.572063 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:20:34.571867 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-sl787" event={"ID":"7f7af955-921e-4eb0-99f5-60af3cc477ec","Type":"ContainerStarted","Data":"062502bbba2fa10355ac52ff6c94a60d5a86b942836fb7aa064b786b43b71710"} Jul 06 19:20:34.595751 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:20:34.589950 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-sl787" podStartSLOduration=1.384150108 podStartE2EDuration="1.589931933s" totalImagesPullingTime="205.781825ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:20:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:20:34.587673825 +0000 UTC m=+659.895032773" watchObservedRunningTime="2026-07-06 19:20:34.589931933 +0000 UTC m=+659.897290883" Jul 06 19:20:44.002213 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:20:44.002167 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-sl787" Jul 06 19:20:44.002853 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:20:44.002231 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-sl787" Jul 06 19:20:44.014485 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:20:44.014449 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-sl787" Jul 06 19:20:44.635795 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:20:44.635761 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-sl787" Jul 06 19:21:20.227995 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:21:20.227936 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:21:25.354490 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:21:25.354370 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:21:25.452521 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:21:25.452492 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:21:26.589385 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:21:26.589356 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:21:26.792438 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:21:26.792407 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:21:31.470385 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:21:31.470352 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:21:38.180412 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:21:38.180348 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:21:42.646447 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:21:42.646416 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:21:42.747582 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:21:42.747541 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:22:15.056316 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:22:15.056285 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:22:15.173841 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:22:15.173787 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:22:17.198514 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:22:17.198477 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:22:17.299297 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:22:17.299264 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:22:37.817378 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:22:37.817345 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:22:39.293645 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:22:39.293614 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:23:20.011811 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:23:20.011772 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:23:20.103224 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:23:20.103190 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:23:20.284543 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:23:20.284515 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:23:26.022308 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:23:26.022278 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:23:26.132634 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:23:26.132601 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:23:26.306801 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:23:26.306765 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:23:26.391984 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:23:26.391956 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:23:32.738398 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:23:32.738332 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:23:32.838220 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:23:32.838141 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:25:09.848906 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:25:09.848865 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:27:26.614378 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:27:26.614280 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rvpvz"] Jul 06 19:27:26.614925 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:27:26.614596 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rvpvz" podUID="62641703-00a9-4b2e-9d0d-d4a5a1540cfd" containerName="manager" containerID="cri-o://7453e9d11490e652b5504b1b6b4572595ce835f3f182f4ab364a7dedc0e52050" gracePeriod=10 Jul 06 19:27:29.068502 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:27:29.068472 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rvpvz" Jul 06 19:27:29.239437 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:27:29.239352 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/62641703-00a9-4b2e-9d0d-d4a5a1540cfd-extensions-socket-volume\") pod \"62641703-00a9-4b2e-9d0d-d4a5a1540cfd\" (UID: \"62641703-00a9-4b2e-9d0d-d4a5a1540cfd\") " Jul 06 19:27:29.239437 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:27:29.239407 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6wl84\" (UniqueName: \"kubernetes.io/projected/62641703-00a9-4b2e-9d0d-d4a5a1540cfd-kube-api-access-6wl84\") pod \"62641703-00a9-4b2e-9d0d-d4a5a1540cfd\" (UID: \"62641703-00a9-4b2e-9d0d-d4a5a1540cfd\") " Jul 06 19:27:29.239758 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:27:29.239732 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/62641703-00a9-4b2e-9d0d-d4a5a1540cfd-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "62641703-00a9-4b2e-9d0d-d4a5a1540cfd" (UID: "62641703-00a9-4b2e-9d0d-d4a5a1540cfd"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:27:29.241529 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:27:29.241506 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/62641703-00a9-4b2e-9d0d-d4a5a1540cfd-kube-api-access-6wl84" (OuterVolumeSpecName: "kube-api-access-6wl84") pod "62641703-00a9-4b2e-9d0d-d4a5a1540cfd" (UID: "62641703-00a9-4b2e-9d0d-d4a5a1540cfd"). InnerVolumeSpecName "kube-api-access-6wl84". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:27:29.340924 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:27:29.340889 2566 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/62641703-00a9-4b2e-9d0d-d4a5a1540cfd-extensions-socket-volume\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:27:29.340924 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:27:29.340919 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6wl84\" (UniqueName: \"kubernetes.io/projected/62641703-00a9-4b2e-9d0d-d4a5a1540cfd-kube-api-access-6wl84\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:27:29.870397 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:27:29.870357 2566 generic.go:356] "Generic (PLEG): container finished" podID="62641703-00a9-4b2e-9d0d-d4a5a1540cfd" containerID="7453e9d11490e652b5504b1b6b4572595ce835f3f182f4ab364a7dedc0e52050" exitCode=0 Jul 06 19:27:29.870569 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:27:29.870413 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rvpvz" Jul 06 19:27:29.870569 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:27:29.870412 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rvpvz" event={"ID":"62641703-00a9-4b2e-9d0d-d4a5a1540cfd","Type":"ContainerDied","Data":"7453e9d11490e652b5504b1b6b4572595ce835f3f182f4ab364a7dedc0e52050"} Jul 06 19:27:29.870569 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:27:29.870533 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rvpvz" event={"ID":"62641703-00a9-4b2e-9d0d-d4a5a1540cfd","Type":"ContainerDied","Data":"607b0de5fefee39121c9fb10bf0a703076daa0a1072011e053d09977bdc3c299"} Jul 06 19:27:29.870569 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:27:29.870562 2566 scope.go:117] "RemoveContainer" containerID="7453e9d11490e652b5504b1b6b4572595ce835f3f182f4ab364a7dedc0e52050" Jul 06 19:27:29.884422 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:27:29.884398 2566 scope.go:117] "RemoveContainer" containerID="7453e9d11490e652b5504b1b6b4572595ce835f3f182f4ab364a7dedc0e52050" Jul 06 19:27:29.884743 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:27:29.884724 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7453e9d11490e652b5504b1b6b4572595ce835f3f182f4ab364a7dedc0e52050\": container with ID starting with 7453e9d11490e652b5504b1b6b4572595ce835f3f182f4ab364a7dedc0e52050 not found: ID does not exist" containerID="7453e9d11490e652b5504b1b6b4572595ce835f3f182f4ab364a7dedc0e52050" Jul 06 19:27:29.884797 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:27:29.884752 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7453e9d11490e652b5504b1b6b4572595ce835f3f182f4ab364a7dedc0e52050"} err="failed to get container status \"7453e9d11490e652b5504b1b6b4572595ce835f3f182f4ab364a7dedc0e52050\": rpc error: code = NotFound desc = could not find container \"7453e9d11490e652b5504b1b6b4572595ce835f3f182f4ab364a7dedc0e52050\": container with ID starting with 7453e9d11490e652b5504b1b6b4572595ce835f3f182f4ab364a7dedc0e52050 not found: ID does not exist" Jul 06 19:27:29.895597 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:27:29.895572 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rvpvz"] Jul 06 19:27:29.902998 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:27:29.902975 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-rvpvz"] Jul 06 19:27:31.237082 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:27:31.237047 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="62641703-00a9-4b2e-9d0d-d4a5a1540cfd" path="/var/lib/kubelet/pods/62641703-00a9-4b2e-9d0d-d4a5a1540cfd/volumes" Jul 06 19:28:06.938978 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:28:06.938941 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:28:27.767316 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:28:27.767277 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-f78vh"] Jul 06 19:28:27.767747 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:28:27.767652 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="62641703-00a9-4b2e-9d0d-d4a5a1540cfd" containerName="manager" Jul 06 19:28:27.767747 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:28:27.767662 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="62641703-00a9-4b2e-9d0d-d4a5a1540cfd" containerName="manager" Jul 06 19:28:27.767747 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:28:27.767741 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="62641703-00a9-4b2e-9d0d-d4a5a1540cfd" containerName="manager" Jul 06 19:28:27.770861 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:28:27.770844 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-f78vh" Jul 06 19:28:27.773245 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:28:27.773222 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-4n949\"" Jul 06 19:28:27.782775 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:28:27.782752 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-f78vh"] Jul 06 19:28:27.846400 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:28:27.846374 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7kvhs\" (UniqueName: \"kubernetes.io/projected/b129a63b-6e6f-4ca8-bd52-05d8638059fd-kube-api-access-7kvhs\") pod \"kuadrant-operator-controller-manager-86878c769f-f78vh\" (UID: \"b129a63b-6e6f-4ca8-bd52-05d8638059fd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-f78vh" Jul 06 19:28:27.846517 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:28:27.846406 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b129a63b-6e6f-4ca8-bd52-05d8638059fd-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-f78vh\" (UID: \"b129a63b-6e6f-4ca8-bd52-05d8638059fd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-f78vh" Jul 06 19:28:27.947682 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:28:27.947660 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7kvhs\" (UniqueName: \"kubernetes.io/projected/b129a63b-6e6f-4ca8-bd52-05d8638059fd-kube-api-access-7kvhs\") pod \"kuadrant-operator-controller-manager-86878c769f-f78vh\" (UID: \"b129a63b-6e6f-4ca8-bd52-05d8638059fd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-f78vh" Jul 06 19:28:27.947802 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:28:27.947688 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b129a63b-6e6f-4ca8-bd52-05d8638059fd-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-f78vh\" (UID: \"b129a63b-6e6f-4ca8-bd52-05d8638059fd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-f78vh" Jul 06 19:28:27.947983 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:28:27.947968 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b129a63b-6e6f-4ca8-bd52-05d8638059fd-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-f78vh\" (UID: \"b129a63b-6e6f-4ca8-bd52-05d8638059fd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-f78vh" Jul 06 19:28:27.958367 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:28:27.958335 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7kvhs\" (UniqueName: \"kubernetes.io/projected/b129a63b-6e6f-4ca8-bd52-05d8638059fd-kube-api-access-7kvhs\") pod \"kuadrant-operator-controller-manager-86878c769f-f78vh\" (UID: \"b129a63b-6e6f-4ca8-bd52-05d8638059fd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-f78vh" Jul 06 19:28:28.081498 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:28:28.081468 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-f78vh" Jul 06 19:28:28.217155 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:28:28.217128 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-f78vh"] Jul 06 19:28:29.202797 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:28:29.202765 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-f78vh" event={"ID":"b129a63b-6e6f-4ca8-bd52-05d8638059fd","Type":"ContainerStarted","Data":"4d2c1a2f9115f21ab88caa8ca2e33dc51055afd5344fdbab807fbb4a7405defe"} Jul 06 19:28:29.203186 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:28:29.202822 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-f78vh" event={"ID":"b129a63b-6e6f-4ca8-bd52-05d8638059fd","Type":"ContainerStarted","Data":"c603187526b9497b3b6c6ea70579f87db98743d3ff3b97fb44feeac8e580a8d2"} Jul 06 19:28:29.203186 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:28:29.202862 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-f78vh" Jul 06 19:28:29.245744 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:28:29.245698 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-f78vh" podStartSLOduration=2.245683578 podStartE2EDuration="2.245683578s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:28:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:28:29.24462409 +0000 UTC m=+1134.551983039" watchObservedRunningTime="2026-07-06 19:28:29.245683578 +0000 UTC m=+1134.553042527" Jul 06 19:28:40.210701 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:28:40.210665 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-f78vh" Jul 06 19:30:24.769214 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:30:24.769156 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:33:18.014414 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:33:18.014362 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:34:07.296724 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:07.296687 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:34:11.837450 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:11.837381 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:34:12.606874 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:12.606834 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:34:20.860484 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:20.860441 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:34:26.049736 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:26.049680 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:34:29.035488 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:29.035414 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:34:29.123913 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:29.123883 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:34:29.432287 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:29.432184 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:34:35.806886 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:35.806848 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-215c8b19-openshift-default-765479746f-zjvht"] Jul 06 19:34:35.810846 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:35.810825 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-215c8b19-openshift-default-765479746f-zjvht" Jul 06 19:34:35.813657 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:35.813628 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-215c8b19-openshift-default-dockercfg-hxdf5\"" Jul 06 19:34:35.839767 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:35.839732 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-215c8b19-openshift-default-765479746f-zjvht"] Jul 06 19:34:35.877271 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:35.877234 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd0af5e7-30e4-4416-86f6-3f9ec19bb8f8-workload-socket\") pod \"e2e-ait-215c8b19-openshift-default-765479746f-zjvht\" (UID: \"bd0af5e7-30e4-4416-86f6-3f9ec19bb8f8\") " pod="openshift-ingress/e2e-ait-215c8b19-openshift-default-765479746f-zjvht" Jul 06 19:34:35.877440 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:35.877304 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bd0af5e7-30e4-4416-86f6-3f9ec19bb8f8-istio-data\") pod \"e2e-ait-215c8b19-openshift-default-765479746f-zjvht\" (UID: \"bd0af5e7-30e4-4416-86f6-3f9ec19bb8f8\") " pod="openshift-ingress/e2e-ait-215c8b19-openshift-default-765479746f-zjvht" Jul 06 19:34:35.877440 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:35.877360 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bd0af5e7-30e4-4416-86f6-3f9ec19bb8f8-workload-certs\") pod \"e2e-ait-215c8b19-openshift-default-765479746f-zjvht\" (UID: \"bd0af5e7-30e4-4416-86f6-3f9ec19bb8f8\") " pod="openshift-ingress/e2e-ait-215c8b19-openshift-default-765479746f-zjvht" Jul 06 19:34:35.877440 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:35.877406 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g8tcs\" (UniqueName: \"kubernetes.io/projected/bd0af5e7-30e4-4416-86f6-3f9ec19bb8f8-kube-api-access-g8tcs\") pod \"e2e-ait-215c8b19-openshift-default-765479746f-zjvht\" (UID: \"bd0af5e7-30e4-4416-86f6-3f9ec19bb8f8\") " pod="openshift-ingress/e2e-ait-215c8b19-openshift-default-765479746f-zjvht" Jul 06 19:34:35.877546 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:35.877473 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd0af5e7-30e4-4416-86f6-3f9ec19bb8f8-credential-socket\") pod \"e2e-ait-215c8b19-openshift-default-765479746f-zjvht\" (UID: \"bd0af5e7-30e4-4416-86f6-3f9ec19bb8f8\") " pod="openshift-ingress/e2e-ait-215c8b19-openshift-default-765479746f-zjvht" Jul 06 19:34:35.877546 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:35.877489 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bd0af5e7-30e4-4416-86f6-3f9ec19bb8f8-istiod-ca-cert\") pod \"e2e-ait-215c8b19-openshift-default-765479746f-zjvht\" (UID: \"bd0af5e7-30e4-4416-86f6-3f9ec19bb8f8\") " pod="openshift-ingress/e2e-ait-215c8b19-openshift-default-765479746f-zjvht" Jul 06 19:34:35.877546 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:35.877509 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bd0af5e7-30e4-4416-86f6-3f9ec19bb8f8-istio-envoy\") pod \"e2e-ait-215c8b19-openshift-default-765479746f-zjvht\" (UID: \"bd0af5e7-30e4-4416-86f6-3f9ec19bb8f8\") " pod="openshift-ingress/e2e-ait-215c8b19-openshift-default-765479746f-zjvht" Jul 06 19:34:35.877546 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:35.877532 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bd0af5e7-30e4-4416-86f6-3f9ec19bb8f8-istio-podinfo\") pod \"e2e-ait-215c8b19-openshift-default-765479746f-zjvht\" (UID: \"bd0af5e7-30e4-4416-86f6-3f9ec19bb8f8\") " pod="openshift-ingress/e2e-ait-215c8b19-openshift-default-765479746f-zjvht" Jul 06 19:34:35.877665 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:35.877569 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bd0af5e7-30e4-4416-86f6-3f9ec19bb8f8-istio-token\") pod \"e2e-ait-215c8b19-openshift-default-765479746f-zjvht\" (UID: \"bd0af5e7-30e4-4416-86f6-3f9ec19bb8f8\") " pod="openshift-ingress/e2e-ait-215c8b19-openshift-default-765479746f-zjvht" Jul 06 19:34:35.978890 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:35.978850 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd0af5e7-30e4-4416-86f6-3f9ec19bb8f8-workload-socket\") pod \"e2e-ait-215c8b19-openshift-default-765479746f-zjvht\" (UID: \"bd0af5e7-30e4-4416-86f6-3f9ec19bb8f8\") " pod="openshift-ingress/e2e-ait-215c8b19-openshift-default-765479746f-zjvht" Jul 06 19:34:35.979097 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:35.978909 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bd0af5e7-30e4-4416-86f6-3f9ec19bb8f8-istio-data\") pod \"e2e-ait-215c8b19-openshift-default-765479746f-zjvht\" (UID: \"bd0af5e7-30e4-4416-86f6-3f9ec19bb8f8\") " pod="openshift-ingress/e2e-ait-215c8b19-openshift-default-765479746f-zjvht" Jul 06 19:34:35.979097 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:35.978938 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bd0af5e7-30e4-4416-86f6-3f9ec19bb8f8-workload-certs\") pod \"e2e-ait-215c8b19-openshift-default-765479746f-zjvht\" (UID: \"bd0af5e7-30e4-4416-86f6-3f9ec19bb8f8\") " pod="openshift-ingress/e2e-ait-215c8b19-openshift-default-765479746f-zjvht" Jul 06 19:34:35.979097 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:35.978970 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g8tcs\" (UniqueName: \"kubernetes.io/projected/bd0af5e7-30e4-4416-86f6-3f9ec19bb8f8-kube-api-access-g8tcs\") pod \"e2e-ait-215c8b19-openshift-default-765479746f-zjvht\" (UID: \"bd0af5e7-30e4-4416-86f6-3f9ec19bb8f8\") " pod="openshift-ingress/e2e-ait-215c8b19-openshift-default-765479746f-zjvht" Jul 06 19:34:35.979097 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:35.979019 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd0af5e7-30e4-4416-86f6-3f9ec19bb8f8-credential-socket\") pod \"e2e-ait-215c8b19-openshift-default-765479746f-zjvht\" (UID: \"bd0af5e7-30e4-4416-86f6-3f9ec19bb8f8\") " pod="openshift-ingress/e2e-ait-215c8b19-openshift-default-765479746f-zjvht" Jul 06 19:34:35.979097 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:35.979046 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bd0af5e7-30e4-4416-86f6-3f9ec19bb8f8-istiod-ca-cert\") pod \"e2e-ait-215c8b19-openshift-default-765479746f-zjvht\" (UID: \"bd0af5e7-30e4-4416-86f6-3f9ec19bb8f8\") " pod="openshift-ingress/e2e-ait-215c8b19-openshift-default-765479746f-zjvht" Jul 06 19:34:35.979352 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:35.979099 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bd0af5e7-30e4-4416-86f6-3f9ec19bb8f8-istio-envoy\") pod \"e2e-ait-215c8b19-openshift-default-765479746f-zjvht\" (UID: \"bd0af5e7-30e4-4416-86f6-3f9ec19bb8f8\") " pod="openshift-ingress/e2e-ait-215c8b19-openshift-default-765479746f-zjvht" Jul 06 19:34:35.979352 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:35.979132 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bd0af5e7-30e4-4416-86f6-3f9ec19bb8f8-istio-podinfo\") pod \"e2e-ait-215c8b19-openshift-default-765479746f-zjvht\" (UID: \"bd0af5e7-30e4-4416-86f6-3f9ec19bb8f8\") " pod="openshift-ingress/e2e-ait-215c8b19-openshift-default-765479746f-zjvht" Jul 06 19:34:35.979352 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:35.979164 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bd0af5e7-30e4-4416-86f6-3f9ec19bb8f8-istio-token\") pod \"e2e-ait-215c8b19-openshift-default-765479746f-zjvht\" (UID: \"bd0af5e7-30e4-4416-86f6-3f9ec19bb8f8\") " pod="openshift-ingress/e2e-ait-215c8b19-openshift-default-765479746f-zjvht" Jul 06 19:34:35.979496 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:35.979378 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd0af5e7-30e4-4416-86f6-3f9ec19bb8f8-workload-socket\") pod \"e2e-ait-215c8b19-openshift-default-765479746f-zjvht\" (UID: \"bd0af5e7-30e4-4416-86f6-3f9ec19bb8f8\") " pod="openshift-ingress/e2e-ait-215c8b19-openshift-default-765479746f-zjvht" Jul 06 19:34:35.979496 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:35.979407 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bd0af5e7-30e4-4416-86f6-3f9ec19bb8f8-workload-certs\") pod \"e2e-ait-215c8b19-openshift-default-765479746f-zjvht\" (UID: \"bd0af5e7-30e4-4416-86f6-3f9ec19bb8f8\") " pod="openshift-ingress/e2e-ait-215c8b19-openshift-default-765479746f-zjvht" Jul 06 19:34:35.979564 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:35.979543 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd0af5e7-30e4-4416-86f6-3f9ec19bb8f8-credential-socket\") pod \"e2e-ait-215c8b19-openshift-default-765479746f-zjvht\" (UID: \"bd0af5e7-30e4-4416-86f6-3f9ec19bb8f8\") " pod="openshift-ingress/e2e-ait-215c8b19-openshift-default-765479746f-zjvht" Jul 06 19:34:35.979698 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:35.979677 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bd0af5e7-30e4-4416-86f6-3f9ec19bb8f8-istio-data\") pod \"e2e-ait-215c8b19-openshift-default-765479746f-zjvht\" (UID: \"bd0af5e7-30e4-4416-86f6-3f9ec19bb8f8\") " pod="openshift-ingress/e2e-ait-215c8b19-openshift-default-765479746f-zjvht" Jul 06 19:34:35.979810 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:35.979791 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bd0af5e7-30e4-4416-86f6-3f9ec19bb8f8-istiod-ca-cert\") pod \"e2e-ait-215c8b19-openshift-default-765479746f-zjvht\" (UID: \"bd0af5e7-30e4-4416-86f6-3f9ec19bb8f8\") " pod="openshift-ingress/e2e-ait-215c8b19-openshift-default-765479746f-zjvht" Jul 06 19:34:35.981719 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:35.981696 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bd0af5e7-30e4-4416-86f6-3f9ec19bb8f8-istio-envoy\") pod \"e2e-ait-215c8b19-openshift-default-765479746f-zjvht\" (UID: \"bd0af5e7-30e4-4416-86f6-3f9ec19bb8f8\") " pod="openshift-ingress/e2e-ait-215c8b19-openshift-default-765479746f-zjvht" Jul 06 19:34:35.981832 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:35.981820 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bd0af5e7-30e4-4416-86f6-3f9ec19bb8f8-istio-podinfo\") pod \"e2e-ait-215c8b19-openshift-default-765479746f-zjvht\" (UID: \"bd0af5e7-30e4-4416-86f6-3f9ec19bb8f8\") " pod="openshift-ingress/e2e-ait-215c8b19-openshift-default-765479746f-zjvht" Jul 06 19:34:35.990680 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:35.990651 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bd0af5e7-30e4-4416-86f6-3f9ec19bb8f8-istio-token\") pod \"e2e-ait-215c8b19-openshift-default-765479746f-zjvht\" (UID: \"bd0af5e7-30e4-4416-86f6-3f9ec19bb8f8\") " pod="openshift-ingress/e2e-ait-215c8b19-openshift-default-765479746f-zjvht" Jul 06 19:34:35.991027 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:35.991005 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g8tcs\" (UniqueName: \"kubernetes.io/projected/bd0af5e7-30e4-4416-86f6-3f9ec19bb8f8-kube-api-access-g8tcs\") pod \"e2e-ait-215c8b19-openshift-default-765479746f-zjvht\" (UID: \"bd0af5e7-30e4-4416-86f6-3f9ec19bb8f8\") " pod="openshift-ingress/e2e-ait-215c8b19-openshift-default-765479746f-zjvht" Jul 06 19:34:36.121333 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:36.121237 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-215c8b19-openshift-default-765479746f-zjvht" Jul 06 19:34:36.285330 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:36.285306 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-215c8b19-openshift-default-765479746f-zjvht"] Jul 06 19:34:36.286672 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:34:36.286643 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd0af5e7_30e4_4416_86f6_3f9ec19bb8f8.slice/crio-76377ddff70ff9c64f24b0f4b212721f464a75a6c130241d0f6b2587c3fe41e6 WatchSource:0}: Error finding container 76377ddff70ff9c64f24b0f4b212721f464a75a6c130241d0f6b2587c3fe41e6: Status 404 returned error can't find the container with id 76377ddff70ff9c64f24b0f4b212721f464a75a6c130241d0f6b2587c3fe41e6 Jul 06 19:34:36.288767 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:36.288737 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 19:34:36.288847 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:36.288798 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 19:34:36.288847 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:36.288828 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 19:34:36.564014 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:36.563977 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:34:36.764414 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:36.764379 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:34:37.082182 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:37.082155 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:34:37.249856 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:37.249826 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:34:37.279375 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:37.279336 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-215c8b19-openshift-default-765479746f-zjvht" event={"ID":"bd0af5e7-30e4-4416-86f6-3f9ec19bb8f8","Type":"ContainerStarted","Data":"bd22f9dce69749c8537069e656794e83b87aa3af30e1424b0eb996e0a8b33410"} Jul 06 19:34:37.279669 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:37.279626 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-215c8b19-openshift-default-765479746f-zjvht" event={"ID":"bd0af5e7-30e4-4416-86f6-3f9ec19bb8f8","Type":"ContainerStarted","Data":"76377ddff70ff9c64f24b0f4b212721f464a75a6c130241d0f6b2587c3fe41e6"} Jul 06 19:34:37.324199 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:37.324136 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-215c8b19-openshift-default-765479746f-zjvht" podStartSLOduration=2.324114261 podStartE2EDuration="2.324114261s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:34:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:34:37.313923981 +0000 UTC m=+1502.621282941" watchObservedRunningTime="2026-07-06 19:34:37.324114261 +0000 UTC m=+1502.631473204" Jul 06 19:34:37.383328 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:37.383237 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:34:37.540887 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:37.540852 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:34:38.020116 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:38.020057 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:34:38.121767 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:38.121723 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-215c8b19-openshift-default-765479746f-zjvht" Jul 06 19:34:38.126862 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:38.126834 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-215c8b19-openshift-default-765479746f-zjvht" Jul 06 19:34:38.191843 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:38.191811 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:34:38.285134 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:38.285108 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-215c8b19-openshift-default-765479746f-zjvht" Jul 06 19:34:38.285969 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:38.285949 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-215c8b19-openshift-default-765479746f-zjvht" Jul 06 19:34:39.717387 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:39.717356 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:34:42.037112 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:42.036478 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:34:42.202444 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:42.202394 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-215c8b19-openshift-default-765479746f-zjvht"] Jul 06 19:34:42.202634 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:42.202586 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-215c8b19-openshift-default-765479746f-zjvht" podUID="bd0af5e7-30e4-4416-86f6-3f9ec19bb8f8" containerName="istio-proxy" containerID="cri-o://bd22f9dce69749c8537069e656794e83b87aa3af30e1424b0eb996e0a8b33410" gracePeriod=30 Jul 06 19:34:42.235171 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:42.235137 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:34:42.331175 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:42.331057 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:34:42.546609 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:42.546560 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:34:44.118713 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:44.118647 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:34:48.758224 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:48.758192 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-215c8b19-openshift-default-765479746f-zjvht" Jul 06 19:34:48.823524 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:48.823485 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:34:48.905446 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:48.905409 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g8tcs\" (UniqueName: \"kubernetes.io/projected/bd0af5e7-30e4-4416-86f6-3f9ec19bb8f8-kube-api-access-g8tcs\") pod \"bd0af5e7-30e4-4416-86f6-3f9ec19bb8f8\" (UID: \"bd0af5e7-30e4-4416-86f6-3f9ec19bb8f8\") " Jul 06 19:34:48.905446 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:48.905445 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bd0af5e7-30e4-4416-86f6-3f9ec19bb8f8-istio-token\") pod \"bd0af5e7-30e4-4416-86f6-3f9ec19bb8f8\" (UID: \"bd0af5e7-30e4-4416-86f6-3f9ec19bb8f8\") " Jul 06 19:34:48.905697 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:48.905475 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd0af5e7-30e4-4416-86f6-3f9ec19bb8f8-credential-socket\") pod \"bd0af5e7-30e4-4416-86f6-3f9ec19bb8f8\" (UID: \"bd0af5e7-30e4-4416-86f6-3f9ec19bb8f8\") " Jul 06 19:34:48.905697 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:48.905519 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bd0af5e7-30e4-4416-86f6-3f9ec19bb8f8-workload-certs\") pod \"bd0af5e7-30e4-4416-86f6-3f9ec19bb8f8\" (UID: \"bd0af5e7-30e4-4416-86f6-3f9ec19bb8f8\") " Jul 06 19:34:48.905697 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:48.905535 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bd0af5e7-30e4-4416-86f6-3f9ec19bb8f8-istio-podinfo\") pod \"bd0af5e7-30e4-4416-86f6-3f9ec19bb8f8\" (UID: \"bd0af5e7-30e4-4416-86f6-3f9ec19bb8f8\") " Jul 06 19:34:48.905697 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:48.905583 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bd0af5e7-30e4-4416-86f6-3f9ec19bb8f8-istiod-ca-cert\") pod \"bd0af5e7-30e4-4416-86f6-3f9ec19bb8f8\" (UID: \"bd0af5e7-30e4-4416-86f6-3f9ec19bb8f8\") " Jul 06 19:34:48.905697 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:48.905600 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bd0af5e7-30e4-4416-86f6-3f9ec19bb8f8-istio-data\") pod \"bd0af5e7-30e4-4416-86f6-3f9ec19bb8f8\" (UID: \"bd0af5e7-30e4-4416-86f6-3f9ec19bb8f8\") " Jul 06 19:34:48.905963 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:48.905782 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd0af5e7-30e4-4416-86f6-3f9ec19bb8f8-workload-socket\") pod \"bd0af5e7-30e4-4416-86f6-3f9ec19bb8f8\" (UID: \"bd0af5e7-30e4-4416-86f6-3f9ec19bb8f8\") " Jul 06 19:34:48.905963 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:48.905822 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bd0af5e7-30e4-4416-86f6-3f9ec19bb8f8-istio-envoy\") pod \"bd0af5e7-30e4-4416-86f6-3f9ec19bb8f8\" (UID: \"bd0af5e7-30e4-4416-86f6-3f9ec19bb8f8\") " Jul 06 19:34:48.905963 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:48.905849 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd0af5e7-30e4-4416-86f6-3f9ec19bb8f8-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "bd0af5e7-30e4-4416-86f6-3f9ec19bb8f8" (UID: "bd0af5e7-30e4-4416-86f6-3f9ec19bb8f8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:34:48.905963 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:48.905908 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd0af5e7-30e4-4416-86f6-3f9ec19bb8f8-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "bd0af5e7-30e4-4416-86f6-3f9ec19bb8f8" (UID: "bd0af5e7-30e4-4416-86f6-3f9ec19bb8f8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:34:48.906187 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:48.906101 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd0af5e7-30e4-4416-86f6-3f9ec19bb8f8-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "bd0af5e7-30e4-4416-86f6-3f9ec19bb8f8" (UID: "bd0af5e7-30e4-4416-86f6-3f9ec19bb8f8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:34:48.906405 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:48.906250 2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd0af5e7-30e4-4416-86f6-3f9ec19bb8f8-credential-socket\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:34:48.906405 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:48.906268 2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bd0af5e7-30e4-4416-86f6-3f9ec19bb8f8-workload-certs\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:34:48.906405 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:48.906282 2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd0af5e7-30e4-4416-86f6-3f9ec19bb8f8-workload-socket\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:34:48.906405 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:48.906319 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bd0af5e7-30e4-4416-86f6-3f9ec19bb8f8-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "bd0af5e7-30e4-4416-86f6-3f9ec19bb8f8" (UID: "bd0af5e7-30e4-4416-86f6-3f9ec19bb8f8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:34:48.906919 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:48.906891 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd0af5e7-30e4-4416-86f6-3f9ec19bb8f8-istio-data" (OuterVolumeSpecName: "istio-data") pod "bd0af5e7-30e4-4416-86f6-3f9ec19bb8f8" (UID: "bd0af5e7-30e4-4416-86f6-3f9ec19bb8f8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:34:48.908007 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:48.907982 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd0af5e7-30e4-4416-86f6-3f9ec19bb8f8-istio-token" (OuterVolumeSpecName: "istio-token") pod "bd0af5e7-30e4-4416-86f6-3f9ec19bb8f8" (UID: "bd0af5e7-30e4-4416-86f6-3f9ec19bb8f8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:34:48.908130 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:48.908039 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd0af5e7-30e4-4416-86f6-3f9ec19bb8f8-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "bd0af5e7-30e4-4416-86f6-3f9ec19bb8f8" (UID: "bd0af5e7-30e4-4416-86f6-3f9ec19bb8f8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:34:48.908130 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:48.908046 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd0af5e7-30e4-4416-86f6-3f9ec19bb8f8-kube-api-access-g8tcs" (OuterVolumeSpecName: "kube-api-access-g8tcs") pod "bd0af5e7-30e4-4416-86f6-3f9ec19bb8f8" (UID: "bd0af5e7-30e4-4416-86f6-3f9ec19bb8f8"). InnerVolumeSpecName "kube-api-access-g8tcs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:34:48.908217 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:48.908148 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/bd0af5e7-30e4-4416-86f6-3f9ec19bb8f8-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "bd0af5e7-30e4-4416-86f6-3f9ec19bb8f8" (UID: "bd0af5e7-30e4-4416-86f6-3f9ec19bb8f8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 19:34:48.928433 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:48.928394 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:34:49.007732 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:49.007699 2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bd0af5e7-30e4-4416-86f6-3f9ec19bb8f8-istio-data\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:34:49.007732 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:49.007728 2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bd0af5e7-30e4-4416-86f6-3f9ec19bb8f8-istio-envoy\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:34:49.007960 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:49.007741 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g8tcs\" (UniqueName: \"kubernetes.io/projected/bd0af5e7-30e4-4416-86f6-3f9ec19bb8f8-kube-api-access-g8tcs\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:34:49.007960 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:49.007756 2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bd0af5e7-30e4-4416-86f6-3f9ec19bb8f8-istio-token\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:34:49.007960 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:49.007766 2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bd0af5e7-30e4-4416-86f6-3f9ec19bb8f8-istio-podinfo\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:34:49.007960 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:49.007775 2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bd0af5e7-30e4-4416-86f6-3f9ec19bb8f8-istiod-ca-cert\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:34:49.024728 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:49.024696 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-e58cbfec-openshift-default-7d889f85fc-vk2c7"] Jul 06 19:34:49.025161 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:49.025142 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd0af5e7-30e4-4416-86f6-3f9ec19bb8f8" containerName="istio-proxy" Jul 06 19:34:49.025161 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:49.025163 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd0af5e7-30e4-4416-86f6-3f9ec19bb8f8" containerName="istio-proxy" Jul 06 19:34:49.025346 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:49.025232 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd0af5e7-30e4-4416-86f6-3f9ec19bb8f8" containerName="istio-proxy" Jul 06 19:34:49.028281 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:49.028259 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-e58cbfec-openshift-default-7d889f85fc-vk2c7" Jul 06 19:34:49.030631 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:49.030609 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-e58cbfec-openshift-default-dockercfg-ftfct\"" Jul 06 19:34:49.048819 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:49.048785 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-e58cbfec-openshift-default-7d889f85fc-vk2c7"] Jul 06 19:34:49.078394 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:49.078289 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:34:49.108548 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:49.108475 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/767926fa-2c86-4bc1-a653-c0b25367ade2-workload-socket\") pod \"e2e-ait-e58cbfec-openshift-default-7d889f85fc-vk2c7\" (UID: \"767926fa-2c86-4bc1-a653-c0b25367ade2\") " pod="openshift-ingress/e2e-ait-e58cbfec-openshift-default-7d889f85fc-vk2c7" Jul 06 19:34:49.108548 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:49.108548 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/767926fa-2c86-4bc1-a653-c0b25367ade2-credential-socket\") pod \"e2e-ait-e58cbfec-openshift-default-7d889f85fc-vk2c7\" (UID: \"767926fa-2c86-4bc1-a653-c0b25367ade2\") " pod="openshift-ingress/e2e-ait-e58cbfec-openshift-default-7d889f85fc-vk2c7" Jul 06 19:34:49.108756 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:49.108570 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/767926fa-2c86-4bc1-a653-c0b25367ade2-istiod-ca-cert\") pod \"e2e-ait-e58cbfec-openshift-default-7d889f85fc-vk2c7\" (UID: \"767926fa-2c86-4bc1-a653-c0b25367ade2\") " pod="openshift-ingress/e2e-ait-e58cbfec-openshift-default-7d889f85fc-vk2c7" Jul 06 19:34:49.108756 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:49.108629 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/767926fa-2c86-4bc1-a653-c0b25367ade2-workload-certs\") pod \"e2e-ait-e58cbfec-openshift-default-7d889f85fc-vk2c7\" (UID: \"767926fa-2c86-4bc1-a653-c0b25367ade2\") " pod="openshift-ingress/e2e-ait-e58cbfec-openshift-default-7d889f85fc-vk2c7" Jul 06 19:34:49.108756 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:49.108683 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d67qb\" (UniqueName: \"kubernetes.io/projected/767926fa-2c86-4bc1-a653-c0b25367ade2-kube-api-access-d67qb\") pod \"e2e-ait-e58cbfec-openshift-default-7d889f85fc-vk2c7\" (UID: \"767926fa-2c86-4bc1-a653-c0b25367ade2\") " pod="openshift-ingress/e2e-ait-e58cbfec-openshift-default-7d889f85fc-vk2c7" Jul 06 19:34:49.108756 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:49.108715 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/767926fa-2c86-4bc1-a653-c0b25367ade2-istio-data\") pod \"e2e-ait-e58cbfec-openshift-default-7d889f85fc-vk2c7\" (UID: \"767926fa-2c86-4bc1-a653-c0b25367ade2\") " pod="openshift-ingress/e2e-ait-e58cbfec-openshift-default-7d889f85fc-vk2c7" Jul 06 19:34:49.108756 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:49.108741 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/767926fa-2c86-4bc1-a653-c0b25367ade2-istio-envoy\") pod \"e2e-ait-e58cbfec-openshift-default-7d889f85fc-vk2c7\" (UID: \"767926fa-2c86-4bc1-a653-c0b25367ade2\") " pod="openshift-ingress/e2e-ait-e58cbfec-openshift-default-7d889f85fc-vk2c7" Jul 06 19:34:49.108924 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:49.108772 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/767926fa-2c86-4bc1-a653-c0b25367ade2-istio-token\") pod \"e2e-ait-e58cbfec-openshift-default-7d889f85fc-vk2c7\" (UID: \"767926fa-2c86-4bc1-a653-c0b25367ade2\") " pod="openshift-ingress/e2e-ait-e58cbfec-openshift-default-7d889f85fc-vk2c7" Jul 06 19:34:49.108924 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:49.108798 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/767926fa-2c86-4bc1-a653-c0b25367ade2-istio-podinfo\") pod \"e2e-ait-e58cbfec-openshift-default-7d889f85fc-vk2c7\" (UID: \"767926fa-2c86-4bc1-a653-c0b25367ade2\") " pod="openshift-ingress/e2e-ait-e58cbfec-openshift-default-7d889f85fc-vk2c7" Jul 06 19:34:49.209887 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:49.209843 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d67qb\" (UniqueName: \"kubernetes.io/projected/767926fa-2c86-4bc1-a653-c0b25367ade2-kube-api-access-d67qb\") pod \"e2e-ait-e58cbfec-openshift-default-7d889f85fc-vk2c7\" (UID: \"767926fa-2c86-4bc1-a653-c0b25367ade2\") " pod="openshift-ingress/e2e-ait-e58cbfec-openshift-default-7d889f85fc-vk2c7" Jul 06 19:34:49.209887 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:49.209892 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/767926fa-2c86-4bc1-a653-c0b25367ade2-istio-data\") pod \"e2e-ait-e58cbfec-openshift-default-7d889f85fc-vk2c7\" (UID: \"767926fa-2c86-4bc1-a653-c0b25367ade2\") " pod="openshift-ingress/e2e-ait-e58cbfec-openshift-default-7d889f85fc-vk2c7" Jul 06 19:34:49.210222 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:49.209912 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/767926fa-2c86-4bc1-a653-c0b25367ade2-istio-envoy\") pod \"e2e-ait-e58cbfec-openshift-default-7d889f85fc-vk2c7\" (UID: \"767926fa-2c86-4bc1-a653-c0b25367ade2\") " pod="openshift-ingress/e2e-ait-e58cbfec-openshift-default-7d889f85fc-vk2c7" Jul 06 19:34:49.210222 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:49.209940 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/767926fa-2c86-4bc1-a653-c0b25367ade2-istio-token\") pod \"e2e-ait-e58cbfec-openshift-default-7d889f85fc-vk2c7\" (UID: \"767926fa-2c86-4bc1-a653-c0b25367ade2\") " pod="openshift-ingress/e2e-ait-e58cbfec-openshift-default-7d889f85fc-vk2c7" Jul 06 19:34:49.210222 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:49.209959 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/767926fa-2c86-4bc1-a653-c0b25367ade2-istio-podinfo\") pod \"e2e-ait-e58cbfec-openshift-default-7d889f85fc-vk2c7\" (UID: \"767926fa-2c86-4bc1-a653-c0b25367ade2\") " pod="openshift-ingress/e2e-ait-e58cbfec-openshift-default-7d889f85fc-vk2c7" Jul 06 19:34:49.210222 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:49.210027 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/767926fa-2c86-4bc1-a653-c0b25367ade2-workload-socket\") pod \"e2e-ait-e58cbfec-openshift-default-7d889f85fc-vk2c7\" (UID: \"767926fa-2c86-4bc1-a653-c0b25367ade2\") " pod="openshift-ingress/e2e-ait-e58cbfec-openshift-default-7d889f85fc-vk2c7" Jul 06 19:34:49.210222 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:49.210058 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/767926fa-2c86-4bc1-a653-c0b25367ade2-credential-socket\") pod \"e2e-ait-e58cbfec-openshift-default-7d889f85fc-vk2c7\" (UID: \"767926fa-2c86-4bc1-a653-c0b25367ade2\") " pod="openshift-ingress/e2e-ait-e58cbfec-openshift-default-7d889f85fc-vk2c7" Jul 06 19:34:49.210222 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:49.210199 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/767926fa-2c86-4bc1-a653-c0b25367ade2-istiod-ca-cert\") pod \"e2e-ait-e58cbfec-openshift-default-7d889f85fc-vk2c7\" (UID: \"767926fa-2c86-4bc1-a653-c0b25367ade2\") " pod="openshift-ingress/e2e-ait-e58cbfec-openshift-default-7d889f85fc-vk2c7" Jul 06 19:34:49.210541 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:49.210234 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/767926fa-2c86-4bc1-a653-c0b25367ade2-workload-certs\") pod \"e2e-ait-e58cbfec-openshift-default-7d889f85fc-vk2c7\" (UID: \"767926fa-2c86-4bc1-a653-c0b25367ade2\") " pod="openshift-ingress/e2e-ait-e58cbfec-openshift-default-7d889f85fc-vk2c7" Jul 06 19:34:49.210541 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:49.210373 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/767926fa-2c86-4bc1-a653-c0b25367ade2-istio-data\") pod \"e2e-ait-e58cbfec-openshift-default-7d889f85fc-vk2c7\" (UID: \"767926fa-2c86-4bc1-a653-c0b25367ade2\") " pod="openshift-ingress/e2e-ait-e58cbfec-openshift-default-7d889f85fc-vk2c7" Jul 06 19:34:49.210541 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:49.210467 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/767926fa-2c86-4bc1-a653-c0b25367ade2-workload-socket\") pod \"e2e-ait-e58cbfec-openshift-default-7d889f85fc-vk2c7\" (UID: \"767926fa-2c86-4bc1-a653-c0b25367ade2\") " pod="openshift-ingress/e2e-ait-e58cbfec-openshift-default-7d889f85fc-vk2c7" Jul 06 19:34:49.210686 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:49.210555 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/767926fa-2c86-4bc1-a653-c0b25367ade2-credential-socket\") pod \"e2e-ait-e58cbfec-openshift-default-7d889f85fc-vk2c7\" (UID: \"767926fa-2c86-4bc1-a653-c0b25367ade2\") " pod="openshift-ingress/e2e-ait-e58cbfec-openshift-default-7d889f85fc-vk2c7" Jul 06 19:34:49.210686 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:49.210611 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/767926fa-2c86-4bc1-a653-c0b25367ade2-workload-certs\") pod \"e2e-ait-e58cbfec-openshift-default-7d889f85fc-vk2c7\" (UID: \"767926fa-2c86-4bc1-a653-c0b25367ade2\") " pod="openshift-ingress/e2e-ait-e58cbfec-openshift-default-7d889f85fc-vk2c7" Jul 06 19:34:49.211102 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:49.211042 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/767926fa-2c86-4bc1-a653-c0b25367ade2-istiod-ca-cert\") pod \"e2e-ait-e58cbfec-openshift-default-7d889f85fc-vk2c7\" (UID: \"767926fa-2c86-4bc1-a653-c0b25367ade2\") " pod="openshift-ingress/e2e-ait-e58cbfec-openshift-default-7d889f85fc-vk2c7" Jul 06 19:34:49.212513 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:49.212483 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/767926fa-2c86-4bc1-a653-c0b25367ade2-istio-envoy\") pod \"e2e-ait-e58cbfec-openshift-default-7d889f85fc-vk2c7\" (UID: \"767926fa-2c86-4bc1-a653-c0b25367ade2\") " pod="openshift-ingress/e2e-ait-e58cbfec-openshift-default-7d889f85fc-vk2c7" Jul 06 19:34:49.212704 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:49.212687 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/767926fa-2c86-4bc1-a653-c0b25367ade2-istio-podinfo\") pod \"e2e-ait-e58cbfec-openshift-default-7d889f85fc-vk2c7\" (UID: \"767926fa-2c86-4bc1-a653-c0b25367ade2\") " pod="openshift-ingress/e2e-ait-e58cbfec-openshift-default-7d889f85fc-vk2c7" Jul 06 19:34:49.216997 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:49.216975 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:34:49.222173 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:49.222148 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/767926fa-2c86-4bc1-a653-c0b25367ade2-istio-token\") pod \"e2e-ait-e58cbfec-openshift-default-7d889f85fc-vk2c7\" (UID: \"767926fa-2c86-4bc1-a653-c0b25367ade2\") " pod="openshift-ingress/e2e-ait-e58cbfec-openshift-default-7d889f85fc-vk2c7" Jul 06 19:34:49.222436 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:49.222415 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d67qb\" (UniqueName: \"kubernetes.io/projected/767926fa-2c86-4bc1-a653-c0b25367ade2-kube-api-access-d67qb\") pod \"e2e-ait-e58cbfec-openshift-default-7d889f85fc-vk2c7\" (UID: \"767926fa-2c86-4bc1-a653-c0b25367ade2\") " pod="openshift-ingress/e2e-ait-e58cbfec-openshift-default-7d889f85fc-vk2c7" Jul 06 19:34:49.340935 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:49.340827 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-e58cbfec-openshift-default-7d889f85fc-vk2c7" Jul 06 19:34:49.351566 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:49.351531 2566 generic.go:356] "Generic (PLEG): container finished" podID="bd0af5e7-30e4-4416-86f6-3f9ec19bb8f8" containerID="bd22f9dce69749c8537069e656794e83b87aa3af30e1424b0eb996e0a8b33410" exitCode=0 Jul 06 19:34:49.351712 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:49.351611 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-215c8b19-openshift-default-765479746f-zjvht" Jul 06 19:34:49.351712 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:49.351619 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-215c8b19-openshift-default-765479746f-zjvht" event={"ID":"bd0af5e7-30e4-4416-86f6-3f9ec19bb8f8","Type":"ContainerDied","Data":"bd22f9dce69749c8537069e656794e83b87aa3af30e1424b0eb996e0a8b33410"} Jul 06 19:34:49.351712 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:49.351699 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-215c8b19-openshift-default-765479746f-zjvht" event={"ID":"bd0af5e7-30e4-4416-86f6-3f9ec19bb8f8","Type":"ContainerDied","Data":"76377ddff70ff9c64f24b0f4b212721f464a75a6c130241d0f6b2587c3fe41e6"} Jul 06 19:34:49.351819 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:49.351721 2566 scope.go:117] "RemoveContainer" containerID="bd22f9dce69749c8537069e656794e83b87aa3af30e1424b0eb996e0a8b33410" Jul 06 19:34:49.361751 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:49.361733 2566 scope.go:117] "RemoveContainer" containerID="bd22f9dce69749c8537069e656794e83b87aa3af30e1424b0eb996e0a8b33410" Jul 06 19:34:49.362129 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:34:49.362099 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bd22f9dce69749c8537069e656794e83b87aa3af30e1424b0eb996e0a8b33410\": container with ID starting with bd22f9dce69749c8537069e656794e83b87aa3af30e1424b0eb996e0a8b33410 not found: ID does not exist" containerID="bd22f9dce69749c8537069e656794e83b87aa3af30e1424b0eb996e0a8b33410" Jul 06 19:34:49.362236 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:49.362141 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bd22f9dce69749c8537069e656794e83b87aa3af30e1424b0eb996e0a8b33410"} err="failed to get container status \"bd22f9dce69749c8537069e656794e83b87aa3af30e1424b0eb996e0a8b33410\": rpc error: code = NotFound desc = could not find container \"bd22f9dce69749c8537069e656794e83b87aa3af30e1424b0eb996e0a8b33410\": container with ID starting with bd22f9dce69749c8537069e656794e83b87aa3af30e1424b0eb996e0a8b33410 not found: ID does not exist" Jul 06 19:34:49.385413 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:49.385375 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-215c8b19-openshift-default-765479746f-zjvht"] Jul 06 19:34:49.396942 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:49.396906 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-215c8b19-openshift-default-765479746f-zjvht"] Jul 06 19:34:49.507018 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:34:49.506980 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod767926fa_2c86_4bc1_a653_c0b25367ade2.slice/crio-f0c6963b990d9549149280f6cedc141089e7093425484d167e53dfc7af9c9fce WatchSource:0}: Error finding container f0c6963b990d9549149280f6cedc141089e7093425484d167e53dfc7af9c9fce: Status 404 returned error can't find the container with id f0c6963b990d9549149280f6cedc141089e7093425484d167e53dfc7af9c9fce Jul 06 19:34:49.509525 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:49.509500 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-e58cbfec-openshift-default-7d889f85fc-vk2c7"] Jul 06 19:34:49.509705 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:49.509673 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 19:34:49.509775 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:49.509752 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 19:34:49.509832 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:49.509800 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 19:34:50.330890 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:50.330838 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:34:50.360134 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:50.360101 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e58cbfec-openshift-default-7d889f85fc-vk2c7" event={"ID":"767926fa-2c86-4bc1-a653-c0b25367ade2","Type":"ContainerStarted","Data":"a2274aa5607776b92c8f3db06a9426c187ef47ccb80446bc98ec6673dc321a59"} Jul 06 19:34:50.360321 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:50.360161 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e58cbfec-openshift-default-7d889f85fc-vk2c7" event={"ID":"767926fa-2c86-4bc1-a653-c0b25367ade2","Type":"ContainerStarted","Data":"f0c6963b990d9549149280f6cedc141089e7093425484d167e53dfc7af9c9fce"} Jul 06 19:34:50.416091 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:50.416015 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-e58cbfec-openshift-default-7d889f85fc-vk2c7" podStartSLOduration=2.41599868 podStartE2EDuration="2.41599868s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:34:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:34:50.413510824 +0000 UTC m=+1515.720869776" watchObservedRunningTime="2026-07-06 19:34:50.41599868 +0000 UTC m=+1515.723357627" Jul 06 19:34:50.442990 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:50.442958 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:34:50.549702 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:50.549661 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:34:51.238949 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:51.238919 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bd0af5e7-30e4-4416-86f6-3f9ec19bb8f8" path="/var/lib/kubelet/pods/bd0af5e7-30e4-4416-86f6-3f9ec19bb8f8/volumes" Jul 06 19:34:51.267388 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:51.267333 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:34:51.341756 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:51.341720 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-e58cbfec-openshift-default-7d889f85fc-vk2c7" Jul 06 19:34:51.347263 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:51.347230 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-e58cbfec-openshift-default-7d889f85fc-vk2c7" Jul 06 19:34:51.365820 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:51.365792 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-e58cbfec-openshift-default-7d889f85fc-vk2c7" Jul 06 19:34:51.366761 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:51.366738 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-e58cbfec-openshift-default-7d889f85fc-vk2c7" Jul 06 19:34:51.370270 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:51.370249 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:34:51.507918 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:51.507831 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:34:52.328899 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:52.328867 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:34:55.343109 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:55.343002 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:34:55.867272 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:34:55.867238 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:35:01.428407 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:01.428373 2566 generic.go:356] "Generic (PLEG): container finished" podID="767926fa-2c86-4bc1-a653-c0b25367ade2" containerID="a2274aa5607776b92c8f3db06a9426c187ef47ccb80446bc98ec6673dc321a59" exitCode=0 Jul 06 19:35:01.428893 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:01.428448 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e58cbfec-openshift-default-7d889f85fc-vk2c7" event={"ID":"767926fa-2c86-4bc1-a653-c0b25367ade2","Type":"ContainerDied","Data":"a2274aa5607776b92c8f3db06a9426c187ef47ccb80446bc98ec6673dc321a59"} Jul 06 19:35:01.428893 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:01.428843 2566 scope.go:117] "RemoveContainer" containerID="a2274aa5607776b92c8f3db06a9426c187ef47ccb80446bc98ec6673dc321a59" Jul 06 19:35:01.430022 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:01.429992 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 19:35:01.430140 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:01.430093 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 19:35:01.430215 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:01.430145 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 19:35:01.475929 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:01.475900 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-e58cbfec-openshift-default-7d889f85fc-vk2c7"] Jul 06 19:35:01.485256 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:01.485227 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:35:01.631767 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:01.631736 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:35:02.437622 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:02.437579 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e58cbfec-openshift-default-7d889f85fc-vk2c7" event={"ID":"767926fa-2c86-4bc1-a653-c0b25367ade2","Type":"ContainerStarted","Data":"3078d493754664d82ea932272f2509bf9a46d07374ca6c54770265c25a7ee439"} Jul 06 19:35:02.438141 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:02.437723 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-e58cbfec-openshift-default-7d889f85fc-vk2c7" podUID="767926fa-2c86-4bc1-a653-c0b25367ade2" containerName="istio-proxy" containerID="cri-o://3078d493754664d82ea932272f2509bf9a46d07374ca6c54770265c25a7ee439" gracePeriod=30 Jul 06 19:35:03.173340 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:03.173307 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:35:06.365773 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:06.365729 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-e58cbfec-openshift-default-7d889f85fc-vk2c7" Jul 06 19:35:07.443271 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:07.443239 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:35:07.547941 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:07.547910 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:35:07.648793 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:07.648754 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-afb205a6-openshift-default-6985dc9b6c-l54dx"] Jul 06 19:35:07.653544 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:07.653520 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-afb205a6-openshift-default-6985dc9b6c-l54dx" Jul 06 19:35:07.657590 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:07.657553 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-afb205a6-openshift-default-dockercfg-nxb5r\"" Jul 06 19:35:07.666709 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:07.666671 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-afb205a6-openshift-default-6985dc9b6c-l54dx"] Jul 06 19:35:07.681589 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:07.681563 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:35:07.683767 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:07.683733 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c9eaa7f1-6820-4ec9-8856-8c0128318dab-workload-certs\") pod \"e2e-derive-afb205a6-openshift-default-6985dc9b6c-l54dx\" (UID: \"c9eaa7f1-6820-4ec9-8856-8c0128318dab\") " pod="openshift-ingress/e2e-derive-afb205a6-openshift-default-6985dc9b6c-l54dx" Jul 06 19:35:07.683882 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:07.683777 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c9eaa7f1-6820-4ec9-8856-8c0128318dab-istiod-ca-cert\") pod \"e2e-derive-afb205a6-openshift-default-6985dc9b6c-l54dx\" (UID: \"c9eaa7f1-6820-4ec9-8856-8c0128318dab\") " pod="openshift-ingress/e2e-derive-afb205a6-openshift-default-6985dc9b6c-l54dx" Jul 06 19:35:07.683882 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:07.683843 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c9eaa7f1-6820-4ec9-8856-8c0128318dab-istio-data\") pod \"e2e-derive-afb205a6-openshift-default-6985dc9b6c-l54dx\" (UID: \"c9eaa7f1-6820-4ec9-8856-8c0128318dab\") " pod="openshift-ingress/e2e-derive-afb205a6-openshift-default-6985dc9b6c-l54dx" Jul 06 19:35:07.683882 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:07.683878 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c9eaa7f1-6820-4ec9-8856-8c0128318dab-istio-envoy\") pod \"e2e-derive-afb205a6-openshift-default-6985dc9b6c-l54dx\" (UID: \"c9eaa7f1-6820-4ec9-8856-8c0128318dab\") " pod="openshift-ingress/e2e-derive-afb205a6-openshift-default-6985dc9b6c-l54dx" Jul 06 19:35:07.684081 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:07.683898 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8r8l4\" (UniqueName: \"kubernetes.io/projected/c9eaa7f1-6820-4ec9-8856-8c0128318dab-kube-api-access-8r8l4\") pod \"e2e-derive-afb205a6-openshift-default-6985dc9b6c-l54dx\" (UID: \"c9eaa7f1-6820-4ec9-8856-8c0128318dab\") " pod="openshift-ingress/e2e-derive-afb205a6-openshift-default-6985dc9b6c-l54dx" Jul 06 19:35:07.684081 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:07.683968 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c9eaa7f1-6820-4ec9-8856-8c0128318dab-istio-podinfo\") pod \"e2e-derive-afb205a6-openshift-default-6985dc9b6c-l54dx\" (UID: \"c9eaa7f1-6820-4ec9-8856-8c0128318dab\") " pod="openshift-ingress/e2e-derive-afb205a6-openshift-default-6985dc9b6c-l54dx" Jul 06 19:35:07.684081 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:07.683993 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9eaa7f1-6820-4ec9-8856-8c0128318dab-credential-socket\") pod \"e2e-derive-afb205a6-openshift-default-6985dc9b6c-l54dx\" (UID: \"c9eaa7f1-6820-4ec9-8856-8c0128318dab\") " pod="openshift-ingress/e2e-derive-afb205a6-openshift-default-6985dc9b6c-l54dx" Jul 06 19:35:07.684081 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:07.684045 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c9eaa7f1-6820-4ec9-8856-8c0128318dab-istio-token\") pod \"e2e-derive-afb205a6-openshift-default-6985dc9b6c-l54dx\" (UID: \"c9eaa7f1-6820-4ec9-8856-8c0128318dab\") " pod="openshift-ingress/e2e-derive-afb205a6-openshift-default-6985dc9b6c-l54dx" Jul 06 19:35:07.684299 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:07.684130 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9eaa7f1-6820-4ec9-8856-8c0128318dab-workload-socket\") pod \"e2e-derive-afb205a6-openshift-default-6985dc9b6c-l54dx\" (UID: \"c9eaa7f1-6820-4ec9-8856-8c0128318dab\") " pod="openshift-ingress/e2e-derive-afb205a6-openshift-default-6985dc9b6c-l54dx" Jul 06 19:35:07.700798 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:07.700738 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-e58cbfec-openshift-default-7d889f85fc-vk2c7" Jul 06 19:35:07.749986 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:07.749946 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:35:07.785161 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:07.785124 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/767926fa-2c86-4bc1-a653-c0b25367ade2-workload-certs\") pod \"767926fa-2c86-4bc1-a653-c0b25367ade2\" (UID: \"767926fa-2c86-4bc1-a653-c0b25367ade2\") " Jul 06 19:35:07.785161 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:07.785160 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/767926fa-2c86-4bc1-a653-c0b25367ade2-istio-envoy\") pod \"767926fa-2c86-4bc1-a653-c0b25367ade2\" (UID: \"767926fa-2c86-4bc1-a653-c0b25367ade2\") " Jul 06 19:35:07.785389 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:07.785193 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/767926fa-2c86-4bc1-a653-c0b25367ade2-credential-socket\") pod \"767926fa-2c86-4bc1-a653-c0b25367ade2\" (UID: \"767926fa-2c86-4bc1-a653-c0b25367ade2\") " Jul 06 19:35:07.785389 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:07.785210 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d67qb\" (UniqueName: \"kubernetes.io/projected/767926fa-2c86-4bc1-a653-c0b25367ade2-kube-api-access-d67qb\") pod \"767926fa-2c86-4bc1-a653-c0b25367ade2\" (UID: \"767926fa-2c86-4bc1-a653-c0b25367ade2\") " Jul 06 19:35:07.785389 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:07.785247 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/767926fa-2c86-4bc1-a653-c0b25367ade2-istio-token\") pod \"767926fa-2c86-4bc1-a653-c0b25367ade2\" (UID: \"767926fa-2c86-4bc1-a653-c0b25367ade2\") " Jul 06 19:35:07.785389 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:07.785297 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/767926fa-2c86-4bc1-a653-c0b25367ade2-istio-podinfo\") pod \"767926fa-2c86-4bc1-a653-c0b25367ade2\" (UID: \"767926fa-2c86-4bc1-a653-c0b25367ade2\") " Jul 06 19:35:07.785389 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:07.785323 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/767926fa-2c86-4bc1-a653-c0b25367ade2-workload-socket\") pod \"767926fa-2c86-4bc1-a653-c0b25367ade2\" (UID: \"767926fa-2c86-4bc1-a653-c0b25367ade2\") " Jul 06 19:35:07.785626 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:07.785514 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/767926fa-2c86-4bc1-a653-c0b25367ade2-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "767926fa-2c86-4bc1-a653-c0b25367ade2" (UID: "767926fa-2c86-4bc1-a653-c0b25367ade2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:35:07.785626 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:07.785522 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/767926fa-2c86-4bc1-a653-c0b25367ade2-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "767926fa-2c86-4bc1-a653-c0b25367ade2" (UID: "767926fa-2c86-4bc1-a653-c0b25367ade2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:35:07.785626 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:07.785600 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/767926fa-2c86-4bc1-a653-c0b25367ade2-istio-data\") pod \"767926fa-2c86-4bc1-a653-c0b25367ade2\" (UID: \"767926fa-2c86-4bc1-a653-c0b25367ade2\") " Jul 06 19:35:07.785626 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:07.785618 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/767926fa-2c86-4bc1-a653-c0b25367ade2-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "767926fa-2c86-4bc1-a653-c0b25367ade2" (UID: "767926fa-2c86-4bc1-a653-c0b25367ade2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:35:07.785830 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:07.785676 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/767926fa-2c86-4bc1-a653-c0b25367ade2-istiod-ca-cert\") pod \"767926fa-2c86-4bc1-a653-c0b25367ade2\" (UID: \"767926fa-2c86-4bc1-a653-c0b25367ade2\") " Jul 06 19:35:07.785884 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:07.785831 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c9eaa7f1-6820-4ec9-8856-8c0128318dab-istio-data\") pod \"e2e-derive-afb205a6-openshift-default-6985dc9b6c-l54dx\" (UID: \"c9eaa7f1-6820-4ec9-8856-8c0128318dab\") " pod="openshift-ingress/e2e-derive-afb205a6-openshift-default-6985dc9b6c-l54dx" Jul 06 19:35:07.785884 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:07.785873 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c9eaa7f1-6820-4ec9-8856-8c0128318dab-istio-envoy\") pod \"e2e-derive-afb205a6-openshift-default-6985dc9b6c-l54dx\" (UID: \"c9eaa7f1-6820-4ec9-8856-8c0128318dab\") " pod="openshift-ingress/e2e-derive-afb205a6-openshift-default-6985dc9b6c-l54dx" Jul 06 19:35:07.785992 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:07.785899 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8r8l4\" (UniqueName: \"kubernetes.io/projected/c9eaa7f1-6820-4ec9-8856-8c0128318dab-kube-api-access-8r8l4\") pod \"e2e-derive-afb205a6-openshift-default-6985dc9b6c-l54dx\" (UID: \"c9eaa7f1-6820-4ec9-8856-8c0128318dab\") " pod="openshift-ingress/e2e-derive-afb205a6-openshift-default-6985dc9b6c-l54dx" Jul 06 19:35:07.785992 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:07.785972 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c9eaa7f1-6820-4ec9-8856-8c0128318dab-istio-podinfo\") pod \"e2e-derive-afb205a6-openshift-default-6985dc9b6c-l54dx\" (UID: \"c9eaa7f1-6820-4ec9-8856-8c0128318dab\") " pod="openshift-ingress/e2e-derive-afb205a6-openshift-default-6985dc9b6c-l54dx" Jul 06 19:35:07.786141 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:07.786008 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9eaa7f1-6820-4ec9-8856-8c0128318dab-credential-socket\") pod \"e2e-derive-afb205a6-openshift-default-6985dc9b6c-l54dx\" (UID: \"c9eaa7f1-6820-4ec9-8856-8c0128318dab\") " pod="openshift-ingress/e2e-derive-afb205a6-openshift-default-6985dc9b6c-l54dx" Jul 06 19:35:07.786141 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:07.786117 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/767926fa-2c86-4bc1-a653-c0b25367ade2-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "767926fa-2c86-4bc1-a653-c0b25367ade2" (UID: "767926fa-2c86-4bc1-a653-c0b25367ade2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:35:07.786250 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:07.786154 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c9eaa7f1-6820-4ec9-8856-8c0128318dab-istio-token\") pod \"e2e-derive-afb205a6-openshift-default-6985dc9b6c-l54dx\" (UID: \"c9eaa7f1-6820-4ec9-8856-8c0128318dab\") " pod="openshift-ingress/e2e-derive-afb205a6-openshift-default-6985dc9b6c-l54dx" Jul 06 19:35:07.786318 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:07.786301 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9eaa7f1-6820-4ec9-8856-8c0128318dab-workload-socket\") pod \"e2e-derive-afb205a6-openshift-default-6985dc9b6c-l54dx\" (UID: \"c9eaa7f1-6820-4ec9-8856-8c0128318dab\") " pod="openshift-ingress/e2e-derive-afb205a6-openshift-default-6985dc9b6c-l54dx" Jul 06 19:35:07.786374 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:07.786358 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c9eaa7f1-6820-4ec9-8856-8c0128318dab-workload-certs\") pod \"e2e-derive-afb205a6-openshift-default-6985dc9b6c-l54dx\" (UID: \"c9eaa7f1-6820-4ec9-8856-8c0128318dab\") " pod="openshift-ingress/e2e-derive-afb205a6-openshift-default-6985dc9b6c-l54dx" Jul 06 19:35:07.786428 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:07.786385 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c9eaa7f1-6820-4ec9-8856-8c0128318dab-istiod-ca-cert\") pod \"e2e-derive-afb205a6-openshift-default-6985dc9b6c-l54dx\" (UID: \"c9eaa7f1-6820-4ec9-8856-8c0128318dab\") " pod="openshift-ingress/e2e-derive-afb205a6-openshift-default-6985dc9b6c-l54dx" Jul 06 19:35:07.786483 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:07.786444 2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/767926fa-2c86-4bc1-a653-c0b25367ade2-workload-socket\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:35:07.786483 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:07.786462 2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/767926fa-2c86-4bc1-a653-c0b25367ade2-istiod-ca-cert\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:35:07.786483 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:07.786477 2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/767926fa-2c86-4bc1-a653-c0b25367ade2-workload-certs\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:35:07.786631 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:07.786491 2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/767926fa-2c86-4bc1-a653-c0b25367ade2-credential-socket\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:35:07.787116 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:07.786984 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c9eaa7f1-6820-4ec9-8856-8c0128318dab-istio-data\") pod \"e2e-derive-afb205a6-openshift-default-6985dc9b6c-l54dx\" (UID: \"c9eaa7f1-6820-4ec9-8856-8c0128318dab\") " pod="openshift-ingress/e2e-derive-afb205a6-openshift-default-6985dc9b6c-l54dx" Jul 06 19:35:07.787116 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:07.787062 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/767926fa-2c86-4bc1-a653-c0b25367ade2-istio-data" (OuterVolumeSpecName: "istio-data") pod "767926fa-2c86-4bc1-a653-c0b25367ade2" (UID: "767926fa-2c86-4bc1-a653-c0b25367ade2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:35:07.787291 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:07.787214 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c9eaa7f1-6820-4ec9-8856-8c0128318dab-workload-certs\") pod \"e2e-derive-afb205a6-openshift-default-6985dc9b6c-l54dx\" (UID: \"c9eaa7f1-6820-4ec9-8856-8c0128318dab\") " pod="openshift-ingress/e2e-derive-afb205a6-openshift-default-6985dc9b6c-l54dx" Jul 06 19:35:07.787401 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:07.787283 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9eaa7f1-6820-4ec9-8856-8c0128318dab-workload-socket\") pod \"e2e-derive-afb205a6-openshift-default-6985dc9b6c-l54dx\" (UID: \"c9eaa7f1-6820-4ec9-8856-8c0128318dab\") " pod="openshift-ingress/e2e-derive-afb205a6-openshift-default-6985dc9b6c-l54dx" Jul 06 19:35:07.787672 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:07.787632 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9eaa7f1-6820-4ec9-8856-8c0128318dab-credential-socket\") pod \"e2e-derive-afb205a6-openshift-default-6985dc9b6c-l54dx\" (UID: \"c9eaa7f1-6820-4ec9-8856-8c0128318dab\") " pod="openshift-ingress/e2e-derive-afb205a6-openshift-default-6985dc9b6c-l54dx" Jul 06 19:35:07.788014 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:07.787994 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c9eaa7f1-6820-4ec9-8856-8c0128318dab-istiod-ca-cert\") pod \"e2e-derive-afb205a6-openshift-default-6985dc9b6c-l54dx\" (UID: \"c9eaa7f1-6820-4ec9-8856-8c0128318dab\") " pod="openshift-ingress/e2e-derive-afb205a6-openshift-default-6985dc9b6c-l54dx" Jul 06 19:35:07.788427 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:07.788393 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/767926fa-2c86-4bc1-a653-c0b25367ade2-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "767926fa-2c86-4bc1-a653-c0b25367ade2" (UID: "767926fa-2c86-4bc1-a653-c0b25367ade2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:35:07.788625 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:07.788588 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/767926fa-2c86-4bc1-a653-c0b25367ade2-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "767926fa-2c86-4bc1-a653-c0b25367ade2" (UID: "767926fa-2c86-4bc1-a653-c0b25367ade2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 19:35:07.788625 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:07.788606 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/767926fa-2c86-4bc1-a653-c0b25367ade2-istio-token" (OuterVolumeSpecName: "istio-token") pod "767926fa-2c86-4bc1-a653-c0b25367ade2" (UID: "767926fa-2c86-4bc1-a653-c0b25367ade2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:35:07.788814 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:07.788649 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/767926fa-2c86-4bc1-a653-c0b25367ade2-kube-api-access-d67qb" (OuterVolumeSpecName: "kube-api-access-d67qb") pod "767926fa-2c86-4bc1-a653-c0b25367ade2" (UID: "767926fa-2c86-4bc1-a653-c0b25367ade2"). InnerVolumeSpecName "kube-api-access-d67qb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:35:07.789353 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:07.789324 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c9eaa7f1-6820-4ec9-8856-8c0128318dab-istio-envoy\") pod \"e2e-derive-afb205a6-openshift-default-6985dc9b6c-l54dx\" (UID: \"c9eaa7f1-6820-4ec9-8856-8c0128318dab\") " pod="openshift-ingress/e2e-derive-afb205a6-openshift-default-6985dc9b6c-l54dx" Jul 06 19:35:07.789569 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:07.789554 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c9eaa7f1-6820-4ec9-8856-8c0128318dab-istio-podinfo\") pod \"e2e-derive-afb205a6-openshift-default-6985dc9b6c-l54dx\" (UID: \"c9eaa7f1-6820-4ec9-8856-8c0128318dab\") " pod="openshift-ingress/e2e-derive-afb205a6-openshift-default-6985dc9b6c-l54dx" Jul 06 19:35:07.793865 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:07.793845 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8r8l4\" (UniqueName: \"kubernetes.io/projected/c9eaa7f1-6820-4ec9-8856-8c0128318dab-kube-api-access-8r8l4\") pod \"e2e-derive-afb205a6-openshift-default-6985dc9b6c-l54dx\" (UID: \"c9eaa7f1-6820-4ec9-8856-8c0128318dab\") " pod="openshift-ingress/e2e-derive-afb205a6-openshift-default-6985dc9b6c-l54dx" Jul 06 19:35:07.794004 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:07.793988 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c9eaa7f1-6820-4ec9-8856-8c0128318dab-istio-token\") pod \"e2e-derive-afb205a6-openshift-default-6985dc9b6c-l54dx\" (UID: \"c9eaa7f1-6820-4ec9-8856-8c0128318dab\") " pod="openshift-ingress/e2e-derive-afb205a6-openshift-default-6985dc9b6c-l54dx" Jul 06 19:35:07.887815 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:07.887786 2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/767926fa-2c86-4bc1-a653-c0b25367ade2-istio-envoy\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:35:07.887815 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:07.887812 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d67qb\" (UniqueName: \"kubernetes.io/projected/767926fa-2c86-4bc1-a653-c0b25367ade2-kube-api-access-d67qb\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:35:07.887815 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:07.887823 2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/767926fa-2c86-4bc1-a653-c0b25367ade2-istio-token\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:35:07.888046 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:07.887831 2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/767926fa-2c86-4bc1-a653-c0b25367ade2-istio-podinfo\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:35:07.888046 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:07.887839 2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/767926fa-2c86-4bc1-a653-c0b25367ade2-istio-data\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:35:07.967724 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:07.967626 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-afb205a6-openshift-default-6985dc9b6c-l54dx" Jul 06 19:35:08.113922 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:08.113890 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-afb205a6-openshift-default-6985dc9b6c-l54dx"] Jul 06 19:35:08.115272 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:35:08.115246 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9eaa7f1_6820_4ec9_8856_8c0128318dab.slice/crio-47d7d04ac21d2ab3028c0b9d5f366f62a56ff00ff1475c12ce010fd57236e0ca WatchSource:0}: Error finding container 47d7d04ac21d2ab3028c0b9d5f366f62a56ff00ff1475c12ce010fd57236e0ca: Status 404 returned error can't find the container with id 47d7d04ac21d2ab3028c0b9d5f366f62a56ff00ff1475c12ce010fd57236e0ca Jul 06 19:35:08.117294 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:08.117257 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 19:35:08.117401 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:08.117337 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 19:35:08.117401 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:08.117364 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 19:35:08.473608 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:08.473574 2566 generic.go:356] "Generic (PLEG): container finished" podID="767926fa-2c86-4bc1-a653-c0b25367ade2" containerID="3078d493754664d82ea932272f2509bf9a46d07374ca6c54770265c25a7ee439" exitCode=0 Jul 06 19:35:08.474102 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:08.473630 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-e58cbfec-openshift-default-7d889f85fc-vk2c7" Jul 06 19:35:08.474102 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:08.473656 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e58cbfec-openshift-default-7d889f85fc-vk2c7" event={"ID":"767926fa-2c86-4bc1-a653-c0b25367ade2","Type":"ContainerDied","Data":"3078d493754664d82ea932272f2509bf9a46d07374ca6c54770265c25a7ee439"} Jul 06 19:35:08.474102 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:08.473707 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-e58cbfec-openshift-default-7d889f85fc-vk2c7" event={"ID":"767926fa-2c86-4bc1-a653-c0b25367ade2","Type":"ContainerDied","Data":"f0c6963b990d9549149280f6cedc141089e7093425484d167e53dfc7af9c9fce"} Jul 06 19:35:08.474102 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:08.473731 2566 scope.go:117] "RemoveContainer" containerID="3078d493754664d82ea932272f2509bf9a46d07374ca6c54770265c25a7ee439" Jul 06 19:35:08.475928 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:08.475906 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-afb205a6-openshift-default-6985dc9b6c-l54dx" event={"ID":"c9eaa7f1-6820-4ec9-8856-8c0128318dab","Type":"ContainerStarted","Data":"8cc38959d60395d4766f6a2d926d07d2db7163da159a154854cf85f80704f8ec"} Jul 06 19:35:08.476096 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:08.475953 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-afb205a6-openshift-default-6985dc9b6c-l54dx" event={"ID":"c9eaa7f1-6820-4ec9-8856-8c0128318dab","Type":"ContainerStarted","Data":"47d7d04ac21d2ab3028c0b9d5f366f62a56ff00ff1475c12ce010fd57236e0ca"} Jul 06 19:35:08.484933 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:08.484910 2566 scope.go:117] "RemoveContainer" containerID="a2274aa5607776b92c8f3db06a9426c187ef47ccb80446bc98ec6673dc321a59" Jul 06 19:35:08.493604 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:08.493564 2566 scope.go:117] "RemoveContainer" containerID="3078d493754664d82ea932272f2509bf9a46d07374ca6c54770265c25a7ee439" Jul 06 19:35:08.493899 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:35:08.493875 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3078d493754664d82ea932272f2509bf9a46d07374ca6c54770265c25a7ee439\": container with ID starting with 3078d493754664d82ea932272f2509bf9a46d07374ca6c54770265c25a7ee439 not found: ID does not exist" containerID="3078d493754664d82ea932272f2509bf9a46d07374ca6c54770265c25a7ee439" Jul 06 19:35:08.493990 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:08.493896 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3078d493754664d82ea932272f2509bf9a46d07374ca6c54770265c25a7ee439"} err="failed to get container status \"3078d493754664d82ea932272f2509bf9a46d07374ca6c54770265c25a7ee439\": rpc error: code = NotFound desc = could not find container \"3078d493754664d82ea932272f2509bf9a46d07374ca6c54770265c25a7ee439\": container with ID starting with 3078d493754664d82ea932272f2509bf9a46d07374ca6c54770265c25a7ee439 not found: ID does not exist" Jul 06 19:35:08.493990 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:08.493913 2566 scope.go:117] "RemoveContainer" containerID="a2274aa5607776b92c8f3db06a9426c187ef47ccb80446bc98ec6673dc321a59" Jul 06 19:35:08.494253 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:35:08.494225 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a2274aa5607776b92c8f3db06a9426c187ef47ccb80446bc98ec6673dc321a59\": container with ID starting with a2274aa5607776b92c8f3db06a9426c187ef47ccb80446bc98ec6673dc321a59 not found: ID does not exist" containerID="a2274aa5607776b92c8f3db06a9426c187ef47ccb80446bc98ec6673dc321a59" Jul 06 19:35:08.494344 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:08.494261 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a2274aa5607776b92c8f3db06a9426c187ef47ccb80446bc98ec6673dc321a59"} err="failed to get container status \"a2274aa5607776b92c8f3db06a9426c187ef47ccb80446bc98ec6673dc321a59\": rpc error: code = NotFound desc = could not find container \"a2274aa5607776b92c8f3db06a9426c187ef47ccb80446bc98ec6673dc321a59\": container with ID starting with a2274aa5607776b92c8f3db06a9426c187ef47ccb80446bc98ec6673dc321a59 not found: ID does not exist" Jul 06 19:35:08.502610 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:08.502554 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-afb205a6-openshift-default-6985dc9b6c-l54dx" podStartSLOduration=1.502541303 podStartE2EDuration="1.502541303s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:35:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:35:08.501900532 +0000 UTC m=+1533.809259479" watchObservedRunningTime="2026-07-06 19:35:08.502541303 +0000 UTC m=+1533.809900251" Jul 06 19:35:08.517629 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:08.517605 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-e58cbfec-openshift-default-7d889f85fc-vk2c7"] Jul 06 19:35:08.522832 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:08.522811 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-e58cbfec-openshift-default-7d889f85fc-vk2c7"] Jul 06 19:35:08.963779 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:08.963747 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:35:08.968428 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:08.968415 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-afb205a6-openshift-default-6985dc9b6c-l54dx" Jul 06 19:35:09.060282 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:09.060247 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:35:09.233582 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:09.233500 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:35:09.239008 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:09.238975 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="767926fa-2c86-4bc1-a653-c0b25367ade2" path="/var/lib/kubelet/pods/767926fa-2c86-4bc1-a653-c0b25367ade2/volumes" Jul 06 19:35:09.595351 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:09.595295 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:35:09.698449 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:09.698408 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:35:09.974260 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:09.974191 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-afb205a6-openshift-default-6985dc9b6c-l54dx" Jul 06 19:35:10.489010 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:10.488978 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-afb205a6-openshift-default-6985dc9b6c-l54dx" Jul 06 19:35:10.490130 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:10.490113 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-afb205a6-openshift-default-6985dc9b6c-l54dx" Jul 06 19:35:11.117593 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:11.117562 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:35:13.240476 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:13.240423 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:35:13.397331 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:13.397302 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-afb205a6-openshift-default-6985dc9b6c-l54dx"] Jul 06 19:35:13.427365 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:13.427337 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:35:13.505168 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:13.505039 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-afb205a6-openshift-default-6985dc9b6c-l54dx" podUID="c9eaa7f1-6820-4ec9-8856-8c0128318dab" containerName="istio-proxy" containerID="cri-o://8cc38959d60395d4766f6a2d926d07d2db7163da159a154854cf85f80704f8ec" gracePeriod=30 Jul 06 19:35:13.564120 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:13.564056 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:35:13.766517 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:13.766429 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:35:15.424943 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:15.424911 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:35:19.353012 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:19.352981 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-afb205a6-openshift-default-6985dc9b6c-l54dx" Jul 06 19:35:19.404152 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:19.404035 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c9eaa7f1-6820-4ec9-8856-8c0128318dab-workload-certs\") pod \"c9eaa7f1-6820-4ec9-8856-8c0128318dab\" (UID: \"c9eaa7f1-6820-4ec9-8856-8c0128318dab\") " Jul 06 19:35:19.404152 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:19.404096 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9eaa7f1-6820-4ec9-8856-8c0128318dab-workload-socket\") pod \"c9eaa7f1-6820-4ec9-8856-8c0128318dab\" (UID: \"c9eaa7f1-6820-4ec9-8856-8c0128318dab\") " Jul 06 19:35:19.404366 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:19.404156 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c9eaa7f1-6820-4ec9-8856-8c0128318dab-istio-data\") pod \"c9eaa7f1-6820-4ec9-8856-8c0128318dab\" (UID: \"c9eaa7f1-6820-4ec9-8856-8c0128318dab\") " Jul 06 19:35:19.404366 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:19.404183 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9eaa7f1-6820-4ec9-8856-8c0128318dab-credential-socket\") pod \"c9eaa7f1-6820-4ec9-8856-8c0128318dab\" (UID: \"c9eaa7f1-6820-4ec9-8856-8c0128318dab\") " Jul 06 19:35:19.404366 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:19.404268 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8r8l4\" (UniqueName: \"kubernetes.io/projected/c9eaa7f1-6820-4ec9-8856-8c0128318dab-kube-api-access-8r8l4\") pod \"c9eaa7f1-6820-4ec9-8856-8c0128318dab\" (UID: \"c9eaa7f1-6820-4ec9-8856-8c0128318dab\") " Jul 06 19:35:19.404366 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:19.404297 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c9eaa7f1-6820-4ec9-8856-8c0128318dab-istio-envoy\") pod \"c9eaa7f1-6820-4ec9-8856-8c0128318dab\" (UID: \"c9eaa7f1-6820-4ec9-8856-8c0128318dab\") " Jul 06 19:35:19.404366 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:19.404327 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c9eaa7f1-6820-4ec9-8856-8c0128318dab-istio-podinfo\") pod \"c9eaa7f1-6820-4ec9-8856-8c0128318dab\" (UID: \"c9eaa7f1-6820-4ec9-8856-8c0128318dab\") " Jul 06 19:35:19.404366 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:19.404326 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c9eaa7f1-6820-4ec9-8856-8c0128318dab-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c9eaa7f1-6820-4ec9-8856-8c0128318dab" (UID: "c9eaa7f1-6820-4ec9-8856-8c0128318dab"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:35:19.404366 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:19.404353 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c9eaa7f1-6820-4ec9-8856-8c0128318dab-istio-token\") pod \"c9eaa7f1-6820-4ec9-8856-8c0128318dab\" (UID: \"c9eaa7f1-6820-4ec9-8856-8c0128318dab\") " Jul 06 19:35:19.404731 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:19.404390 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c9eaa7f1-6820-4ec9-8856-8c0128318dab-istiod-ca-cert\") pod \"c9eaa7f1-6820-4ec9-8856-8c0128318dab\" (UID: \"c9eaa7f1-6820-4ec9-8856-8c0128318dab\") " Jul 06 19:35:19.404731 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:19.404430 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c9eaa7f1-6820-4ec9-8856-8c0128318dab-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c9eaa7f1-6820-4ec9-8856-8c0128318dab" (UID: "c9eaa7f1-6820-4ec9-8856-8c0128318dab"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:35:19.404731 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:19.404703 2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c9eaa7f1-6820-4ec9-8856-8c0128318dab-workload-certs\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:35:19.404731 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:19.404722 2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9eaa7f1-6820-4ec9-8856-8c0128318dab-workload-socket\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:35:19.404937 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:19.404831 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c9eaa7f1-6820-4ec9-8856-8c0128318dab-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c9eaa7f1-6820-4ec9-8856-8c0128318dab" (UID: "c9eaa7f1-6820-4ec9-8856-8c0128318dab"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:35:19.404937 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:19.404860 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c9eaa7f1-6820-4ec9-8856-8c0128318dab-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c9eaa7f1-6820-4ec9-8856-8c0128318dab" (UID: "c9eaa7f1-6820-4ec9-8856-8c0128318dab"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:35:19.405286 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:19.405245 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c9eaa7f1-6820-4ec9-8856-8c0128318dab-istio-data" (OuterVolumeSpecName: "istio-data") pod "c9eaa7f1-6820-4ec9-8856-8c0128318dab" (UID: "c9eaa7f1-6820-4ec9-8856-8c0128318dab"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:35:19.406735 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:19.406706 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c9eaa7f1-6820-4ec9-8856-8c0128318dab-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c9eaa7f1-6820-4ec9-8856-8c0128318dab" (UID: "c9eaa7f1-6820-4ec9-8856-8c0128318dab"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:35:19.407156 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:19.407130 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c9eaa7f1-6820-4ec9-8856-8c0128318dab-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c9eaa7f1-6820-4ec9-8856-8c0128318dab" (UID: "c9eaa7f1-6820-4ec9-8856-8c0128318dab"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 19:35:19.407272 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:19.407224 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c9eaa7f1-6820-4ec9-8856-8c0128318dab-kube-api-access-8r8l4" (OuterVolumeSpecName: "kube-api-access-8r8l4") pod "c9eaa7f1-6820-4ec9-8856-8c0128318dab" (UID: "c9eaa7f1-6820-4ec9-8856-8c0128318dab"). InnerVolumeSpecName "kube-api-access-8r8l4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:35:19.407272 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:19.407234 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c9eaa7f1-6820-4ec9-8856-8c0128318dab-istio-token" (OuterVolumeSpecName: "istio-token") pod "c9eaa7f1-6820-4ec9-8856-8c0128318dab" (UID: "c9eaa7f1-6820-4ec9-8856-8c0128318dab"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:35:19.505792 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:19.505753 2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c9eaa7f1-6820-4ec9-8856-8c0128318dab-istio-data\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:35:19.505792 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:19.505785 2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9eaa7f1-6820-4ec9-8856-8c0128318dab-credential-socket\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:35:19.505792 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:19.505794 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8r8l4\" (UniqueName: \"kubernetes.io/projected/c9eaa7f1-6820-4ec9-8856-8c0128318dab-kube-api-access-8r8l4\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:35:19.505792 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:19.505803 2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c9eaa7f1-6820-4ec9-8856-8c0128318dab-istio-envoy\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:35:19.506055 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:19.505811 2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c9eaa7f1-6820-4ec9-8856-8c0128318dab-istio-podinfo\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:35:19.506055 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:19.505820 2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c9eaa7f1-6820-4ec9-8856-8c0128318dab-istio-token\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:35:19.506055 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:19.505827 2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c9eaa7f1-6820-4ec9-8856-8c0128318dab-istiod-ca-cert\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:35:19.542970 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:19.542934 2566 generic.go:356] "Generic (PLEG): container finished" podID="c9eaa7f1-6820-4ec9-8856-8c0128318dab" containerID="8cc38959d60395d4766f6a2d926d07d2db7163da159a154854cf85f80704f8ec" exitCode=0 Jul 06 19:35:19.543154 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:19.542987 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-afb205a6-openshift-default-6985dc9b6c-l54dx" Jul 06 19:35:19.543154 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:19.542984 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-afb205a6-openshift-default-6985dc9b6c-l54dx" event={"ID":"c9eaa7f1-6820-4ec9-8856-8c0128318dab","Type":"ContainerDied","Data":"8cc38959d60395d4766f6a2d926d07d2db7163da159a154854cf85f80704f8ec"} Jul 06 19:35:19.543154 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:19.543046 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-afb205a6-openshift-default-6985dc9b6c-l54dx" event={"ID":"c9eaa7f1-6820-4ec9-8856-8c0128318dab","Type":"ContainerDied","Data":"47d7d04ac21d2ab3028c0b9d5f366f62a56ff00ff1475c12ce010fd57236e0ca"} Jul 06 19:35:19.543154 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:19.543086 2566 scope.go:117] "RemoveContainer" containerID="8cc38959d60395d4766f6a2d926d07d2db7163da159a154854cf85f80704f8ec" Jul 06 19:35:19.553818 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:19.553799 2566 scope.go:117] "RemoveContainer" containerID="8cc38959d60395d4766f6a2d926d07d2db7163da159a154854cf85f80704f8ec" Jul 06 19:35:19.554115 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:35:19.554094 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8cc38959d60395d4766f6a2d926d07d2db7163da159a154854cf85f80704f8ec\": container with ID starting with 8cc38959d60395d4766f6a2d926d07d2db7163da159a154854cf85f80704f8ec not found: ID does not exist" containerID="8cc38959d60395d4766f6a2d926d07d2db7163da159a154854cf85f80704f8ec" Jul 06 19:35:19.554177 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:19.554122 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8cc38959d60395d4766f6a2d926d07d2db7163da159a154854cf85f80704f8ec"} err="failed to get container status \"8cc38959d60395d4766f6a2d926d07d2db7163da159a154854cf85f80704f8ec\": rpc error: code = NotFound desc = could not find container \"8cc38959d60395d4766f6a2d926d07d2db7163da159a154854cf85f80704f8ec\": container with ID starting with 8cc38959d60395d4766f6a2d926d07d2db7163da159a154854cf85f80704f8ec not found: ID does not exist" Jul 06 19:35:19.566855 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:19.566825 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-afb205a6-openshift-default-6985dc9b6c-l54dx"] Jul 06 19:35:19.573847 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:19.573822 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-afb205a6-openshift-default-6985dc9b6c-l54dx"] Jul 06 19:35:19.589892 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:19.589867 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:35:19.681542 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:19.681463 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:35:20.067389 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:20.067359 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:35:20.166058 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:20.166027 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:35:20.264752 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:20.264721 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:35:21.237443 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:21.237413 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c9eaa7f1-6820-4ec9-8856-8c0128318dab" path="/var/lib/kubelet/pods/c9eaa7f1-6820-4ec9-8856-8c0128318dab/volumes" Jul 06 19:35:21.659930 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:21.659895 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:35:27.448747 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:27.448716 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:35:28.688530 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:28.688496 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:35:54.092866 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:35:54.092774 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:36:00.174648 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:36:00.174608 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:36:00.281671 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:36:00.281639 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:36:00.399511 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:36:00.399470 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:36:16.286257 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:36:16.286220 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:36:22.311911 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:36:22.311878 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:36:22.417045 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:36:22.417011 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:36:22.521561 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:36:22.521521 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:36:33.034914 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:36:33.034832 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:36:33.864551 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:36:33.864507 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:36:39.866916 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:36:39.866881 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:36:40.016342 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:36:40.016309 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:36:40.219862 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:36:40.219779 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:36:40.317348 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:36:40.317322 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:36:40.968894 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:36:40.968835 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:36:41.103705 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:36:41.103676 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:36:41.202854 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:36:41.202825 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:36:42.779737 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:36:42.779707 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:36:48.489095 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:36:48.489038 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:36:48.589537 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:36:48.589506 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:36:49.096968 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:36:49.096934 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:36:49.198465 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:36:49.198426 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:36:49.853661 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:36:49.853620 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:36:55.967424 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:36:55.967381 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:36:56.047694 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:36:56.047664 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:36:56.228518 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:36:56.228431 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:36:56.508461 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:36:56.508366 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-02f24e54-openshift-default-6dc6dd4cf9-pjcsw"] Jul 06 19:36:56.509363 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:36:56.509340 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c9eaa7f1-6820-4ec9-8856-8c0128318dab" containerName="istio-proxy" Jul 06 19:36:56.509464 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:36:56.509366 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="c9eaa7f1-6820-4ec9-8856-8c0128318dab" containerName="istio-proxy" Jul 06 19:36:56.509464 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:36:56.509445 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="767926fa-2c86-4bc1-a653-c0b25367ade2" containerName="istio-proxy" Jul 06 19:36:56.509464 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:36:56.509454 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="767926fa-2c86-4bc1-a653-c0b25367ade2" containerName="istio-proxy" Jul 06 19:36:56.509623 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:36:56.509467 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="767926fa-2c86-4bc1-a653-c0b25367ade2" containerName="istio-proxy" Jul 06 19:36:56.509623 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:36:56.509476 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="767926fa-2c86-4bc1-a653-c0b25367ade2" containerName="istio-proxy" Jul 06 19:36:56.509723 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:36:56.509633 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="767926fa-2c86-4bc1-a653-c0b25367ade2" containerName="istio-proxy" Jul 06 19:36:56.509723 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:36:56.509653 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="c9eaa7f1-6820-4ec9-8856-8c0128318dab" containerName="istio-proxy" Jul 06 19:36:56.515498 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:36:56.515440 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-02f24e54-openshift-default-6dc6dd4cf9-pjcsw" Jul 06 19:36:56.518813 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:36:56.518773 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-02f24e54-openshift-default-dockercfg-rqw8n\"" Jul 06 19:36:56.534434 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:36:56.534397 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-02f24e54-openshift-default-6dc6dd4cf9-pjcsw"] Jul 06 19:36:56.541394 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:36:56.541369 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:36:56.585210 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:36:56.585163 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8-workload-socket\") pod \"e2e-mt-02f24e54-openshift-default-6dc6dd4cf9-pjcsw\" (UID: \"4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8\") " pod="openshift-ingress/e2e-mt-02f24e54-openshift-default-6dc6dd4cf9-pjcsw" Jul 06 19:36:56.585210 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:36:56.585210 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8-credential-socket\") pod \"e2e-mt-02f24e54-openshift-default-6dc6dd4cf9-pjcsw\" (UID: \"4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8\") " pod="openshift-ingress/e2e-mt-02f24e54-openshift-default-6dc6dd4cf9-pjcsw" Jul 06 19:36:56.585483 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:36:56.585237 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8-istio-podinfo\") pod \"e2e-mt-02f24e54-openshift-default-6dc6dd4cf9-pjcsw\" (UID: \"4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8\") " pod="openshift-ingress/e2e-mt-02f24e54-openshift-default-6dc6dd4cf9-pjcsw" Jul 06 19:36:56.585483 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:36:56.585316 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xkjlv\" (UniqueName: \"kubernetes.io/projected/4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8-kube-api-access-xkjlv\") pod \"e2e-mt-02f24e54-openshift-default-6dc6dd4cf9-pjcsw\" (UID: \"4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8\") " pod="openshift-ingress/e2e-mt-02f24e54-openshift-default-6dc6dd4cf9-pjcsw" Jul 06 19:36:56.585483 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:36:56.585349 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8-istio-token\") pod \"e2e-mt-02f24e54-openshift-default-6dc6dd4cf9-pjcsw\" (UID: \"4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8\") " pod="openshift-ingress/e2e-mt-02f24e54-openshift-default-6dc6dd4cf9-pjcsw" Jul 06 19:36:56.585483 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:36:56.585381 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8-istio-envoy\") pod \"e2e-mt-02f24e54-openshift-default-6dc6dd4cf9-pjcsw\" (UID: \"4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8\") " pod="openshift-ingress/e2e-mt-02f24e54-openshift-default-6dc6dd4cf9-pjcsw" Jul 06 19:36:56.585483 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:36:56.585429 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8-workload-certs\") pod \"e2e-mt-02f24e54-openshift-default-6dc6dd4cf9-pjcsw\" (UID: \"4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8\") " pod="openshift-ingress/e2e-mt-02f24e54-openshift-default-6dc6dd4cf9-pjcsw" Jul 06 19:36:56.585483 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:36:56.585485 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8-istiod-ca-cert\") pod \"e2e-mt-02f24e54-openshift-default-6dc6dd4cf9-pjcsw\" (UID: \"4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8\") " pod="openshift-ingress/e2e-mt-02f24e54-openshift-default-6dc6dd4cf9-pjcsw" Jul 06 19:36:56.585798 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:36:56.585503 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8-istio-data\") pod \"e2e-mt-02f24e54-openshift-default-6dc6dd4cf9-pjcsw\" (UID: \"4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8\") " pod="openshift-ingress/e2e-mt-02f24e54-openshift-default-6dc6dd4cf9-pjcsw" Jul 06 19:36:56.686685 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:36:56.686643 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8-istiod-ca-cert\") pod \"e2e-mt-02f24e54-openshift-default-6dc6dd4cf9-pjcsw\" (UID: \"4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8\") " pod="openshift-ingress/e2e-mt-02f24e54-openshift-default-6dc6dd4cf9-pjcsw" Jul 06 19:36:56.686685 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:36:56.686687 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8-istio-data\") pod \"e2e-mt-02f24e54-openshift-default-6dc6dd4cf9-pjcsw\" (UID: \"4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8\") " pod="openshift-ingress/e2e-mt-02f24e54-openshift-default-6dc6dd4cf9-pjcsw" Jul 06 19:36:56.686931 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:36:56.686731 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8-workload-socket\") pod \"e2e-mt-02f24e54-openshift-default-6dc6dd4cf9-pjcsw\" (UID: \"4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8\") " pod="openshift-ingress/e2e-mt-02f24e54-openshift-default-6dc6dd4cf9-pjcsw" Jul 06 19:36:56.686931 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:36:56.686753 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8-credential-socket\") pod \"e2e-mt-02f24e54-openshift-default-6dc6dd4cf9-pjcsw\" (UID: \"4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8\") " pod="openshift-ingress/e2e-mt-02f24e54-openshift-default-6dc6dd4cf9-pjcsw" Jul 06 19:36:56.686931 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:36:56.686768 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8-istio-podinfo\") pod \"e2e-mt-02f24e54-openshift-default-6dc6dd4cf9-pjcsw\" (UID: \"4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8\") " pod="openshift-ingress/e2e-mt-02f24e54-openshift-default-6dc6dd4cf9-pjcsw" Jul 06 19:36:56.686931 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:36:56.686785 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xkjlv\" (UniqueName: \"kubernetes.io/projected/4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8-kube-api-access-xkjlv\") pod \"e2e-mt-02f24e54-openshift-default-6dc6dd4cf9-pjcsw\" (UID: \"4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8\") " pod="openshift-ingress/e2e-mt-02f24e54-openshift-default-6dc6dd4cf9-pjcsw" Jul 06 19:36:56.686931 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:36:56.686804 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8-istio-token\") pod \"e2e-mt-02f24e54-openshift-default-6dc6dd4cf9-pjcsw\" (UID: \"4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8\") " pod="openshift-ingress/e2e-mt-02f24e54-openshift-default-6dc6dd4cf9-pjcsw" Jul 06 19:36:56.686931 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:36:56.686836 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8-istio-envoy\") pod \"e2e-mt-02f24e54-openshift-default-6dc6dd4cf9-pjcsw\" (UID: \"4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8\") " pod="openshift-ingress/e2e-mt-02f24e54-openshift-default-6dc6dd4cf9-pjcsw" Jul 06 19:36:56.686931 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:36:56.686870 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8-workload-certs\") pod \"e2e-mt-02f24e54-openshift-default-6dc6dd4cf9-pjcsw\" (UID: \"4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8\") " pod="openshift-ingress/e2e-mt-02f24e54-openshift-default-6dc6dd4cf9-pjcsw" Jul 06 19:36:56.687338 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:36:56.687268 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8-workload-socket\") pod \"e2e-mt-02f24e54-openshift-default-6dc6dd4cf9-pjcsw\" (UID: \"4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8\") " pod="openshift-ingress/e2e-mt-02f24e54-openshift-default-6dc6dd4cf9-pjcsw" Jul 06 19:36:56.687338 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:36:56.687313 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8-workload-certs\") pod \"e2e-mt-02f24e54-openshift-default-6dc6dd4cf9-pjcsw\" (UID: \"4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8\") " pod="openshift-ingress/e2e-mt-02f24e54-openshift-default-6dc6dd4cf9-pjcsw" Jul 06 19:36:56.687445 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:36:56.687393 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8-credential-socket\") pod \"e2e-mt-02f24e54-openshift-default-6dc6dd4cf9-pjcsw\" (UID: \"4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8\") " pod="openshift-ingress/e2e-mt-02f24e54-openshift-default-6dc6dd4cf9-pjcsw" Jul 06 19:36:56.687507 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:36:56.687483 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8-istio-data\") pod \"e2e-mt-02f24e54-openshift-default-6dc6dd4cf9-pjcsw\" (UID: \"4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8\") " pod="openshift-ingress/e2e-mt-02f24e54-openshift-default-6dc6dd4cf9-pjcsw" Jul 06 19:36:56.687589 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:36:56.687545 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8-istiod-ca-cert\") pod \"e2e-mt-02f24e54-openshift-default-6dc6dd4cf9-pjcsw\" (UID: \"4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8\") " pod="openshift-ingress/e2e-mt-02f24e54-openshift-default-6dc6dd4cf9-pjcsw" Jul 06 19:36:56.689852 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:36:56.689824 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8-istio-envoy\") pod \"e2e-mt-02f24e54-openshift-default-6dc6dd4cf9-pjcsw\" (UID: \"4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8\") " pod="openshift-ingress/e2e-mt-02f24e54-openshift-default-6dc6dd4cf9-pjcsw" Jul 06 19:36:56.691940 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:36:56.691904 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8-istio-podinfo\") pod \"e2e-mt-02f24e54-openshift-default-6dc6dd4cf9-pjcsw\" (UID: \"4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8\") " pod="openshift-ingress/e2e-mt-02f24e54-openshift-default-6dc6dd4cf9-pjcsw" Jul 06 19:36:56.695531 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:36:56.695505 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8-istio-token\") pod \"e2e-mt-02f24e54-openshift-default-6dc6dd4cf9-pjcsw\" (UID: \"4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8\") " pod="openshift-ingress/e2e-mt-02f24e54-openshift-default-6dc6dd4cf9-pjcsw" Jul 06 19:36:56.695641 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:36:56.695574 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xkjlv\" (UniqueName: \"kubernetes.io/projected/4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8-kube-api-access-xkjlv\") pod \"e2e-mt-02f24e54-openshift-default-6dc6dd4cf9-pjcsw\" (UID: \"4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8\") " pod="openshift-ingress/e2e-mt-02f24e54-openshift-default-6dc6dd4cf9-pjcsw" Jul 06 19:36:56.743494 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:36:56.743454 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:36:56.826331 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:36:56.826300 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-02f24e54-openshift-default-6dc6dd4cf9-pjcsw" Jul 06 19:36:56.984173 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:36:56.984143 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-02f24e54-openshift-default-6dc6dd4cf9-pjcsw"] Jul 06 19:36:56.985727 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:36:56.985702 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ecf01c5_2bc9_4868_bacf_dd53ed8d36d8.slice/crio-b564f07e005273bdf5bdf75f1bffbc66dc8a34dbae7b4085859341319cc28a63 WatchSource:0}: Error finding container b564f07e005273bdf5bdf75f1bffbc66dc8a34dbae7b4085859341319cc28a63: Status 404 returned error can't find the container with id b564f07e005273bdf5bdf75f1bffbc66dc8a34dbae7b4085859341319cc28a63 Jul 06 19:36:56.987804 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:36:56.987769 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 19:36:56.987889 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:36:56.987838 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 19:36:56.987889 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:36:56.987865 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 19:36:57.078670 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:36:57.078586 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-02f24e54-openshift-default-6dc6dd4cf9-pjcsw" event={"ID":"4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8","Type":"ContainerStarted","Data":"f1db6fb3c6bdd783f833ee28a17fff01ff82cdd847de907fccafe536d7289c4d"} Jul 06 19:36:57.078670 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:36:57.078660 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-02f24e54-openshift-default-6dc6dd4cf9-pjcsw" event={"ID":"4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8","Type":"ContainerStarted","Data":"b564f07e005273bdf5bdf75f1bffbc66dc8a34dbae7b4085859341319cc28a63"} Jul 06 19:36:57.102693 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:36:57.102619 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-02f24e54-openshift-default-6dc6dd4cf9-pjcsw" podStartSLOduration=1.102598951 podStartE2EDuration="1.102598951s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:36:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:36:57.101932837 +0000 UTC m=+1642.409291801" watchObservedRunningTime="2026-07-06 19:36:57.102598951 +0000 UTC m=+1642.409957899" Jul 06 19:36:57.827315 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:36:57.827282 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-02f24e54-openshift-default-6dc6dd4cf9-pjcsw" Jul 06 19:36:58.831467 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:36:58.831436 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-02f24e54-openshift-default-6dc6dd4cf9-pjcsw" Jul 06 19:36:59.089412 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:36:59.089324 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-02f24e54-openshift-default-6dc6dd4cf9-pjcsw" Jul 06 19:36:59.090405 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:36:59.090386 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-02f24e54-openshift-default-6dc6dd4cf9-pjcsw" Jul 06 19:36:59.243649 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:36:59.243620 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:36:59.353042 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:36:59.352947 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:36:59.469880 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:36:59.469848 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:36:59.851354 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:36:59.851322 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:37:00.103779 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:00.103683 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:37:00.205807 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:00.205769 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:37:03.759738 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:03.759707 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:37:04.437946 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:04.437898 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:37:04.540944 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:04.540909 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:37:04.876688 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:04.876656 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:37:11.114559 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:11.114498 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:37:11.216006 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:11.215973 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:37:11.550499 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:11.550428 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:37:11.941943 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:11.941908 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:37:12.036258 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:12.036213 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:37:12.166174 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:12.166145 2566 generic.go:356] "Generic (PLEG): container finished" podID="4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8" containerID="f1db6fb3c6bdd783f833ee28a17fff01ff82cdd847de907fccafe536d7289c4d" exitCode=0 Jul 06 19:37:12.166546 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:12.166220 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-02f24e54-openshift-default-6dc6dd4cf9-pjcsw" event={"ID":"4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8","Type":"ContainerDied","Data":"f1db6fb3c6bdd783f833ee28a17fff01ff82cdd847de907fccafe536d7289c4d"} Jul 06 19:37:12.166625 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:12.166612 2566 scope.go:117] "RemoveContainer" containerID="f1db6fb3c6bdd783f833ee28a17fff01ff82cdd847de907fccafe536d7289c4d" Jul 06 19:37:12.167597 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:12.167563 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 19:37:12.167715 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:12.167642 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 19:37:12.167715 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:12.167690 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 19:37:13.173327 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:13.173291 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-02f24e54-openshift-default-6dc6dd4cf9-pjcsw" event={"ID":"4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8","Type":"ContainerStarted","Data":"8ba0728f3240e3dfca1be667cfc5622e6ec558f22a9ddc6907557cc7e2e93d5b"} Jul 06 19:37:13.827566 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:13.827529 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-02f24e54-openshift-default-6dc6dd4cf9-pjcsw" Jul 06 19:37:13.828816 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:13.828787 2566 patch_prober.go:28] interesting pod/e2e-mt-02f24e54-openshift-default-6dc6dd4cf9-pjcsw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Jul 06 19:37:13.828950 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:13.828854 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-02f24e54-openshift-default-6dc6dd4cf9-pjcsw" podUID="4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Jul 06 19:37:14.089633 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:14.089541 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-02f24e54-openshift-default-6dc6dd4cf9-pjcsw" Jul 06 19:37:14.827732 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:14.827694 2566 patch_prober.go:28] interesting pod/e2e-mt-02f24e54-openshift-default-6dc6dd4cf9-pjcsw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Jul 06 19:37:14.828154 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:14.827758 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-02f24e54-openshift-default-6dc6dd4cf9-pjcsw" podUID="4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Jul 06 19:37:15.827466 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:15.827425 2566 patch_prober.go:28] interesting pod/e2e-mt-02f24e54-openshift-default-6dc6dd4cf9-pjcsw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Jul 06 19:37:15.827668 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:15.827487 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-02f24e54-openshift-default-6dc6dd4cf9-pjcsw" podUID="4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Jul 06 19:37:16.827450 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:16.827416 2566 patch_prober.go:28] interesting pod/e2e-mt-02f24e54-openshift-default-6dc6dd4cf9-pjcsw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Jul 06 19:37:16.827829 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:16.827473 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-02f24e54-openshift-default-6dc6dd4cf9-pjcsw" podUID="4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Jul 06 19:37:17.831001 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:17.830969 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-02f24e54-openshift-default-6dc6dd4cf9-pjcsw" Jul 06 19:37:17.832036 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:17.832019 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-02f24e54-openshift-default-6dc6dd4cf9-pjcsw" Jul 06 19:37:18.594978 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:18.594950 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:37:18.695716 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:18.695683 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:37:18.829610 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:18.829576 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-02f24e54-openshift-default-6dc6dd4cf9-pjcsw"] Jul 06 19:37:18.849267 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:18.849178 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:37:19.087518 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:19.087487 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:37:19.205726 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:19.205638 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-02f24e54-openshift-default-6dc6dd4cf9-pjcsw" podUID="4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8" containerName="istio-proxy" containerID="cri-o://8ba0728f3240e3dfca1be667cfc5622e6ec558f22a9ddc6907557cc7e2e93d5b" gracePeriod=30 Jul 06 19:37:19.243361 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:19.243327 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:37:19.342101 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:19.342051 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:37:19.440225 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:19.440188 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:37:24.471211 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:24.471152 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-02f24e54-openshift-default-6dc6dd4cf9-pjcsw" Jul 06 19:37:24.562399 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:24.562360 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8-istio-data\") pod \"4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8\" (UID: \"4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8\") " Jul 06 19:37:24.562656 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:24.562432 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8-istio-token\") pod \"4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8\" (UID: \"4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8\") " Jul 06 19:37:24.562656 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:24.562483 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8-istiod-ca-cert\") pod \"4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8\" (UID: \"4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8\") " Jul 06 19:37:24.562656 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:24.562512 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8-istio-podinfo\") pod \"4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8\" (UID: \"4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8\") " Jul 06 19:37:24.562656 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:24.562638 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8-credential-socket\") pod \"4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8\" (UID: \"4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8\") " Jul 06 19:37:24.562850 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:24.562684 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8-workload-socket\") pod \"4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8\" (UID: \"4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8\") " Jul 06 19:37:24.562850 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:24.562723 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8-workload-certs\") pod \"4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8\" (UID: \"4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8\") " Jul 06 19:37:24.562850 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:24.562755 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xkjlv\" (UniqueName: \"kubernetes.io/projected/4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8-kube-api-access-xkjlv\") pod \"4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8\" (UID: \"4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8\") " Jul 06 19:37:24.563001 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:24.562892 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8" (UID: "4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:37:24.563001 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:24.562942 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8" (UID: "4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:37:24.563141 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:24.563009 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8" (UID: "4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:37:24.563141 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:24.563024 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8" (UID: "4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:37:24.563242 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:24.563176 2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8-istiod-ca-cert\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:37:24.563242 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:24.563197 2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8-credential-socket\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:37:24.563242 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:24.563212 2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8-workload-socket\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:37:24.563242 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:24.563225 2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8-workload-certs\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:37:24.563394 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:24.563286 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8-istio-data" (OuterVolumeSpecName: "istio-data") pod "4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8" (UID: "4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:37:24.564858 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:24.564831 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8-istio-token" (OuterVolumeSpecName: "istio-token") pod "4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8" (UID: "4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:37:24.564986 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:24.564893 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8" (UID: "4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 19:37:24.565034 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:24.565015 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8-kube-api-access-xkjlv" (OuterVolumeSpecName: "kube-api-access-xkjlv") pod "4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8" (UID: "4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8"). InnerVolumeSpecName "kube-api-access-xkjlv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:37:24.664035 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:24.664001 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8-istio-envoy\") pod \"4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8\" (UID: \"4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8\") " Jul 06 19:37:24.664248 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:24.664234 2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8-istio-data\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:37:24.664293 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:24.664251 2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8-istio-token\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:37:24.664293 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:24.664260 2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8-istio-podinfo\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:37:24.664293 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:24.664271 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xkjlv\" (UniqueName: \"kubernetes.io/projected/4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8-kube-api-access-xkjlv\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:37:24.666105 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:24.666061 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8" (UID: "4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:37:24.764851 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:24.764763 2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8-istio-envoy\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:37:25.247370 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:25.247332 2566 generic.go:356] "Generic (PLEG): container finished" podID="4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8" containerID="8ba0728f3240e3dfca1be667cfc5622e6ec558f22a9ddc6907557cc7e2e93d5b" exitCode=0 Jul 06 19:37:25.247536 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:25.247404 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-02f24e54-openshift-default-6dc6dd4cf9-pjcsw" Jul 06 19:37:25.247536 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:25.247417 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-02f24e54-openshift-default-6dc6dd4cf9-pjcsw" event={"ID":"4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8","Type":"ContainerDied","Data":"8ba0728f3240e3dfca1be667cfc5622e6ec558f22a9ddc6907557cc7e2e93d5b"} Jul 06 19:37:25.247536 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:25.247475 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-02f24e54-openshift-default-6dc6dd4cf9-pjcsw" event={"ID":"4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8","Type":"ContainerDied","Data":"b564f07e005273bdf5bdf75f1bffbc66dc8a34dbae7b4085859341319cc28a63"} Jul 06 19:37:25.247536 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:25.247502 2566 scope.go:117] "RemoveContainer" containerID="8ba0728f3240e3dfca1be667cfc5622e6ec558f22a9ddc6907557cc7e2e93d5b" Jul 06 19:37:25.259886 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:25.259859 2566 scope.go:117] "RemoveContainer" containerID="f1db6fb3c6bdd783f833ee28a17fff01ff82cdd847de907fccafe536d7289c4d" Jul 06 19:37:25.268917 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:25.268891 2566 scope.go:117] "RemoveContainer" containerID="8ba0728f3240e3dfca1be667cfc5622e6ec558f22a9ddc6907557cc7e2e93d5b" Jul 06 19:37:25.269324 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:37:25.269294 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8ba0728f3240e3dfca1be667cfc5622e6ec558f22a9ddc6907557cc7e2e93d5b\": container with ID starting with 8ba0728f3240e3dfca1be667cfc5622e6ec558f22a9ddc6907557cc7e2e93d5b not found: ID does not exist" containerID="8ba0728f3240e3dfca1be667cfc5622e6ec558f22a9ddc6907557cc7e2e93d5b" Jul 06 19:37:25.269439 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:25.269333 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8ba0728f3240e3dfca1be667cfc5622e6ec558f22a9ddc6907557cc7e2e93d5b"} err="failed to get container status \"8ba0728f3240e3dfca1be667cfc5622e6ec558f22a9ddc6907557cc7e2e93d5b\": rpc error: code = NotFound desc = could not find container \"8ba0728f3240e3dfca1be667cfc5622e6ec558f22a9ddc6907557cc7e2e93d5b\": container with ID starting with 8ba0728f3240e3dfca1be667cfc5622e6ec558f22a9ddc6907557cc7e2e93d5b not found: ID does not exist" Jul 06 19:37:25.269439 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:25.269358 2566 scope.go:117] "RemoveContainer" containerID="f1db6fb3c6bdd783f833ee28a17fff01ff82cdd847de907fccafe536d7289c4d" Jul 06 19:37:25.269659 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:37:25.269638 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f1db6fb3c6bdd783f833ee28a17fff01ff82cdd847de907fccafe536d7289c4d\": container with ID starting with f1db6fb3c6bdd783f833ee28a17fff01ff82cdd847de907fccafe536d7289c4d not found: ID does not exist" containerID="f1db6fb3c6bdd783f833ee28a17fff01ff82cdd847de907fccafe536d7289c4d" Jul 06 19:37:25.269724 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:25.269663 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f1db6fb3c6bdd783f833ee28a17fff01ff82cdd847de907fccafe536d7289c4d"} err="failed to get container status \"f1db6fb3c6bdd783f833ee28a17fff01ff82cdd847de907fccafe536d7289c4d\": rpc error: code = NotFound desc = could not find container \"f1db6fb3c6bdd783f833ee28a17fff01ff82cdd847de907fccafe536d7289c4d\": container with ID starting with f1db6fb3c6bdd783f833ee28a17fff01ff82cdd847de907fccafe536d7289c4d not found: ID does not exist" Jul 06 19:37:25.272229 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:25.272209 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-02f24e54-openshift-default-6dc6dd4cf9-pjcsw"] Jul 06 19:37:25.277795 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:25.277775 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-02f24e54-openshift-default-6dc6dd4cf9-pjcsw"] Jul 06 19:37:25.853002 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:25.852974 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:37:25.961749 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:25.961716 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:37:26.054264 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:26.054228 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:37:26.315483 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:26.315450 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:37:27.236947 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:27.236915 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8" path="/var/lib/kubelet/pods/4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8/volumes" Jul 06 19:37:32.650162 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:32.650125 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:37:32.750033 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:32.749998 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:37:39.505394 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:39.505359 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:37:39.844256 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:39.844212 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-7f0c7a26-openshift-default-7dd7f6bd87-9w2z4"] Jul 06 19:37:39.845134 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:39.845097 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8" containerName="istio-proxy" Jul 06 19:37:39.845134 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:39.845123 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8" containerName="istio-proxy" Jul 06 19:37:39.845325 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:39.845179 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8" containerName="istio-proxy" Jul 06 19:37:39.845325 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:39.845188 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8" containerName="istio-proxy" Jul 06 19:37:39.845519 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:39.845497 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="767926fa-2c86-4bc1-a653-c0b25367ade2" containerName="istio-proxy" Jul 06 19:37:39.845585 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:39.845528 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8" containerName="istio-proxy" Jul 06 19:37:39.845585 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:39.845540 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="4ecf01c5-2bc9-4868-bacf-dd53ed8d36d8" containerName="istio-proxy" Jul 06 19:37:39.849998 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:39.849968 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7f0c7a26-openshift-default-7dd7f6bd87-9w2z4" Jul 06 19:37:39.852833 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:39.852811 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-7f0c7a26-openshift-default-dockercfg-9bltp\"" Jul 06 19:37:39.853986 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:39.853958 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-7f0c7a26-openshift-default-7dd7f6bd87-9w2z4"] Jul 06 19:37:39.878859 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:39.878806 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:37:39.892842 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:39.892810 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0e63126f-87e1-470d-95bf-e503af0c3092-istio-token\") pod \"e2e-mt-7f0c7a26-openshift-default-7dd7f6bd87-9w2z4\" (UID: \"0e63126f-87e1-470d-95bf-e503af0c3092\") " pod="openshift-ingress/e2e-mt-7f0c7a26-openshift-default-7dd7f6bd87-9w2z4" Jul 06 19:37:39.893038 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:39.892861 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0e63126f-87e1-470d-95bf-e503af0c3092-istio-envoy\") pod \"e2e-mt-7f0c7a26-openshift-default-7dd7f6bd87-9w2z4\" (UID: \"0e63126f-87e1-470d-95bf-e503af0c3092\") " pod="openshift-ingress/e2e-mt-7f0c7a26-openshift-default-7dd7f6bd87-9w2z4" Jul 06 19:37:39.893038 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:39.892960 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e63126f-87e1-470d-95bf-e503af0c3092-workload-socket\") pod \"e2e-mt-7f0c7a26-openshift-default-7dd7f6bd87-9w2z4\" (UID: \"0e63126f-87e1-470d-95bf-e503af0c3092\") " pod="openshift-ingress/e2e-mt-7f0c7a26-openshift-default-7dd7f6bd87-9w2z4" Jul 06 19:37:39.893038 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:39.893012 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e63126f-87e1-470d-95bf-e503af0c3092-credential-socket\") pod \"e2e-mt-7f0c7a26-openshift-default-7dd7f6bd87-9w2z4\" (UID: \"0e63126f-87e1-470d-95bf-e503af0c3092\") " pod="openshift-ingress/e2e-mt-7f0c7a26-openshift-default-7dd7f6bd87-9w2z4" Jul 06 19:37:39.893241 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:39.893048 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0e63126f-87e1-470d-95bf-e503af0c3092-workload-certs\") pod \"e2e-mt-7f0c7a26-openshift-default-7dd7f6bd87-9w2z4\" (UID: \"0e63126f-87e1-470d-95bf-e503af0c3092\") " pod="openshift-ingress/e2e-mt-7f0c7a26-openshift-default-7dd7f6bd87-9w2z4" Jul 06 19:37:39.893241 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:39.893126 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0e63126f-87e1-470d-95bf-e503af0c3092-istio-podinfo\") pod \"e2e-mt-7f0c7a26-openshift-default-7dd7f6bd87-9w2z4\" (UID: \"0e63126f-87e1-470d-95bf-e503af0c3092\") " pod="openshift-ingress/e2e-mt-7f0c7a26-openshift-default-7dd7f6bd87-9w2z4" Jul 06 19:37:39.893241 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:39.893166 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0e63126f-87e1-470d-95bf-e503af0c3092-istiod-ca-cert\") pod \"e2e-mt-7f0c7a26-openshift-default-7dd7f6bd87-9w2z4\" (UID: \"0e63126f-87e1-470d-95bf-e503af0c3092\") " pod="openshift-ingress/e2e-mt-7f0c7a26-openshift-default-7dd7f6bd87-9w2z4" Jul 06 19:37:39.893241 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:39.893213 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0e63126f-87e1-470d-95bf-e503af0c3092-istio-data\") pod \"e2e-mt-7f0c7a26-openshift-default-7dd7f6bd87-9w2z4\" (UID: \"0e63126f-87e1-470d-95bf-e503af0c3092\") " pod="openshift-ingress/e2e-mt-7f0c7a26-openshift-default-7dd7f6bd87-9w2z4" Jul 06 19:37:39.893241 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:39.893237 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dx2zl\" (UniqueName: \"kubernetes.io/projected/0e63126f-87e1-470d-95bf-e503af0c3092-kube-api-access-dx2zl\") pod \"e2e-mt-7f0c7a26-openshift-default-7dd7f6bd87-9w2z4\" (UID: \"0e63126f-87e1-470d-95bf-e503af0c3092\") " pod="openshift-ingress/e2e-mt-7f0c7a26-openshift-default-7dd7f6bd87-9w2z4" Jul 06 19:37:39.976304 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:39.976274 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:37:39.994524 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:39.994492 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0e63126f-87e1-470d-95bf-e503af0c3092-istio-token\") pod \"e2e-mt-7f0c7a26-openshift-default-7dd7f6bd87-9w2z4\" (UID: \"0e63126f-87e1-470d-95bf-e503af0c3092\") " pod="openshift-ingress/e2e-mt-7f0c7a26-openshift-default-7dd7f6bd87-9w2z4" Jul 06 19:37:39.994730 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:39.994538 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0e63126f-87e1-470d-95bf-e503af0c3092-istio-envoy\") pod \"e2e-mt-7f0c7a26-openshift-default-7dd7f6bd87-9w2z4\" (UID: \"0e63126f-87e1-470d-95bf-e503af0c3092\") " pod="openshift-ingress/e2e-mt-7f0c7a26-openshift-default-7dd7f6bd87-9w2z4" Jul 06 19:37:39.994730 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:39.994562 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e63126f-87e1-470d-95bf-e503af0c3092-workload-socket\") pod \"e2e-mt-7f0c7a26-openshift-default-7dd7f6bd87-9w2z4\" (UID: \"0e63126f-87e1-470d-95bf-e503af0c3092\") " pod="openshift-ingress/e2e-mt-7f0c7a26-openshift-default-7dd7f6bd87-9w2z4" Jul 06 19:37:39.994730 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:39.994684 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e63126f-87e1-470d-95bf-e503af0c3092-credential-socket\") pod \"e2e-mt-7f0c7a26-openshift-default-7dd7f6bd87-9w2z4\" (UID: \"0e63126f-87e1-470d-95bf-e503af0c3092\") " pod="openshift-ingress/e2e-mt-7f0c7a26-openshift-default-7dd7f6bd87-9w2z4" Jul 06 19:37:39.994896 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:39.994748 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0e63126f-87e1-470d-95bf-e503af0c3092-workload-certs\") pod \"e2e-mt-7f0c7a26-openshift-default-7dd7f6bd87-9w2z4\" (UID: \"0e63126f-87e1-470d-95bf-e503af0c3092\") " pod="openshift-ingress/e2e-mt-7f0c7a26-openshift-default-7dd7f6bd87-9w2z4" Jul 06 19:37:39.994896 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:39.994794 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0e63126f-87e1-470d-95bf-e503af0c3092-istio-podinfo\") pod \"e2e-mt-7f0c7a26-openshift-default-7dd7f6bd87-9w2z4\" (UID: \"0e63126f-87e1-470d-95bf-e503af0c3092\") " pod="openshift-ingress/e2e-mt-7f0c7a26-openshift-default-7dd7f6bd87-9w2z4" Jul 06 19:37:39.994896 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:39.994824 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0e63126f-87e1-470d-95bf-e503af0c3092-istiod-ca-cert\") pod \"e2e-mt-7f0c7a26-openshift-default-7dd7f6bd87-9w2z4\" (UID: \"0e63126f-87e1-470d-95bf-e503af0c3092\") " pod="openshift-ingress/e2e-mt-7f0c7a26-openshift-default-7dd7f6bd87-9w2z4" Jul 06 19:37:39.994896 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:39.994856 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0e63126f-87e1-470d-95bf-e503af0c3092-istio-data\") pod \"e2e-mt-7f0c7a26-openshift-default-7dd7f6bd87-9w2z4\" (UID: \"0e63126f-87e1-470d-95bf-e503af0c3092\") " pod="openshift-ingress/e2e-mt-7f0c7a26-openshift-default-7dd7f6bd87-9w2z4" Jul 06 19:37:39.994896 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:39.994881 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dx2zl\" (UniqueName: \"kubernetes.io/projected/0e63126f-87e1-470d-95bf-e503af0c3092-kube-api-access-dx2zl\") pod \"e2e-mt-7f0c7a26-openshift-default-7dd7f6bd87-9w2z4\" (UID: \"0e63126f-87e1-470d-95bf-e503af0c3092\") " pod="openshift-ingress/e2e-mt-7f0c7a26-openshift-default-7dd7f6bd87-9w2z4" Jul 06 19:37:39.995208 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:39.994918 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e63126f-87e1-470d-95bf-e503af0c3092-workload-socket\") pod \"e2e-mt-7f0c7a26-openshift-default-7dd7f6bd87-9w2z4\" (UID: \"0e63126f-87e1-470d-95bf-e503af0c3092\") " pod="openshift-ingress/e2e-mt-7f0c7a26-openshift-default-7dd7f6bd87-9w2z4" Jul 06 19:37:39.995208 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:39.994977 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e63126f-87e1-470d-95bf-e503af0c3092-credential-socket\") pod \"e2e-mt-7f0c7a26-openshift-default-7dd7f6bd87-9w2z4\" (UID: \"0e63126f-87e1-470d-95bf-e503af0c3092\") " pod="openshift-ingress/e2e-mt-7f0c7a26-openshift-default-7dd7f6bd87-9w2z4" Jul 06 19:37:39.995208 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:39.995116 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0e63126f-87e1-470d-95bf-e503af0c3092-workload-certs\") pod \"e2e-mt-7f0c7a26-openshift-default-7dd7f6bd87-9w2z4\" (UID: \"0e63126f-87e1-470d-95bf-e503af0c3092\") " pod="openshift-ingress/e2e-mt-7f0c7a26-openshift-default-7dd7f6bd87-9w2z4" Jul 06 19:37:39.995208 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:39.995186 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0e63126f-87e1-470d-95bf-e503af0c3092-istio-data\") pod \"e2e-mt-7f0c7a26-openshift-default-7dd7f6bd87-9w2z4\" (UID: \"0e63126f-87e1-470d-95bf-e503af0c3092\") " pod="openshift-ingress/e2e-mt-7f0c7a26-openshift-default-7dd7f6bd87-9w2z4" Jul 06 19:37:39.995550 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:39.995526 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0e63126f-87e1-470d-95bf-e503af0c3092-istiod-ca-cert\") pod \"e2e-mt-7f0c7a26-openshift-default-7dd7f6bd87-9w2z4\" (UID: \"0e63126f-87e1-470d-95bf-e503af0c3092\") " pod="openshift-ingress/e2e-mt-7f0c7a26-openshift-default-7dd7f6bd87-9w2z4" Jul 06 19:37:39.997089 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:39.997043 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0e63126f-87e1-470d-95bf-e503af0c3092-istio-envoy\") pod \"e2e-mt-7f0c7a26-openshift-default-7dd7f6bd87-9w2z4\" (UID: \"0e63126f-87e1-470d-95bf-e503af0c3092\") " pod="openshift-ingress/e2e-mt-7f0c7a26-openshift-default-7dd7f6bd87-9w2z4" Jul 06 19:37:39.997436 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:39.997414 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0e63126f-87e1-470d-95bf-e503af0c3092-istio-podinfo\") pod \"e2e-mt-7f0c7a26-openshift-default-7dd7f6bd87-9w2z4\" (UID: \"0e63126f-87e1-470d-95bf-e503af0c3092\") " pod="openshift-ingress/e2e-mt-7f0c7a26-openshift-default-7dd7f6bd87-9w2z4" Jul 06 19:37:40.004632 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:40.004612 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0e63126f-87e1-470d-95bf-e503af0c3092-istio-token\") pod \"e2e-mt-7f0c7a26-openshift-default-7dd7f6bd87-9w2z4\" (UID: \"0e63126f-87e1-470d-95bf-e503af0c3092\") " pod="openshift-ingress/e2e-mt-7f0c7a26-openshift-default-7dd7f6bd87-9w2z4" Jul 06 19:37:40.004924 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:40.004901 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dx2zl\" (UniqueName: \"kubernetes.io/projected/0e63126f-87e1-470d-95bf-e503af0c3092-kube-api-access-dx2zl\") pod \"e2e-mt-7f0c7a26-openshift-default-7dd7f6bd87-9w2z4\" (UID: \"0e63126f-87e1-470d-95bf-e503af0c3092\") " pod="openshift-ingress/e2e-mt-7f0c7a26-openshift-default-7dd7f6bd87-9w2z4" Jul 06 19:37:40.161546 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:40.161438 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7f0c7a26-openshift-default-7dd7f6bd87-9w2z4" Jul 06 19:37:40.306652 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:40.306626 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-7f0c7a26-openshift-default-7dd7f6bd87-9w2z4"] Jul 06 19:37:40.308191 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:37:40.308153 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e63126f_87e1_470d_95bf_e503af0c3092.slice/crio-6dd4127126e19ec771d19f1fed7035eff3bfe3d65ce31a6683cefe0c49a1d302 WatchSource:0}: Error finding container 6dd4127126e19ec771d19f1fed7035eff3bfe3d65ce31a6683cefe0c49a1d302: Status 404 returned error can't find the container with id 6dd4127126e19ec771d19f1fed7035eff3bfe3d65ce31a6683cefe0c49a1d302 Jul 06 19:37:40.310737 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:40.310704 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 19:37:40.310825 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:40.310769 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 19:37:40.310825 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:40.310801 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 19:37:40.334166 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:40.334125 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7f0c7a26-openshift-default-7dd7f6bd87-9w2z4" event={"ID":"0e63126f-87e1-470d-95bf-e503af0c3092","Type":"ContainerStarted","Data":"6dd4127126e19ec771d19f1fed7035eff3bfe3d65ce31a6683cefe0c49a1d302"} Jul 06 19:37:41.341303 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:41.341260 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7f0c7a26-openshift-default-7dd7f6bd87-9w2z4" event={"ID":"0e63126f-87e1-470d-95bf-e503af0c3092","Type":"ContainerStarted","Data":"4fcc0a8e1f5057e73628ba3e3ccba7a57c1909092cefd0f93f84e174988b0ce4"} Jul 06 19:37:41.364557 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:41.364510 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-7f0c7a26-openshift-default-7dd7f6bd87-9w2z4" podStartSLOduration=2.36448617 podStartE2EDuration="2.36448617s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:37:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:37:41.362643923 +0000 UTC m=+1686.670002870" watchObservedRunningTime="2026-07-06 19:37:41.36448617 +0000 UTC m=+1686.671845119" Jul 06 19:37:42.161572 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:42.161536 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-7f0c7a26-openshift-default-7dd7f6bd87-9w2z4" Jul 06 19:37:42.166213 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:42.166187 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-7f0c7a26-openshift-default-7dd7f6bd87-9w2z4" Jul 06 19:37:42.350431 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:42.350394 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-7f0c7a26-openshift-default-7dd7f6bd87-9w2z4" Jul 06 19:37:42.351322 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:42.351299 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-7f0c7a26-openshift-default-7dd7f6bd87-9w2z4" Jul 06 19:37:43.286803 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:43.286772 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:37:43.403098 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:43.403040 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:37:43.485161 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:43.485130 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:37:43.670983 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:43.670894 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:37:43.993875 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:43.993786 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:37:44.093481 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:44.093450 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:37:47.748134 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:47.748096 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:37:57.352117 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:57.352054 2566 patch_prober.go:28] interesting pod/e2e-mt-7f0c7a26-openshift-default-7dd7f6bd87-9w2z4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body= Jul 06 19:37:57.352514 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:57.352178 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-7f0c7a26-openshift-default-7dd7f6bd87-9w2z4" podUID="0e63126f-87e1-470d-95bf-e503af0c3092" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" Jul 06 19:37:57.445891 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:57.445802 2566 generic.go:356] "Generic (PLEG): container finished" podID="0e63126f-87e1-470d-95bf-e503af0c3092" containerID="4fcc0a8e1f5057e73628ba3e3ccba7a57c1909092cefd0f93f84e174988b0ce4" exitCode=0 Jul 06 19:37:57.445891 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:57.445874 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7f0c7a26-openshift-default-7dd7f6bd87-9w2z4" event={"ID":"0e63126f-87e1-470d-95bf-e503af0c3092","Type":"ContainerDied","Data":"4fcc0a8e1f5057e73628ba3e3ccba7a57c1909092cefd0f93f84e174988b0ce4"} Jul 06 19:37:57.446403 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:57.446387 2566 scope.go:117] "RemoveContainer" containerID="4fcc0a8e1f5057e73628ba3e3ccba7a57c1909092cefd0f93f84e174988b0ce4" Jul 06 19:37:57.447473 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:57.447446 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 19:37:57.447550 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:57.447528 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 19:37:57.447588 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:57.447578 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 19:37:57.531347 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:57.531317 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:37:58.458176 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:58.458134 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7f0c7a26-openshift-default-7dd7f6bd87-9w2z4" event={"ID":"0e63126f-87e1-470d-95bf-e503af0c3092","Type":"ContainerStarted","Data":"adc609c7dc8af4f4449dc1bc10455ba2ca82f668802cd78db1effeeeaeebec0c"} Jul 06 19:37:59.161675 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:59.161636 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-7f0c7a26-openshift-default-7dd7f6bd87-9w2z4" Jul 06 19:37:59.161954 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:59.161929 2566 patch_prober.go:28] interesting pod/e2e-mt-7f0c7a26-openshift-default-7dd7f6bd87-9w2z4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body= Jul 06 19:37:59.162044 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:37:59.161986 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7f0c7a26-openshift-default-7dd7f6bd87-9w2z4" podUID="0e63126f-87e1-470d-95bf-e503af0c3092" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" Jul 06 19:38:00.161966 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:38:00.161926 2566 patch_prober.go:28] interesting pod/e2e-mt-7f0c7a26-openshift-default-7dd7f6bd87-9w2z4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body= Jul 06 19:38:00.162465 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:38:00.162010 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7f0c7a26-openshift-default-7dd7f6bd87-9w2z4" podUID="0e63126f-87e1-470d-95bf-e503af0c3092" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" Jul 06 19:38:01.162619 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:38:01.162576 2566 patch_prober.go:28] interesting pod/e2e-mt-7f0c7a26-openshift-default-7dd7f6bd87-9w2z4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body= Jul 06 19:38:01.163031 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:38:01.162656 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7f0c7a26-openshift-default-7dd7f6bd87-9w2z4" podUID="0e63126f-87e1-470d-95bf-e503af0c3092" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" Jul 06 19:38:02.104832 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:38:02.104797 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:38:02.161902 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:38:02.161851 2566 patch_prober.go:28] interesting pod/e2e-mt-7f0c7a26-openshift-default-7dd7f6bd87-9w2z4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body= Jul 06 19:38:02.162129 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:38:02.161934 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-7f0c7a26-openshift-default-7dd7f6bd87-9w2z4" podUID="0e63126f-87e1-470d-95bf-e503af0c3092" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" Jul 06 19:38:02.204754 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:38:02.204717 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:38:03.165788 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:38:03.165756 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-7f0c7a26-openshift-default-7dd7f6bd87-9w2z4" Jul 06 19:38:03.166123 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:38:03.166102 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-7f0c7a26-openshift-default-7dd7f6bd87-9w2z4" Jul 06 19:38:03.166674 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:38:03.166661 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-7f0c7a26-openshift-default-7dd7f6bd87-9w2z4" Jul 06 19:38:04.794780 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:38:04.794750 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:38:04.898574 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:38:04.898541 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:38:20.101138 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:38:20.101104 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:38:20.611457 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:38:20.611428 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:38:26.175469 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:38:26.175432 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:38:31.218116 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:38:31.218055 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:39:08.608615 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:39:08.608581 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:39:08.919199 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:39:08.919109 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:39:14.694908 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:39:14.694867 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:39:14.796715 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:39:14.796684 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:39:14.952401 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:39:14.952315 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-7f0c7a26-openshift-default-7dd7f6bd87-9w2z4"] Jul 06 19:39:14.952584 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:39:14.952558 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-7f0c7a26-openshift-default-7dd7f6bd87-9w2z4" podUID="0e63126f-87e1-470d-95bf-e503af0c3092" containerName="istio-proxy" containerID="cri-o://adc609c7dc8af4f4449dc1bc10455ba2ca82f668802cd78db1effeeeaeebec0c" gracePeriod=30 Jul 06 19:39:14.992318 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:39:14.992288 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:39:15.215758 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:39:15.215660 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:39:16.238171 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:39:16.238137 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:39:16.330667 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:39:16.330614 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:39:16.428388 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:39:16.428355 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:39:17.225200 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:39:17.225163 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:39:17.313200 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:39:17.313173 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:39:18.168752 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:39:18.168714 2566 patch_prober.go:28] interesting pod/e2e-mt-7f0c7a26-openshift-default-7dd7f6bd87-9w2z4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 06 19:39:18.168933 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:39:18.168793 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-7f0c7a26-openshift-default-7dd7f6bd87-9w2z4" podUID="0e63126f-87e1-470d-95bf-e503af0c3092" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 06 19:39:18.789588 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:39:18.789557 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:39:20.207566 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:39:20.207538 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7f0c7a26-openshift-default-7dd7f6bd87-9w2z4" Jul 06 19:39:20.315901 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:39:20.315868 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dx2zl\" (UniqueName: \"kubernetes.io/projected/0e63126f-87e1-470d-95bf-e503af0c3092-kube-api-access-dx2zl\") pod \"0e63126f-87e1-470d-95bf-e503af0c3092\" (UID: \"0e63126f-87e1-470d-95bf-e503af0c3092\") " Jul 06 19:39:20.316131 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:39:20.315919 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0e63126f-87e1-470d-95bf-e503af0c3092-istio-envoy\") pod \"0e63126f-87e1-470d-95bf-e503af0c3092\" (UID: \"0e63126f-87e1-470d-95bf-e503af0c3092\") " Jul 06 19:39:20.316131 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:39:20.315948 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0e63126f-87e1-470d-95bf-e503af0c3092-istio-data\") pod \"0e63126f-87e1-470d-95bf-e503af0c3092\" (UID: \"0e63126f-87e1-470d-95bf-e503af0c3092\") " Jul 06 19:39:20.316131 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:39:20.316022 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0e63126f-87e1-470d-95bf-e503af0c3092-istio-podinfo\") pod \"0e63126f-87e1-470d-95bf-e503af0c3092\" (UID: \"0e63126f-87e1-470d-95bf-e503af0c3092\") " Jul 06 19:39:20.316131 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:39:20.316097 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0e63126f-87e1-470d-95bf-e503af0c3092-istio-token\") pod \"0e63126f-87e1-470d-95bf-e503af0c3092\" (UID: \"0e63126f-87e1-470d-95bf-e503af0c3092\") " Jul 06 19:39:20.316131 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:39:20.316121 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0e63126f-87e1-470d-95bf-e503af0c3092-workload-certs\") pod \"0e63126f-87e1-470d-95bf-e503af0c3092\" (UID: \"0e63126f-87e1-470d-95bf-e503af0c3092\") " Jul 06 19:39:20.316382 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:39:20.316143 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0e63126f-87e1-470d-95bf-e503af0c3092-istiod-ca-cert\") pod \"0e63126f-87e1-470d-95bf-e503af0c3092\" (UID: \"0e63126f-87e1-470d-95bf-e503af0c3092\") " Jul 06 19:39:20.316382 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:39:20.316179 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e63126f-87e1-470d-95bf-e503af0c3092-workload-socket\") pod \"0e63126f-87e1-470d-95bf-e503af0c3092\" (UID: \"0e63126f-87e1-470d-95bf-e503af0c3092\") " Jul 06 19:39:20.316382 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:39:20.316207 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e63126f-87e1-470d-95bf-e503af0c3092-credential-socket\") pod \"0e63126f-87e1-470d-95bf-e503af0c3092\" (UID: \"0e63126f-87e1-470d-95bf-e503af0c3092\") " Jul 06 19:39:20.316646 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:39:20.316597 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e63126f-87e1-470d-95bf-e503af0c3092-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "0e63126f-87e1-470d-95bf-e503af0c3092" (UID: "0e63126f-87e1-470d-95bf-e503af0c3092"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:39:20.316725 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:39:20.316645 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0e63126f-87e1-470d-95bf-e503af0c3092-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "0e63126f-87e1-470d-95bf-e503af0c3092" (UID: "0e63126f-87e1-470d-95bf-e503af0c3092"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:39:20.316725 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:39:20.316672 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e63126f-87e1-470d-95bf-e503af0c3092-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "0e63126f-87e1-470d-95bf-e503af0c3092" (UID: "0e63126f-87e1-470d-95bf-e503af0c3092"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:39:20.316853 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:39:20.316815 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e63126f-87e1-470d-95bf-e503af0c3092-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "0e63126f-87e1-470d-95bf-e503af0c3092" (UID: "0e63126f-87e1-470d-95bf-e503af0c3092"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:39:20.317058 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:39:20.317035 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e63126f-87e1-470d-95bf-e503af0c3092-istio-data" (OuterVolumeSpecName: "istio-data") pod "0e63126f-87e1-470d-95bf-e503af0c3092" (UID: "0e63126f-87e1-470d-95bf-e503af0c3092"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:39:20.318376 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:39:20.318356 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e63126f-87e1-470d-95bf-e503af0c3092-kube-api-access-dx2zl" (OuterVolumeSpecName: "kube-api-access-dx2zl") pod "0e63126f-87e1-470d-95bf-e503af0c3092" (UID: "0e63126f-87e1-470d-95bf-e503af0c3092"). InnerVolumeSpecName "kube-api-access-dx2zl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:39:20.318564 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:39:20.318537 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e63126f-87e1-470d-95bf-e503af0c3092-istio-token" (OuterVolumeSpecName: "istio-token") pod "0e63126f-87e1-470d-95bf-e503af0c3092" (UID: "0e63126f-87e1-470d-95bf-e503af0c3092"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:39:20.318725 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:39:20.318710 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0e63126f-87e1-470d-95bf-e503af0c3092-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "0e63126f-87e1-470d-95bf-e503af0c3092" (UID: "0e63126f-87e1-470d-95bf-e503af0c3092"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 19:39:20.318806 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:39:20.318793 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e63126f-87e1-470d-95bf-e503af0c3092-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "0e63126f-87e1-470d-95bf-e503af0c3092" (UID: "0e63126f-87e1-470d-95bf-e503af0c3092"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:39:20.417837 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:39:20.417803 2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0e63126f-87e1-470d-95bf-e503af0c3092-istio-envoy\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:39:20.417837 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:39:20.417831 2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0e63126f-87e1-470d-95bf-e503af0c3092-istio-data\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:39:20.417837 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:39:20.417839 2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0e63126f-87e1-470d-95bf-e503af0c3092-istio-podinfo\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:39:20.418059 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:39:20.417847 2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0e63126f-87e1-470d-95bf-e503af0c3092-istio-token\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:39:20.418059 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:39:20.417857 2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0e63126f-87e1-470d-95bf-e503af0c3092-workload-certs\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:39:20.418059 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:39:20.417864 2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0e63126f-87e1-470d-95bf-e503af0c3092-istiod-ca-cert\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:39:20.418059 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:39:20.417871 2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e63126f-87e1-470d-95bf-e503af0c3092-workload-socket\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:39:20.418059 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:39:20.417878 2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e63126f-87e1-470d-95bf-e503af0c3092-credential-socket\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:39:20.418059 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:39:20.417886 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dx2zl\" (UniqueName: \"kubernetes.io/projected/0e63126f-87e1-470d-95bf-e503af0c3092-kube-api-access-dx2zl\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:39:20.921310 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:39:20.921270 2566 generic.go:356] "Generic (PLEG): container finished" podID="0e63126f-87e1-470d-95bf-e503af0c3092" containerID="adc609c7dc8af4f4449dc1bc10455ba2ca82f668802cd78db1effeeeaeebec0c" exitCode=0 Jul 06 19:39:20.921491 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:39:20.921326 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-7f0c7a26-openshift-default-7dd7f6bd87-9w2z4" Jul 06 19:39:20.921491 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:39:20.921356 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7f0c7a26-openshift-default-7dd7f6bd87-9w2z4" event={"ID":"0e63126f-87e1-470d-95bf-e503af0c3092","Type":"ContainerDied","Data":"adc609c7dc8af4f4449dc1bc10455ba2ca82f668802cd78db1effeeeaeebec0c"} Jul 06 19:39:20.921491 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:39:20.921415 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-7f0c7a26-openshift-default-7dd7f6bd87-9w2z4" event={"ID":"0e63126f-87e1-470d-95bf-e503af0c3092","Type":"ContainerDied","Data":"6dd4127126e19ec771d19f1fed7035eff3bfe3d65ce31a6683cefe0c49a1d302"} Jul 06 19:39:20.921491 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:39:20.921441 2566 scope.go:117] "RemoveContainer" containerID="adc609c7dc8af4f4449dc1bc10455ba2ca82f668802cd78db1effeeeaeebec0c" Jul 06 19:39:20.931542 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:39:20.931522 2566 scope.go:117] "RemoveContainer" containerID="4fcc0a8e1f5057e73628ba3e3ccba7a57c1909092cefd0f93f84e174988b0ce4" Jul 06 19:39:20.941951 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:39:20.941919 2566 scope.go:117] "RemoveContainer" containerID="adc609c7dc8af4f4449dc1bc10455ba2ca82f668802cd78db1effeeeaeebec0c" Jul 06 19:39:20.942291 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:39:20.942267 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"adc609c7dc8af4f4449dc1bc10455ba2ca82f668802cd78db1effeeeaeebec0c\": container with ID starting with adc609c7dc8af4f4449dc1bc10455ba2ca82f668802cd78db1effeeeaeebec0c not found: ID does not exist" containerID="adc609c7dc8af4f4449dc1bc10455ba2ca82f668802cd78db1effeeeaeebec0c" Jul 06 19:39:20.942425 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:39:20.942305 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"adc609c7dc8af4f4449dc1bc10455ba2ca82f668802cd78db1effeeeaeebec0c"} err="failed to get container status \"adc609c7dc8af4f4449dc1bc10455ba2ca82f668802cd78db1effeeeaeebec0c\": rpc error: code = NotFound desc = could not find container \"adc609c7dc8af4f4449dc1bc10455ba2ca82f668802cd78db1effeeeaeebec0c\": container with ID starting with adc609c7dc8af4f4449dc1bc10455ba2ca82f668802cd78db1effeeeaeebec0c not found: ID does not exist" Jul 06 19:39:20.942425 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:39:20.942332 2566 scope.go:117] "RemoveContainer" containerID="4fcc0a8e1f5057e73628ba3e3ccba7a57c1909092cefd0f93f84e174988b0ce4" Jul 06 19:39:20.942646 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:39:20.942619 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4fcc0a8e1f5057e73628ba3e3ccba7a57c1909092cefd0f93f84e174988b0ce4\": container with ID starting with 4fcc0a8e1f5057e73628ba3e3ccba7a57c1909092cefd0f93f84e174988b0ce4 not found: ID does not exist" containerID="4fcc0a8e1f5057e73628ba3e3ccba7a57c1909092cefd0f93f84e174988b0ce4" Jul 06 19:39:20.942686 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:39:20.942660 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4fcc0a8e1f5057e73628ba3e3ccba7a57c1909092cefd0f93f84e174988b0ce4"} err="failed to get container status \"4fcc0a8e1f5057e73628ba3e3ccba7a57c1909092cefd0f93f84e174988b0ce4\": rpc error: code = NotFound desc = could not find container \"4fcc0a8e1f5057e73628ba3e3ccba7a57c1909092cefd0f93f84e174988b0ce4\": container with ID starting with 4fcc0a8e1f5057e73628ba3e3ccba7a57c1909092cefd0f93f84e174988b0ce4 not found: ID does not exist" Jul 06 19:39:20.945133 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:39:20.945111 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-7f0c7a26-openshift-default-7dd7f6bd87-9w2z4"] Jul 06 19:39:20.950689 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:39:20.950652 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-7f0c7a26-openshift-default-7dd7f6bd87-9w2z4"] Jul 06 19:39:21.240849 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:39:21.240764 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0e63126f-87e1-470d-95bf-e503af0c3092" path="/var/lib/kubelet/pods/0e63126f-87e1-470d-95bf-e503af0c3092/volumes" Jul 06 19:39:24.919364 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:39:24.919333 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:39:25.021177 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:39:25.021145 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:39:25.645694 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:39:25.645656 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:39:31.756429 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:39:31.756393 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:39:31.858214 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:39:31.858174 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:39:31.957320 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:39:31.957289 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:39:42.489139 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:39:42.489102 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:39:42.819352 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:39:42.819319 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:40:03.751765 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:03.751707 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:40:09.668429 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:09.668394 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:40:09.768577 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:09.768535 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:40:10.119820 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:10.119789 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:40:10.330318 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:10.330272 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-8cc450-openshift-default-697477bb7f-chh9s"] Jul 06 19:40:10.330841 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:10.330827 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e63126f-87e1-470d-95bf-e503af0c3092" containerName="istio-proxy" Jul 06 19:40:10.330884 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:10.330846 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e63126f-87e1-470d-95bf-e503af0c3092" containerName="istio-proxy" Jul 06 19:40:10.330884 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:10.330865 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e63126f-87e1-470d-95bf-e503af0c3092" containerName="istio-proxy" Jul 06 19:40:10.330884 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:10.330874 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e63126f-87e1-470d-95bf-e503af0c3092" containerName="istio-proxy" Jul 06 19:40:10.330978 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:10.330971 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="0e63126f-87e1-470d-95bf-e503af0c3092" containerName="istio-proxy" Jul 06 19:40:10.331010 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:10.330981 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="0e63126f-87e1-470d-95bf-e503af0c3092" containerName="istio-proxy" Jul 06 19:40:10.334507 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:10.334481 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-8cc450-openshift-default-697477bb7f-chh9s" Jul 06 19:40:10.337756 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:10.337733 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-8cc450-openshift-default-dockercfg-4rk7l\"" Jul 06 19:40:10.349699 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:10.349672 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-8cc450-openshift-default-697477bb7f-chh9s"] Jul 06 19:40:10.367464 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:10.367426 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:40:10.446045 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:10.445947 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/30805f05-ecc5-4fe4-a00f-4b3c5c2a066b-istio-podinfo\") pod \"e2e-inf-a-8cc450-openshift-default-697477bb7f-chh9s\" (UID: \"30805f05-ecc5-4fe4-a00f-4b3c5c2a066b\") " pod="openshift-ingress/e2e-inf-a-8cc450-openshift-default-697477bb7f-chh9s" Jul 06 19:40:10.446045 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:10.445985 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/30805f05-ecc5-4fe4-a00f-4b3c5c2a066b-istio-token\") pod \"e2e-inf-a-8cc450-openshift-default-697477bb7f-chh9s\" (UID: \"30805f05-ecc5-4fe4-a00f-4b3c5c2a066b\") " pod="openshift-ingress/e2e-inf-a-8cc450-openshift-default-697477bb7f-chh9s" Jul 06 19:40:10.446045 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:10.446022 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/30805f05-ecc5-4fe4-a00f-4b3c5c2a066b-workload-certs\") pod \"e2e-inf-a-8cc450-openshift-default-697477bb7f-chh9s\" (UID: \"30805f05-ecc5-4fe4-a00f-4b3c5c2a066b\") " pod="openshift-ingress/e2e-inf-a-8cc450-openshift-default-697477bb7f-chh9s" Jul 06 19:40:10.446426 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:10.446120 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/30805f05-ecc5-4fe4-a00f-4b3c5c2a066b-credential-socket\") pod \"e2e-inf-a-8cc450-openshift-default-697477bb7f-chh9s\" (UID: \"30805f05-ecc5-4fe4-a00f-4b3c5c2a066b\") " pod="openshift-ingress/e2e-inf-a-8cc450-openshift-default-697477bb7f-chh9s" Jul 06 19:40:10.446426 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:10.446169 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qzxs8\" (UniqueName: \"kubernetes.io/projected/30805f05-ecc5-4fe4-a00f-4b3c5c2a066b-kube-api-access-qzxs8\") pod \"e2e-inf-a-8cc450-openshift-default-697477bb7f-chh9s\" (UID: \"30805f05-ecc5-4fe4-a00f-4b3c5c2a066b\") " pod="openshift-ingress/e2e-inf-a-8cc450-openshift-default-697477bb7f-chh9s" Jul 06 19:40:10.446426 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:10.446200 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/30805f05-ecc5-4fe4-a00f-4b3c5c2a066b-workload-socket\") pod \"e2e-inf-a-8cc450-openshift-default-697477bb7f-chh9s\" (UID: \"30805f05-ecc5-4fe4-a00f-4b3c5c2a066b\") " pod="openshift-ingress/e2e-inf-a-8cc450-openshift-default-697477bb7f-chh9s" Jul 06 19:40:10.446426 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:10.446224 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/30805f05-ecc5-4fe4-a00f-4b3c5c2a066b-istio-data\") pod \"e2e-inf-a-8cc450-openshift-default-697477bb7f-chh9s\" (UID: \"30805f05-ecc5-4fe4-a00f-4b3c5c2a066b\") " pod="openshift-ingress/e2e-inf-a-8cc450-openshift-default-697477bb7f-chh9s" Jul 06 19:40:10.446426 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:10.446274 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/30805f05-ecc5-4fe4-a00f-4b3c5c2a066b-istiod-ca-cert\") pod \"e2e-inf-a-8cc450-openshift-default-697477bb7f-chh9s\" (UID: \"30805f05-ecc5-4fe4-a00f-4b3c5c2a066b\") " pod="openshift-ingress/e2e-inf-a-8cc450-openshift-default-697477bb7f-chh9s" Jul 06 19:40:10.446426 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:10.446323 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/30805f05-ecc5-4fe4-a00f-4b3c5c2a066b-istio-envoy\") pod \"e2e-inf-a-8cc450-openshift-default-697477bb7f-chh9s\" (UID: \"30805f05-ecc5-4fe4-a00f-4b3c5c2a066b\") " pod="openshift-ingress/e2e-inf-a-8cc450-openshift-default-697477bb7f-chh9s" Jul 06 19:40:10.546832 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:10.546778 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/30805f05-ecc5-4fe4-a00f-4b3c5c2a066b-istiod-ca-cert\") pod \"e2e-inf-a-8cc450-openshift-default-697477bb7f-chh9s\" (UID: \"30805f05-ecc5-4fe4-a00f-4b3c5c2a066b\") " pod="openshift-ingress/e2e-inf-a-8cc450-openshift-default-697477bb7f-chh9s" Jul 06 19:40:10.546832 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:10.546834 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/30805f05-ecc5-4fe4-a00f-4b3c5c2a066b-istio-envoy\") pod \"e2e-inf-a-8cc450-openshift-default-697477bb7f-chh9s\" (UID: \"30805f05-ecc5-4fe4-a00f-4b3c5c2a066b\") " pod="openshift-ingress/e2e-inf-a-8cc450-openshift-default-697477bb7f-chh9s" Jul 06 19:40:10.547131 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:10.546873 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/30805f05-ecc5-4fe4-a00f-4b3c5c2a066b-istio-podinfo\") pod \"e2e-inf-a-8cc450-openshift-default-697477bb7f-chh9s\" (UID: \"30805f05-ecc5-4fe4-a00f-4b3c5c2a066b\") " pod="openshift-ingress/e2e-inf-a-8cc450-openshift-default-697477bb7f-chh9s" Jul 06 19:40:10.547131 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:10.546894 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/30805f05-ecc5-4fe4-a00f-4b3c5c2a066b-istio-token\") pod \"e2e-inf-a-8cc450-openshift-default-697477bb7f-chh9s\" (UID: \"30805f05-ecc5-4fe4-a00f-4b3c5c2a066b\") " pod="openshift-ingress/e2e-inf-a-8cc450-openshift-default-697477bb7f-chh9s" Jul 06 19:40:10.547131 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:10.546929 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/30805f05-ecc5-4fe4-a00f-4b3c5c2a066b-workload-certs\") pod \"e2e-inf-a-8cc450-openshift-default-697477bb7f-chh9s\" (UID: \"30805f05-ecc5-4fe4-a00f-4b3c5c2a066b\") " pod="openshift-ingress/e2e-inf-a-8cc450-openshift-default-697477bb7f-chh9s" Jul 06 19:40:10.547131 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:10.546988 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/30805f05-ecc5-4fe4-a00f-4b3c5c2a066b-credential-socket\") pod \"e2e-inf-a-8cc450-openshift-default-697477bb7f-chh9s\" (UID: \"30805f05-ecc5-4fe4-a00f-4b3c5c2a066b\") " pod="openshift-ingress/e2e-inf-a-8cc450-openshift-default-697477bb7f-chh9s" Jul 06 19:40:10.547131 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:10.547037 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qzxs8\" (UniqueName: \"kubernetes.io/projected/30805f05-ecc5-4fe4-a00f-4b3c5c2a066b-kube-api-access-qzxs8\") pod \"e2e-inf-a-8cc450-openshift-default-697477bb7f-chh9s\" (UID: \"30805f05-ecc5-4fe4-a00f-4b3c5c2a066b\") " pod="openshift-ingress/e2e-inf-a-8cc450-openshift-default-697477bb7f-chh9s" Jul 06 19:40:10.547131 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:10.547097 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/30805f05-ecc5-4fe4-a00f-4b3c5c2a066b-workload-socket\") pod \"e2e-inf-a-8cc450-openshift-default-697477bb7f-chh9s\" (UID: \"30805f05-ecc5-4fe4-a00f-4b3c5c2a066b\") " pod="openshift-ingress/e2e-inf-a-8cc450-openshift-default-697477bb7f-chh9s" Jul 06 19:40:10.547131 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:10.547121 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/30805f05-ecc5-4fe4-a00f-4b3c5c2a066b-istio-data\") pod \"e2e-inf-a-8cc450-openshift-default-697477bb7f-chh9s\" (UID: \"30805f05-ecc5-4fe4-a00f-4b3c5c2a066b\") " pod="openshift-ingress/e2e-inf-a-8cc450-openshift-default-697477bb7f-chh9s" Jul 06 19:40:10.547481 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:10.547430 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/30805f05-ecc5-4fe4-a00f-4b3c5c2a066b-credential-socket\") pod \"e2e-inf-a-8cc450-openshift-default-697477bb7f-chh9s\" (UID: \"30805f05-ecc5-4fe4-a00f-4b3c5c2a066b\") " pod="openshift-ingress/e2e-inf-a-8cc450-openshift-default-697477bb7f-chh9s" Jul 06 19:40:10.547481 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:10.547469 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/30805f05-ecc5-4fe4-a00f-4b3c5c2a066b-istio-data\") pod \"e2e-inf-a-8cc450-openshift-default-697477bb7f-chh9s\" (UID: \"30805f05-ecc5-4fe4-a00f-4b3c5c2a066b\") " pod="openshift-ingress/e2e-inf-a-8cc450-openshift-default-697477bb7f-chh9s" Jul 06 19:40:10.547748 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:10.547721 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/30805f05-ecc5-4fe4-a00f-4b3c5c2a066b-istiod-ca-cert\") pod \"e2e-inf-a-8cc450-openshift-default-697477bb7f-chh9s\" (UID: \"30805f05-ecc5-4fe4-a00f-4b3c5c2a066b\") " pod="openshift-ingress/e2e-inf-a-8cc450-openshift-default-697477bb7f-chh9s" Jul 06 19:40:10.547888 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:10.547737 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/30805f05-ecc5-4fe4-a00f-4b3c5c2a066b-workload-certs\") pod \"e2e-inf-a-8cc450-openshift-default-697477bb7f-chh9s\" (UID: \"30805f05-ecc5-4fe4-a00f-4b3c5c2a066b\") " pod="openshift-ingress/e2e-inf-a-8cc450-openshift-default-697477bb7f-chh9s" Jul 06 19:40:10.547888 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:10.547870 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/30805f05-ecc5-4fe4-a00f-4b3c5c2a066b-workload-socket\") pod \"e2e-inf-a-8cc450-openshift-default-697477bb7f-chh9s\" (UID: \"30805f05-ecc5-4fe4-a00f-4b3c5c2a066b\") " pod="openshift-ingress/e2e-inf-a-8cc450-openshift-default-697477bb7f-chh9s" Jul 06 19:40:10.549539 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:10.549520 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/30805f05-ecc5-4fe4-a00f-4b3c5c2a066b-istio-podinfo\") pod \"e2e-inf-a-8cc450-openshift-default-697477bb7f-chh9s\" (UID: \"30805f05-ecc5-4fe4-a00f-4b3c5c2a066b\") " pod="openshift-ingress/e2e-inf-a-8cc450-openshift-default-697477bb7f-chh9s" Jul 06 19:40:10.549633 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:10.549542 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/30805f05-ecc5-4fe4-a00f-4b3c5c2a066b-istio-envoy\") pod \"e2e-inf-a-8cc450-openshift-default-697477bb7f-chh9s\" (UID: \"30805f05-ecc5-4fe4-a00f-4b3c5c2a066b\") " pod="openshift-ingress/e2e-inf-a-8cc450-openshift-default-697477bb7f-chh9s" Jul 06 19:40:10.555685 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:10.555663 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/30805f05-ecc5-4fe4-a00f-4b3c5c2a066b-istio-token\") pod \"e2e-inf-a-8cc450-openshift-default-697477bb7f-chh9s\" (UID: \"30805f05-ecc5-4fe4-a00f-4b3c5c2a066b\") " pod="openshift-ingress/e2e-inf-a-8cc450-openshift-default-697477bb7f-chh9s" Jul 06 19:40:10.555910 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:10.555890 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qzxs8\" (UniqueName: \"kubernetes.io/projected/30805f05-ecc5-4fe4-a00f-4b3c5c2a066b-kube-api-access-qzxs8\") pod \"e2e-inf-a-8cc450-openshift-default-697477bb7f-chh9s\" (UID: \"30805f05-ecc5-4fe4-a00f-4b3c5c2a066b\") " pod="openshift-ingress/e2e-inf-a-8cc450-openshift-default-697477bb7f-chh9s" Jul 06 19:40:10.617527 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:10.617481 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:40:10.645518 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:10.645479 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-8cc450-openshift-default-697477bb7f-chh9s" Jul 06 19:40:10.793189 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:10.793160 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-8cc450-openshift-default-697477bb7f-chh9s"] Jul 06 19:40:10.794964 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:40:10.794938 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30805f05_ecc5_4fe4_a00f_4b3c5c2a066b.slice/crio-a4311d52b9c1f60acc462b876e41d196f5cf50f516fdde2ab67a85237e9f4fcb WatchSource:0}: Error finding container a4311d52b9c1f60acc462b876e41d196f5cf50f516fdde2ab67a85237e9f4fcb: Status 404 returned error can't find the container with id a4311d52b9c1f60acc462b876e41d196f5cf50f516fdde2ab67a85237e9f4fcb Jul 06 19:40:10.797448 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:10.797414 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 19:40:10.797528 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:10.797487 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 19:40:10.797528 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:10.797518 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 19:40:11.009550 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:11.009457 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:40:11.200026 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:11.199997 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-8cc450-openshift-default-697477bb7f-chh9s" event={"ID":"30805f05-ecc5-4fe4-a00f-4b3c5c2a066b","Type":"ContainerStarted","Data":"331509789e504014985c29c58e1591803aeabb4b6c28b58a1f7f16cff7cf0e84"} Jul 06 19:40:11.200258 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:11.200047 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-8cc450-openshift-default-697477bb7f-chh9s" event={"ID":"30805f05-ecc5-4fe4-a00f-4b3c5c2a066b","Type":"ContainerStarted","Data":"a4311d52b9c1f60acc462b876e41d196f5cf50f516fdde2ab67a85237e9f4fcb"} Jul 06 19:40:11.224512 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:11.224437 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-8cc450-openshift-default-697477bb7f-chh9s" podStartSLOduration=1.224416311 podStartE2EDuration="1.224416311s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:40:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:40:11.223636555 +0000 UTC m=+1836.530995503" watchObservedRunningTime="2026-07-06 19:40:11.224416311 +0000 UTC m=+1836.531775259" Jul 06 19:40:11.645940 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:11.645901 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-8cc450-openshift-default-697477bb7f-chh9s" Jul 06 19:40:11.650917 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:11.650890 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-8cc450-openshift-default-697477bb7f-chh9s" Jul 06 19:40:12.205756 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:12.205728 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-8cc450-openshift-default-697477bb7f-chh9s" Jul 06 19:40:12.206848 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:12.206829 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-8cc450-openshift-default-697477bb7f-chh9s" Jul 06 19:40:13.212279 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:13.212249 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:40:13.313817 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:13.313778 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:40:13.467250 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:13.467172 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:40:13.617409 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:13.617378 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:40:14.047989 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:14.047959 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:40:14.152819 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:14.152787 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:40:17.712889 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:17.712860 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:40:18.058473 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:18.058439 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:40:18.329657 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:18.329351 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-930663-openshift-default-bb5586749-2tmch"] Jul 06 19:40:18.333514 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:18.333494 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-930663-openshift-default-bb5586749-2tmch" Jul 06 19:40:18.335867 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:18.335846 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-930663-openshift-default-dockercfg-tlnmc\"" Jul 06 19:40:18.345056 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:18.345028 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-930663-openshift-default-bb5586749-2tmch"] Jul 06 19:40:18.358497 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:18.358470 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:40:18.407669 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:18.407636 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/be3f449b-0641-429e-a91c-11c777f5a995-istio-data\") pod \"e2e-inf-b-930663-openshift-default-bb5586749-2tmch\" (UID: \"be3f449b-0641-429e-a91c-11c777f5a995\") " pod="openshift-ingress/e2e-inf-b-930663-openshift-default-bb5586749-2tmch" Jul 06 19:40:18.407669 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:18.407671 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/be3f449b-0641-429e-a91c-11c777f5a995-istiod-ca-cert\") pod \"e2e-inf-b-930663-openshift-default-bb5586749-2tmch\" (UID: \"be3f449b-0641-429e-a91c-11c777f5a995\") " pod="openshift-ingress/e2e-inf-b-930663-openshift-default-bb5586749-2tmch" Jul 06 19:40:18.407894 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:18.407692 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-btln2\" (UniqueName: \"kubernetes.io/projected/be3f449b-0641-429e-a91c-11c777f5a995-kube-api-access-btln2\") pod \"e2e-inf-b-930663-openshift-default-bb5586749-2tmch\" (UID: \"be3f449b-0641-429e-a91c-11c777f5a995\") " pod="openshift-ingress/e2e-inf-b-930663-openshift-default-bb5586749-2tmch" Jul 06 19:40:18.407894 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:18.407786 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/be3f449b-0641-429e-a91c-11c777f5a995-workload-certs\") pod \"e2e-inf-b-930663-openshift-default-bb5586749-2tmch\" (UID: \"be3f449b-0641-429e-a91c-11c777f5a995\") " pod="openshift-ingress/e2e-inf-b-930663-openshift-default-bb5586749-2tmch" Jul 06 19:40:18.407894 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:18.407832 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/be3f449b-0641-429e-a91c-11c777f5a995-istio-podinfo\") pod \"e2e-inf-b-930663-openshift-default-bb5586749-2tmch\" (UID: \"be3f449b-0641-429e-a91c-11c777f5a995\") " pod="openshift-ingress/e2e-inf-b-930663-openshift-default-bb5586749-2tmch" Jul 06 19:40:18.407894 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:18.407884 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/be3f449b-0641-429e-a91c-11c777f5a995-credential-socket\") pod \"e2e-inf-b-930663-openshift-default-bb5586749-2tmch\" (UID: \"be3f449b-0641-429e-a91c-11c777f5a995\") " pod="openshift-ingress/e2e-inf-b-930663-openshift-default-bb5586749-2tmch" Jul 06 19:40:18.408015 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:18.407901 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/be3f449b-0641-429e-a91c-11c777f5a995-istio-envoy\") pod \"e2e-inf-b-930663-openshift-default-bb5586749-2tmch\" (UID: \"be3f449b-0641-429e-a91c-11c777f5a995\") " pod="openshift-ingress/e2e-inf-b-930663-openshift-default-bb5586749-2tmch" Jul 06 19:40:18.408015 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:18.407945 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/be3f449b-0641-429e-a91c-11c777f5a995-istio-token\") pod \"e2e-inf-b-930663-openshift-default-bb5586749-2tmch\" (UID: \"be3f449b-0641-429e-a91c-11c777f5a995\") " pod="openshift-ingress/e2e-inf-b-930663-openshift-default-bb5586749-2tmch" Jul 06 19:40:18.408015 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:18.407971 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/be3f449b-0641-429e-a91c-11c777f5a995-workload-socket\") pod \"e2e-inf-b-930663-openshift-default-bb5586749-2tmch\" (UID: \"be3f449b-0641-429e-a91c-11c777f5a995\") " pod="openshift-ingress/e2e-inf-b-930663-openshift-default-bb5586749-2tmch" Jul 06 19:40:18.462391 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:18.462356 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:40:18.508571 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:18.508524 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/be3f449b-0641-429e-a91c-11c777f5a995-istio-token\") pod \"e2e-inf-b-930663-openshift-default-bb5586749-2tmch\" (UID: \"be3f449b-0641-429e-a91c-11c777f5a995\") " pod="openshift-ingress/e2e-inf-b-930663-openshift-default-bb5586749-2tmch" Jul 06 19:40:18.508777 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:18.508589 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/be3f449b-0641-429e-a91c-11c777f5a995-workload-socket\") pod \"e2e-inf-b-930663-openshift-default-bb5586749-2tmch\" (UID: \"be3f449b-0641-429e-a91c-11c777f5a995\") " pod="openshift-ingress/e2e-inf-b-930663-openshift-default-bb5586749-2tmch" Jul 06 19:40:18.508777 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:18.508634 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/be3f449b-0641-429e-a91c-11c777f5a995-istio-data\") pod \"e2e-inf-b-930663-openshift-default-bb5586749-2tmch\" (UID: \"be3f449b-0641-429e-a91c-11c777f5a995\") " pod="openshift-ingress/e2e-inf-b-930663-openshift-default-bb5586749-2tmch" Jul 06 19:40:18.508777 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:18.508662 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/be3f449b-0641-429e-a91c-11c777f5a995-istiod-ca-cert\") pod \"e2e-inf-b-930663-openshift-default-bb5586749-2tmch\" (UID: \"be3f449b-0641-429e-a91c-11c777f5a995\") " pod="openshift-ingress/e2e-inf-b-930663-openshift-default-bb5586749-2tmch" Jul 06 19:40:18.508777 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:18.508693 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-btln2\" (UniqueName: \"kubernetes.io/projected/be3f449b-0641-429e-a91c-11c777f5a995-kube-api-access-btln2\") pod \"e2e-inf-b-930663-openshift-default-bb5586749-2tmch\" (UID: \"be3f449b-0641-429e-a91c-11c777f5a995\") " pod="openshift-ingress/e2e-inf-b-930663-openshift-default-bb5586749-2tmch" Jul 06 19:40:18.508777 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:18.508753 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/be3f449b-0641-429e-a91c-11c777f5a995-workload-certs\") pod \"e2e-inf-b-930663-openshift-default-bb5586749-2tmch\" (UID: \"be3f449b-0641-429e-a91c-11c777f5a995\") " pod="openshift-ingress/e2e-inf-b-930663-openshift-default-bb5586749-2tmch" Jul 06 19:40:18.509081 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:18.508813 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/be3f449b-0641-429e-a91c-11c777f5a995-istio-podinfo\") pod \"e2e-inf-b-930663-openshift-default-bb5586749-2tmch\" (UID: \"be3f449b-0641-429e-a91c-11c777f5a995\") " pod="openshift-ingress/e2e-inf-b-930663-openshift-default-bb5586749-2tmch" Jul 06 19:40:18.509081 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:18.508857 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/be3f449b-0641-429e-a91c-11c777f5a995-credential-socket\") pod \"e2e-inf-b-930663-openshift-default-bb5586749-2tmch\" (UID: \"be3f449b-0641-429e-a91c-11c777f5a995\") " pod="openshift-ingress/e2e-inf-b-930663-openshift-default-bb5586749-2tmch" Jul 06 19:40:18.509081 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:18.508880 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/be3f449b-0641-429e-a91c-11c777f5a995-istio-envoy\") pod \"e2e-inf-b-930663-openshift-default-bb5586749-2tmch\" (UID: \"be3f449b-0641-429e-a91c-11c777f5a995\") " pod="openshift-ingress/e2e-inf-b-930663-openshift-default-bb5586749-2tmch" Jul 06 19:40:18.509081 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:18.509050 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/be3f449b-0641-429e-a91c-11c777f5a995-workload-socket\") pod \"e2e-inf-b-930663-openshift-default-bb5586749-2tmch\" (UID: \"be3f449b-0641-429e-a91c-11c777f5a995\") " pod="openshift-ingress/e2e-inf-b-930663-openshift-default-bb5586749-2tmch" Jul 06 19:40:18.510260 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:18.509420 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/be3f449b-0641-429e-a91c-11c777f5a995-istio-data\") pod \"e2e-inf-b-930663-openshift-default-bb5586749-2tmch\" (UID: \"be3f449b-0641-429e-a91c-11c777f5a995\") " pod="openshift-ingress/e2e-inf-b-930663-openshift-default-bb5586749-2tmch" Jul 06 19:40:18.510260 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:18.509545 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/be3f449b-0641-429e-a91c-11c777f5a995-workload-certs\") pod \"e2e-inf-b-930663-openshift-default-bb5586749-2tmch\" (UID: \"be3f449b-0641-429e-a91c-11c777f5a995\") " pod="openshift-ingress/e2e-inf-b-930663-openshift-default-bb5586749-2tmch" Jul 06 19:40:18.510260 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:18.510206 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/be3f449b-0641-429e-a91c-11c777f5a995-istiod-ca-cert\") pod \"e2e-inf-b-930663-openshift-default-bb5586749-2tmch\" (UID: \"be3f449b-0641-429e-a91c-11c777f5a995\") " pod="openshift-ingress/e2e-inf-b-930663-openshift-default-bb5586749-2tmch" Jul 06 19:40:18.510761 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:18.510735 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/be3f449b-0641-429e-a91c-11c777f5a995-credential-socket\") pod \"e2e-inf-b-930663-openshift-default-bb5586749-2tmch\" (UID: \"be3f449b-0641-429e-a91c-11c777f5a995\") " pod="openshift-ingress/e2e-inf-b-930663-openshift-default-bb5586749-2tmch" Jul 06 19:40:18.511720 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:18.511695 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/be3f449b-0641-429e-a91c-11c777f5a995-istio-envoy\") pod \"e2e-inf-b-930663-openshift-default-bb5586749-2tmch\" (UID: \"be3f449b-0641-429e-a91c-11c777f5a995\") " pod="openshift-ingress/e2e-inf-b-930663-openshift-default-bb5586749-2tmch" Jul 06 19:40:18.511833 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:18.511777 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/be3f449b-0641-429e-a91c-11c777f5a995-istio-podinfo\") pod \"e2e-inf-b-930663-openshift-default-bb5586749-2tmch\" (UID: \"be3f449b-0641-429e-a91c-11c777f5a995\") " pod="openshift-ingress/e2e-inf-b-930663-openshift-default-bb5586749-2tmch" Jul 06 19:40:18.519567 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:18.519532 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-btln2\" (UniqueName: \"kubernetes.io/projected/be3f449b-0641-429e-a91c-11c777f5a995-kube-api-access-btln2\") pod \"e2e-inf-b-930663-openshift-default-bb5586749-2tmch\" (UID: \"be3f449b-0641-429e-a91c-11c777f5a995\") " pod="openshift-ingress/e2e-inf-b-930663-openshift-default-bb5586749-2tmch" Jul 06 19:40:18.519675 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:18.519575 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/be3f449b-0641-429e-a91c-11c777f5a995-istio-token\") pod \"e2e-inf-b-930663-openshift-default-bb5586749-2tmch\" (UID: \"be3f449b-0641-429e-a91c-11c777f5a995\") " pod="openshift-ingress/e2e-inf-b-930663-openshift-default-bb5586749-2tmch" Jul 06 19:40:18.646133 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:18.646009 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-930663-openshift-default-bb5586749-2tmch" Jul 06 19:40:18.789527 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:18.789499 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-930663-openshift-default-bb5586749-2tmch"] Jul 06 19:40:18.791139 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:40:18.791106 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe3f449b_0641_429e_a91c_11c777f5a995.slice/crio-c271bb24e60292e483f4911116ec402f163bea6cee387ee9abbd7e704f92c00f WatchSource:0}: Error finding container c271bb24e60292e483f4911116ec402f163bea6cee387ee9abbd7e704f92c00f: Status 404 returned error can't find the container with id c271bb24e60292e483f4911116ec402f163bea6cee387ee9abbd7e704f92c00f Jul 06 19:40:18.793226 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:18.793196 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 19:40:18.793322 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:18.793263 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 19:40:18.793322 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:18.793294 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 19:40:19.246699 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:19.246657 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-930663-openshift-default-bb5586749-2tmch" event={"ID":"be3f449b-0641-429e-a91c-11c777f5a995","Type":"ContainerStarted","Data":"76ca795812c4d6e9bd516812ca46092faaaf42582d86a77141e4b2c49ace7fef"} Jul 06 19:40:19.246909 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:19.246724 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-930663-openshift-default-bb5586749-2tmch" event={"ID":"be3f449b-0641-429e-a91c-11c777f5a995","Type":"ContainerStarted","Data":"c271bb24e60292e483f4911116ec402f163bea6cee387ee9abbd7e704f92c00f"} Jul 06 19:40:19.278973 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:19.278918 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-930663-openshift-default-bb5586749-2tmch" podStartSLOduration=1.278898618 podStartE2EDuration="1.278898618s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:40:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:40:19.274471999 +0000 UTC m=+1844.581830948" watchObservedRunningTime="2026-07-06 19:40:19.278898618 +0000 UTC m=+1844.586257566" Jul 06 19:40:19.647255 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:19.647211 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-930663-openshift-default-bb5586749-2tmch" Jul 06 19:40:19.651888 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:19.651863 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-930663-openshift-default-bb5586749-2tmch" Jul 06 19:40:20.252710 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:20.252682 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-930663-openshift-default-bb5586749-2tmch" Jul 06 19:40:20.253616 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:20.253598 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-930663-openshift-default-bb5586749-2tmch" Jul 06 19:40:21.405789 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:21.405749 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:40:21.506173 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:21.506141 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:40:21.603273 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:21.603242 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:40:21.949678 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:21.949648 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:40:22.131755 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:22.131722 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:40:22.232680 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:22.232586 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:40:25.833324 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:25.833294 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:40:27.947218 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:27.946706 2566 patch_prober.go:28] interesting pod/e2e-inf-a-8cc450-openshift-default-697477bb7f-chh9s container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": read tcp 10.132.0.2:58042->10.132.0.60:15021: read: connection reset by peer" start-of-body= Jul 06 19:40:27.947218 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:27.946783 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-8cc450-openshift-default-697477bb7f-chh9s" podUID="30805f05-ecc5-4fe4-a00f-4b3c5c2a066b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": read tcp 10.132.0.2:58042->10.132.0.60:15021: read: connection reset by peer" Jul 06 19:40:28.300822 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:28.300790 2566 generic.go:356] "Generic (PLEG): container finished" podID="30805f05-ecc5-4fe4-a00f-4b3c5c2a066b" containerID="331509789e504014985c29c58e1591803aeabb4b6c28b58a1f7f16cff7cf0e84" exitCode=0 Jul 06 19:40:28.300988 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:28.300866 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-8cc450-openshift-default-697477bb7f-chh9s" event={"ID":"30805f05-ecc5-4fe4-a00f-4b3c5c2a066b","Type":"ContainerDied","Data":"331509789e504014985c29c58e1591803aeabb4b6c28b58a1f7f16cff7cf0e84"} Jul 06 19:40:28.301340 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:28.301326 2566 scope.go:117] "RemoveContainer" containerID="331509789e504014985c29c58e1591803aeabb4b6c28b58a1f7f16cff7cf0e84" Jul 06 19:40:28.302355 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:28.302319 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 19:40:28.302479 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:28.302398 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 19:40:28.302479 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:28.302456 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 19:40:29.308558 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:29.308523 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-8cc450-openshift-default-697477bb7f-chh9s" event={"ID":"30805f05-ecc5-4fe4-a00f-4b3c5c2a066b","Type":"ContainerStarted","Data":"005f7dca0c4c70c536c7dd9cd4059be15b95dc1cdddc3309a274e35741df9fac"} Jul 06 19:40:29.646757 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:29.646655 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-8cc450-openshift-default-697477bb7f-chh9s" Jul 06 19:40:29.647491 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:29.647465 2566 patch_prober.go:28] interesting pod/e2e-inf-a-8cc450-openshift-default-697477bb7f-chh9s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body= Jul 06 19:40:29.647711 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:29.647679 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-8cc450-openshift-default-697477bb7f-chh9s" podUID="30805f05-ecc5-4fe4-a00f-4b3c5c2a066b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" Jul 06 19:40:30.646262 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:30.646221 2566 patch_prober.go:28] interesting pod/e2e-inf-a-8cc450-openshift-default-697477bb7f-chh9s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body= Jul 06 19:40:30.646770 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:30.646296 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-8cc450-openshift-default-697477bb7f-chh9s" podUID="30805f05-ecc5-4fe4-a00f-4b3c5c2a066b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" Jul 06 19:40:31.645963 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:31.645921 2566 patch_prober.go:28] interesting pod/e2e-inf-a-8cc450-openshift-default-697477bb7f-chh9s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body= Jul 06 19:40:31.646334 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:31.646303 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-8cc450-openshift-default-697477bb7f-chh9s" podUID="30805f05-ecc5-4fe4-a00f-4b3c5c2a066b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" Jul 06 19:40:32.646826 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:32.646781 2566 patch_prober.go:28] interesting pod/e2e-inf-a-8cc450-openshift-default-697477bb7f-chh9s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body= Jul 06 19:40:32.647476 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:32.647439 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-8cc450-openshift-default-697477bb7f-chh9s" podUID="30805f05-ecc5-4fe4-a00f-4b3c5c2a066b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" Jul 06 19:40:33.649734 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:33.649652 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-8cc450-openshift-default-697477bb7f-chh9s" Jul 06 19:40:33.650170 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:33.650102 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-8cc450-openshift-default-697477bb7f-chh9s" Jul 06 19:40:33.650813 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:33.650796 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-8cc450-openshift-default-697477bb7f-chh9s" Jul 06 19:40:34.226532 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:34.226501 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:40:35.410107 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:35.410079 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:40:36.177099 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:36.177033 2566 patch_prober.go:28] interesting pod/e2e-inf-b-930663-openshift-default-bb5586749-2tmch container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": read tcp 10.132.0.2:50616->10.132.0.61:15021: read: connection reset by peer" start-of-body= Jul 06 19:40:36.177270 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:36.177133 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-930663-openshift-default-bb5586749-2tmch" podUID="be3f449b-0641-429e-a91c-11c777f5a995" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": read tcp 10.132.0.2:50616->10.132.0.61:15021: read: connection reset by peer" Jul 06 19:40:36.358053 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:36.357975 2566 generic.go:356] "Generic (PLEG): container finished" podID="be3f449b-0641-429e-a91c-11c777f5a995" containerID="76ca795812c4d6e9bd516812ca46092faaaf42582d86a77141e4b2c49ace7fef" exitCode=0 Jul 06 19:40:36.358219 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:36.358048 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-930663-openshift-default-bb5586749-2tmch" event={"ID":"be3f449b-0641-429e-a91c-11c777f5a995","Type":"ContainerDied","Data":"76ca795812c4d6e9bd516812ca46092faaaf42582d86a77141e4b2c49ace7fef"} Jul 06 19:40:36.358481 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:36.358457 2566 scope.go:117] "RemoveContainer" containerID="76ca795812c4d6e9bd516812ca46092faaaf42582d86a77141e4b2c49ace7fef" Jul 06 19:40:36.359462 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:36.359430 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 19:40:36.359551 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:36.359501 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 19:40:36.359592 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:36.359549 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 19:40:37.367941 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:37.367907 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-930663-openshift-default-bb5586749-2tmch" event={"ID":"be3f449b-0641-429e-a91c-11c777f5a995","Type":"ContainerStarted","Data":"d7fed498453c94a326b00b884062ec4ef15fd2c48cdb5f967ff5b1a627d15daf"} Jul 06 19:40:37.646880 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:37.646771 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-930663-openshift-default-bb5586749-2tmch" Jul 06 19:40:37.647195 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:37.646849 2566 patch_prober.go:28] interesting pod/e2e-inf-b-930663-openshift-default-bb5586749-2tmch container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Jul 06 19:40:37.647413 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:37.647376 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-930663-openshift-default-bb5586749-2tmch" podUID="be3f449b-0641-429e-a91c-11c777f5a995" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Jul 06 19:40:38.646563 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:38.646524 2566 patch_prober.go:28] interesting pod/e2e-inf-b-930663-openshift-default-bb5586749-2tmch container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Jul 06 19:40:38.647203 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:38.647169 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-930663-openshift-default-bb5586749-2tmch" podUID="be3f449b-0641-429e-a91c-11c777f5a995" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Jul 06 19:40:39.647322 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:39.647280 2566 patch_prober.go:28] interesting pod/e2e-inf-b-930663-openshift-default-bb5586749-2tmch container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Jul 06 19:40:39.647907 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:39.647873 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-930663-openshift-default-bb5586749-2tmch" podUID="be3f449b-0641-429e-a91c-11c777f5a995" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Jul 06 19:40:40.646556 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:40.646507 2566 patch_prober.go:28] interesting pod/e2e-inf-b-930663-openshift-default-bb5586749-2tmch container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Jul 06 19:40:40.646781 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:40.646582 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-930663-openshift-default-bb5586749-2tmch" podUID="be3f449b-0641-429e-a91c-11c777f5a995" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Jul 06 19:40:41.650395 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:41.650366 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-930663-openshift-default-bb5586749-2tmch" Jul 06 19:40:41.650835 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:41.650779 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-930663-openshift-default-bb5586749-2tmch" Jul 06 19:40:41.651592 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:41.651570 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-930663-openshift-default-bb5586749-2tmch" Jul 06 19:40:44.061355 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:44.061320 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:40:44.161553 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:40:44.161524 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:41:00.504656 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:41:00.504573 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:41:24.283116 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:41:24.283058 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:41:25.282446 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:41:25.282411 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:41:25.566376 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:41:25.566339 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:41:25.934338 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:41:25.934257 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:41:26.036547 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:41:26.036510 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:41:26.167661 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:41:26.167633 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:41:26.296319 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:41:26.296288 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:41:31.004636 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:41:31.004598 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:41:38.742847 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:41:38.742814 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:42:07.937549 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:07.937506 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:42:08.214543 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:08.214452 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-8cc450-openshift-default-697477bb7f-chh9s"] Jul 06 19:42:08.214715 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:08.214667 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-8cc450-openshift-default-697477bb7f-chh9s" podUID="30805f05-ecc5-4fe4-a00f-4b3c5c2a066b" containerName="istio-proxy" containerID="cri-o://005f7dca0c4c70c536c7dd9cd4059be15b95dc1cdddc3309a274e35741df9fac" gracePeriod=30 Jul 06 19:42:08.216055 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:08.216030 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:42:08.441269 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:08.441232 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:42:08.610474 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:08.610442 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:42:09.013342 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:09.013257 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:42:09.117327 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:09.117294 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:42:09.215341 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:09.215305 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:42:09.337797 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:09.337760 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:42:11.991639 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:11.991605 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:42:12.091512 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:12.091478 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:42:13.468781 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:13.468758 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-8cc450-openshift-default-697477bb7f-chh9s" Jul 06 19:42:13.587936 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:13.587888 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/30805f05-ecc5-4fe4-a00f-4b3c5c2a066b-istiod-ca-cert\") pod \"30805f05-ecc5-4fe4-a00f-4b3c5c2a066b\" (UID: \"30805f05-ecc5-4fe4-a00f-4b3c5c2a066b\") " Jul 06 19:42:13.587936 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:13.587948 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/30805f05-ecc5-4fe4-a00f-4b3c5c2a066b-istio-token\") pod \"30805f05-ecc5-4fe4-a00f-4b3c5c2a066b\" (UID: \"30805f05-ecc5-4fe4-a00f-4b3c5c2a066b\") " Jul 06 19:42:13.588266 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:13.587974 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/30805f05-ecc5-4fe4-a00f-4b3c5c2a066b-istio-podinfo\") pod \"30805f05-ecc5-4fe4-a00f-4b3c5c2a066b\" (UID: \"30805f05-ecc5-4fe4-a00f-4b3c5c2a066b\") " Jul 06 19:42:13.588266 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:13.587999 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/30805f05-ecc5-4fe4-a00f-4b3c5c2a066b-workload-certs\") pod \"30805f05-ecc5-4fe4-a00f-4b3c5c2a066b\" (UID: \"30805f05-ecc5-4fe4-a00f-4b3c5c2a066b\") " Jul 06 19:42:13.588266 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:13.588020 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/30805f05-ecc5-4fe4-a00f-4b3c5c2a066b-credential-socket\") pod \"30805f05-ecc5-4fe4-a00f-4b3c5c2a066b\" (UID: \"30805f05-ecc5-4fe4-a00f-4b3c5c2a066b\") " Jul 06 19:42:13.588266 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:13.588040 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/30805f05-ecc5-4fe4-a00f-4b3c5c2a066b-istio-envoy\") pod \"30805f05-ecc5-4fe4-a00f-4b3c5c2a066b\" (UID: \"30805f05-ecc5-4fe4-a00f-4b3c5c2a066b\") " Jul 06 19:42:13.588266 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:13.588111 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qzxs8\" (UniqueName: \"kubernetes.io/projected/30805f05-ecc5-4fe4-a00f-4b3c5c2a066b-kube-api-access-qzxs8\") pod \"30805f05-ecc5-4fe4-a00f-4b3c5c2a066b\" (UID: \"30805f05-ecc5-4fe4-a00f-4b3c5c2a066b\") " Jul 06 19:42:13.588266 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:13.588140 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/30805f05-ecc5-4fe4-a00f-4b3c5c2a066b-workload-socket\") pod \"30805f05-ecc5-4fe4-a00f-4b3c5c2a066b\" (UID: \"30805f05-ecc5-4fe4-a00f-4b3c5c2a066b\") " Jul 06 19:42:13.588266 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:13.588223 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/30805f05-ecc5-4fe4-a00f-4b3c5c2a066b-istio-data\") pod \"30805f05-ecc5-4fe4-a00f-4b3c5c2a066b\" (UID: \"30805f05-ecc5-4fe4-a00f-4b3c5c2a066b\") " Jul 06 19:42:13.588625 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:13.588449 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30805f05-ecc5-4fe4-a00f-4b3c5c2a066b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "30805f05-ecc5-4fe4-a00f-4b3c5c2a066b" (UID: "30805f05-ecc5-4fe4-a00f-4b3c5c2a066b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:42:13.588625 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:13.588471 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/30805f05-ecc5-4fe4-a00f-4b3c5c2a066b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "30805f05-ecc5-4fe4-a00f-4b3c5c2a066b" (UID: "30805f05-ecc5-4fe4-a00f-4b3c5c2a066b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:42:13.588625 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:13.588469 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30805f05-ecc5-4fe4-a00f-4b3c5c2a066b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "30805f05-ecc5-4fe4-a00f-4b3c5c2a066b" (UID: "30805f05-ecc5-4fe4-a00f-4b3c5c2a066b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:42:13.588625 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:13.588533 2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/30805f05-ecc5-4fe4-a00f-4b3c5c2a066b-workload-certs\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:42:13.588999 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:13.588681 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30805f05-ecc5-4fe4-a00f-4b3c5c2a066b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "30805f05-ecc5-4fe4-a00f-4b3c5c2a066b" (UID: "30805f05-ecc5-4fe4-a00f-4b3c5c2a066b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:42:13.589215 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:13.589194 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30805f05-ecc5-4fe4-a00f-4b3c5c2a066b-istio-data" (OuterVolumeSpecName: "istio-data") pod "30805f05-ecc5-4fe4-a00f-4b3c5c2a066b" (UID: "30805f05-ecc5-4fe4-a00f-4b3c5c2a066b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:42:13.590423 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:13.590395 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/30805f05-ecc5-4fe4-a00f-4b3c5c2a066b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "30805f05-ecc5-4fe4-a00f-4b3c5c2a066b" (UID: "30805f05-ecc5-4fe4-a00f-4b3c5c2a066b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 19:42:13.590523 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:13.590433 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30805f05-ecc5-4fe4-a00f-4b3c5c2a066b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "30805f05-ecc5-4fe4-a00f-4b3c5c2a066b" (UID: "30805f05-ecc5-4fe4-a00f-4b3c5c2a066b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:42:13.590888 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:13.590872 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/30805f05-ecc5-4fe4-a00f-4b3c5c2a066b-kube-api-access-qzxs8" (OuterVolumeSpecName: "kube-api-access-qzxs8") pod "30805f05-ecc5-4fe4-a00f-4b3c5c2a066b" (UID: "30805f05-ecc5-4fe4-a00f-4b3c5c2a066b"). InnerVolumeSpecName "kube-api-access-qzxs8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:42:13.590888 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:13.590880 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/30805f05-ecc5-4fe4-a00f-4b3c5c2a066b-istio-token" (OuterVolumeSpecName: "istio-token") pod "30805f05-ecc5-4fe4-a00f-4b3c5c2a066b" (UID: "30805f05-ecc5-4fe4-a00f-4b3c5c2a066b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:42:13.689901 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:13.689867 2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/30805f05-ecc5-4fe4-a00f-4b3c5c2a066b-istiod-ca-cert\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:42:13.689901 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:13.689898 2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/30805f05-ecc5-4fe4-a00f-4b3c5c2a066b-istio-token\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:42:13.689901 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:13.689908 2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/30805f05-ecc5-4fe4-a00f-4b3c5c2a066b-istio-podinfo\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:42:13.690176 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:13.689916 2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/30805f05-ecc5-4fe4-a00f-4b3c5c2a066b-credential-socket\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:42:13.690176 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:13.689927 2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/30805f05-ecc5-4fe4-a00f-4b3c5c2a066b-istio-envoy\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:42:13.690176 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:13.689936 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qzxs8\" (UniqueName: \"kubernetes.io/projected/30805f05-ecc5-4fe4-a00f-4b3c5c2a066b-kube-api-access-qzxs8\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:42:13.690176 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:13.689952 2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/30805f05-ecc5-4fe4-a00f-4b3c5c2a066b-workload-socket\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:42:13.690176 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:13.689962 2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/30805f05-ecc5-4fe4-a00f-4b3c5c2a066b-istio-data\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:42:13.908881 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:13.908794 2566 generic.go:356] "Generic (PLEG): container finished" podID="30805f05-ecc5-4fe4-a00f-4b3c5c2a066b" containerID="005f7dca0c4c70c536c7dd9cd4059be15b95dc1cdddc3309a274e35741df9fac" exitCode=0 Jul 06 19:42:13.908881 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:13.908843 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-8cc450-openshift-default-697477bb7f-chh9s" Jul 06 19:42:13.908881 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:13.908870 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-8cc450-openshift-default-697477bb7f-chh9s" event={"ID":"30805f05-ecc5-4fe4-a00f-4b3c5c2a066b","Type":"ContainerDied","Data":"005f7dca0c4c70c536c7dd9cd4059be15b95dc1cdddc3309a274e35741df9fac"} Jul 06 19:42:13.909124 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:13.908924 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-8cc450-openshift-default-697477bb7f-chh9s" event={"ID":"30805f05-ecc5-4fe4-a00f-4b3c5c2a066b","Type":"ContainerDied","Data":"a4311d52b9c1f60acc462b876e41d196f5cf50f516fdde2ab67a85237e9f4fcb"} Jul 06 19:42:13.909124 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:13.908945 2566 scope.go:117] "RemoveContainer" containerID="005f7dca0c4c70c536c7dd9cd4059be15b95dc1cdddc3309a274e35741df9fac" Jul 06 19:42:13.918934 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:13.918915 2566 scope.go:117] "RemoveContainer" containerID="331509789e504014985c29c58e1591803aeabb4b6c28b58a1f7f16cff7cf0e84" Jul 06 19:42:13.927848 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:13.927826 2566 scope.go:117] "RemoveContainer" containerID="005f7dca0c4c70c536c7dd9cd4059be15b95dc1cdddc3309a274e35741df9fac" Jul 06 19:42:13.928128 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:42:13.928102 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"005f7dca0c4c70c536c7dd9cd4059be15b95dc1cdddc3309a274e35741df9fac\": container with ID starting with 005f7dca0c4c70c536c7dd9cd4059be15b95dc1cdddc3309a274e35741df9fac not found: ID does not exist" containerID="005f7dca0c4c70c536c7dd9cd4059be15b95dc1cdddc3309a274e35741df9fac" Jul 06 19:42:13.928225 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:13.928140 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"005f7dca0c4c70c536c7dd9cd4059be15b95dc1cdddc3309a274e35741df9fac"} err="failed to get container status \"005f7dca0c4c70c536c7dd9cd4059be15b95dc1cdddc3309a274e35741df9fac\": rpc error: code = NotFound desc = could not find container \"005f7dca0c4c70c536c7dd9cd4059be15b95dc1cdddc3309a274e35741df9fac\": container with ID starting with 005f7dca0c4c70c536c7dd9cd4059be15b95dc1cdddc3309a274e35741df9fac not found: ID does not exist" Jul 06 19:42:13.928225 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:13.928157 2566 scope.go:117] "RemoveContainer" containerID="331509789e504014985c29c58e1591803aeabb4b6c28b58a1f7f16cff7cf0e84" Jul 06 19:42:13.928374 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:42:13.928360 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"331509789e504014985c29c58e1591803aeabb4b6c28b58a1f7f16cff7cf0e84\": container with ID starting with 331509789e504014985c29c58e1591803aeabb4b6c28b58a1f7f16cff7cf0e84 not found: ID does not exist" containerID="331509789e504014985c29c58e1591803aeabb4b6c28b58a1f7f16cff7cf0e84" Jul 06 19:42:13.928418 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:13.928380 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"331509789e504014985c29c58e1591803aeabb4b6c28b58a1f7f16cff7cf0e84"} err="failed to get container status \"331509789e504014985c29c58e1591803aeabb4b6c28b58a1f7f16cff7cf0e84\": rpc error: code = NotFound desc = could not find container \"331509789e504014985c29c58e1591803aeabb4b6c28b58a1f7f16cff7cf0e84\": container with ID starting with 331509789e504014985c29c58e1591803aeabb4b6c28b58a1f7f16cff7cf0e84 not found: ID does not exist" Jul 06 19:42:13.933607 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:13.933585 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-8cc450-openshift-default-697477bb7f-chh9s"] Jul 06 19:42:13.941369 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:13.941341 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-8cc450-openshift-default-697477bb7f-chh9s"] Jul 06 19:42:14.941974 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:14.941920 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:42:15.067132 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:15.067102 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:42:15.207842 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:15.207746 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-930663-openshift-default-bb5586749-2tmch"] Jul 06 19:42:15.208020 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:15.207987 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-930663-openshift-default-bb5586749-2tmch" podUID="be3f449b-0641-429e-a91c-11c777f5a995" containerName="istio-proxy" containerID="cri-o://d7fed498453c94a326b00b884062ec4ef15fd2c48cdb5f967ff5b1a627d15daf" gracePeriod=30 Jul 06 19:42:15.223307 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:15.223279 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:42:15.236796 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:15.236765 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="30805f05-ecc5-4fe4-a00f-4b3c5c2a066b" path="/var/lib/kubelet/pods/30805f05-ecc5-4fe4-a00f-4b3c5c2a066b/volumes" Jul 06 19:42:15.481055 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:15.480969 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:42:15.812872 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:15.812840 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:42:16.144136 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:16.144026 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-3fd8ac-openshift-default-7b9dcc85-6csm2"] Jul 06 19:42:16.144643 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:16.144624 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="30805f05-ecc5-4fe4-a00f-4b3c5c2a066b" containerName="istio-proxy" Jul 06 19:42:16.144726 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:16.144647 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="30805f05-ecc5-4fe4-a00f-4b3c5c2a066b" containerName="istio-proxy" Jul 06 19:42:16.144726 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:16.144666 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="30805f05-ecc5-4fe4-a00f-4b3c5c2a066b" containerName="istio-proxy" Jul 06 19:42:16.144726 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:16.144674 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="30805f05-ecc5-4fe4-a00f-4b3c5c2a066b" containerName="istio-proxy" Jul 06 19:42:16.144868 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:16.144776 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="30805f05-ecc5-4fe4-a00f-4b3c5c2a066b" containerName="istio-proxy" Jul 06 19:42:16.151509 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:16.149339 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-3fd8ac-openshift-default-7b9dcc85-6csm2" Jul 06 19:42:16.155245 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:16.155206 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-3fd8ac-openshift-default-dockercfg-qllgz\"" Jul 06 19:42:16.175679 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:16.175643 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-3fd8ac-openshift-default-7b9dcc85-6csm2"] Jul 06 19:42:16.205475 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:16.205434 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:42:16.213671 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:16.213635 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e656ebd-0edb-4769-83f2-27ad450e322a-workload-socket\") pod \"e2e-api-a-3fd8ac-openshift-default-7b9dcc85-6csm2\" (UID: \"6e656ebd-0edb-4769-83f2-27ad450e322a\") " pod="openshift-ingress/e2e-api-a-3fd8ac-openshift-default-7b9dcc85-6csm2" Jul 06 19:42:16.213671 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:16.213673 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e656ebd-0edb-4769-83f2-27ad450e322a-workload-certs\") pod \"e2e-api-a-3fd8ac-openshift-default-7b9dcc85-6csm2\" (UID: \"6e656ebd-0edb-4769-83f2-27ad450e322a\") " pod="openshift-ingress/e2e-api-a-3fd8ac-openshift-default-7b9dcc85-6csm2" Jul 06 19:42:16.213947 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:16.213693 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6e656ebd-0edb-4769-83f2-27ad450e322a-istio-podinfo\") pod \"e2e-api-a-3fd8ac-openshift-default-7b9dcc85-6csm2\" (UID: \"6e656ebd-0edb-4769-83f2-27ad450e322a\") " pod="openshift-ingress/e2e-api-a-3fd8ac-openshift-default-7b9dcc85-6csm2" Jul 06 19:42:16.213947 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:16.213767 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e656ebd-0edb-4769-83f2-27ad450e322a-istio-token\") pod \"e2e-api-a-3fd8ac-openshift-default-7b9dcc85-6csm2\" (UID: \"6e656ebd-0edb-4769-83f2-27ad450e322a\") " pod="openshift-ingress/e2e-api-a-3fd8ac-openshift-default-7b9dcc85-6csm2" Jul 06 19:42:16.213947 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:16.213864 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v6vnl\" (UniqueName: \"kubernetes.io/projected/6e656ebd-0edb-4769-83f2-27ad450e322a-kube-api-access-v6vnl\") pod \"e2e-api-a-3fd8ac-openshift-default-7b9dcc85-6csm2\" (UID: \"6e656ebd-0edb-4769-83f2-27ad450e322a\") " pod="openshift-ingress/e2e-api-a-3fd8ac-openshift-default-7b9dcc85-6csm2" Jul 06 19:42:16.213947 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:16.213926 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e656ebd-0edb-4769-83f2-27ad450e322a-credential-socket\") pod \"e2e-api-a-3fd8ac-openshift-default-7b9dcc85-6csm2\" (UID: \"6e656ebd-0edb-4769-83f2-27ad450e322a\") " pod="openshift-ingress/e2e-api-a-3fd8ac-openshift-default-7b9dcc85-6csm2" Jul 06 19:42:16.214192 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:16.213958 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6e656ebd-0edb-4769-83f2-27ad450e322a-istio-data\") pod \"e2e-api-a-3fd8ac-openshift-default-7b9dcc85-6csm2\" (UID: \"6e656ebd-0edb-4769-83f2-27ad450e322a\") " pod="openshift-ingress/e2e-api-a-3fd8ac-openshift-default-7b9dcc85-6csm2" Jul 06 19:42:16.214192 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:16.213998 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6e656ebd-0edb-4769-83f2-27ad450e322a-istio-envoy\") pod \"e2e-api-a-3fd8ac-openshift-default-7b9dcc85-6csm2\" (UID: \"6e656ebd-0edb-4769-83f2-27ad450e322a\") " pod="openshift-ingress/e2e-api-a-3fd8ac-openshift-default-7b9dcc85-6csm2" Jul 06 19:42:16.214192 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:16.214020 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6e656ebd-0edb-4769-83f2-27ad450e322a-istiod-ca-cert\") pod \"e2e-api-a-3fd8ac-openshift-default-7b9dcc85-6csm2\" (UID: \"6e656ebd-0edb-4769-83f2-27ad450e322a\") " pod="openshift-ingress/e2e-api-a-3fd8ac-openshift-default-7b9dcc85-6csm2" Jul 06 19:42:16.314629 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:16.314587 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e656ebd-0edb-4769-83f2-27ad450e322a-credential-socket\") pod \"e2e-api-a-3fd8ac-openshift-default-7b9dcc85-6csm2\" (UID: \"6e656ebd-0edb-4769-83f2-27ad450e322a\") " pod="openshift-ingress/e2e-api-a-3fd8ac-openshift-default-7b9dcc85-6csm2" Jul 06 19:42:16.314629 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:16.314635 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6e656ebd-0edb-4769-83f2-27ad450e322a-istio-data\") pod \"e2e-api-a-3fd8ac-openshift-default-7b9dcc85-6csm2\" (UID: \"6e656ebd-0edb-4769-83f2-27ad450e322a\") " pod="openshift-ingress/e2e-api-a-3fd8ac-openshift-default-7b9dcc85-6csm2" Jul 06 19:42:16.314908 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:16.314763 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6e656ebd-0edb-4769-83f2-27ad450e322a-istio-envoy\") pod \"e2e-api-a-3fd8ac-openshift-default-7b9dcc85-6csm2\" (UID: \"6e656ebd-0edb-4769-83f2-27ad450e322a\") " pod="openshift-ingress/e2e-api-a-3fd8ac-openshift-default-7b9dcc85-6csm2" Jul 06 19:42:16.314908 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:16.314814 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6e656ebd-0edb-4769-83f2-27ad450e322a-istiod-ca-cert\") pod \"e2e-api-a-3fd8ac-openshift-default-7b9dcc85-6csm2\" (UID: \"6e656ebd-0edb-4769-83f2-27ad450e322a\") " pod="openshift-ingress/e2e-api-a-3fd8ac-openshift-default-7b9dcc85-6csm2" Jul 06 19:42:16.314908 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:16.314887 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e656ebd-0edb-4769-83f2-27ad450e322a-workload-socket\") pod \"e2e-api-a-3fd8ac-openshift-default-7b9dcc85-6csm2\" (UID: \"6e656ebd-0edb-4769-83f2-27ad450e322a\") " pod="openshift-ingress/e2e-api-a-3fd8ac-openshift-default-7b9dcc85-6csm2" Jul 06 19:42:16.315060 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:16.314913 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e656ebd-0edb-4769-83f2-27ad450e322a-workload-certs\") pod \"e2e-api-a-3fd8ac-openshift-default-7b9dcc85-6csm2\" (UID: \"6e656ebd-0edb-4769-83f2-27ad450e322a\") " pod="openshift-ingress/e2e-api-a-3fd8ac-openshift-default-7b9dcc85-6csm2" Jul 06 19:42:16.315060 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:16.314948 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6e656ebd-0edb-4769-83f2-27ad450e322a-istio-podinfo\") pod \"e2e-api-a-3fd8ac-openshift-default-7b9dcc85-6csm2\" (UID: \"6e656ebd-0edb-4769-83f2-27ad450e322a\") " pod="openshift-ingress/e2e-api-a-3fd8ac-openshift-default-7b9dcc85-6csm2" Jul 06 19:42:16.315060 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:16.314996 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e656ebd-0edb-4769-83f2-27ad450e322a-istio-token\") pod \"e2e-api-a-3fd8ac-openshift-default-7b9dcc85-6csm2\" (UID: \"6e656ebd-0edb-4769-83f2-27ad450e322a\") " pod="openshift-ingress/e2e-api-a-3fd8ac-openshift-default-7b9dcc85-6csm2" Jul 06 19:42:16.315060 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:16.315041 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6e656ebd-0edb-4769-83f2-27ad450e322a-istio-data\") pod \"e2e-api-a-3fd8ac-openshift-default-7b9dcc85-6csm2\" (UID: \"6e656ebd-0edb-4769-83f2-27ad450e322a\") " pod="openshift-ingress/e2e-api-a-3fd8ac-openshift-default-7b9dcc85-6csm2" Jul 06 19:42:16.315060 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:16.315056 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v6vnl\" (UniqueName: \"kubernetes.io/projected/6e656ebd-0edb-4769-83f2-27ad450e322a-kube-api-access-v6vnl\") pod \"e2e-api-a-3fd8ac-openshift-default-7b9dcc85-6csm2\" (UID: \"6e656ebd-0edb-4769-83f2-27ad450e322a\") " pod="openshift-ingress/e2e-api-a-3fd8ac-openshift-default-7b9dcc85-6csm2" Jul 06 19:42:16.315547 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:16.315472 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e656ebd-0edb-4769-83f2-27ad450e322a-workload-socket\") pod \"e2e-api-a-3fd8ac-openshift-default-7b9dcc85-6csm2\" (UID: \"6e656ebd-0edb-4769-83f2-27ad450e322a\") " pod="openshift-ingress/e2e-api-a-3fd8ac-openshift-default-7b9dcc85-6csm2" Jul 06 19:42:16.315700 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:16.315561 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e656ebd-0edb-4769-83f2-27ad450e322a-credential-socket\") pod \"e2e-api-a-3fd8ac-openshift-default-7b9dcc85-6csm2\" (UID: \"6e656ebd-0edb-4769-83f2-27ad450e322a\") " pod="openshift-ingress/e2e-api-a-3fd8ac-openshift-default-7b9dcc85-6csm2" Jul 06 19:42:16.315700 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:16.315521 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e656ebd-0edb-4769-83f2-27ad450e322a-workload-certs\") pod \"e2e-api-a-3fd8ac-openshift-default-7b9dcc85-6csm2\" (UID: \"6e656ebd-0edb-4769-83f2-27ad450e322a\") " pod="openshift-ingress/e2e-api-a-3fd8ac-openshift-default-7b9dcc85-6csm2" Jul 06 19:42:16.315819 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:16.315760 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6e656ebd-0edb-4769-83f2-27ad450e322a-istiod-ca-cert\") pod \"e2e-api-a-3fd8ac-openshift-default-7b9dcc85-6csm2\" (UID: \"6e656ebd-0edb-4769-83f2-27ad450e322a\") " pod="openshift-ingress/e2e-api-a-3fd8ac-openshift-default-7b9dcc85-6csm2" Jul 06 19:42:16.317171 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:16.317153 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6e656ebd-0edb-4769-83f2-27ad450e322a-istio-envoy\") pod \"e2e-api-a-3fd8ac-openshift-default-7b9dcc85-6csm2\" (UID: \"6e656ebd-0edb-4769-83f2-27ad450e322a\") " pod="openshift-ingress/e2e-api-a-3fd8ac-openshift-default-7b9dcc85-6csm2" Jul 06 19:42:16.317857 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:16.317840 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6e656ebd-0edb-4769-83f2-27ad450e322a-istio-podinfo\") pod \"e2e-api-a-3fd8ac-openshift-default-7b9dcc85-6csm2\" (UID: \"6e656ebd-0edb-4769-83f2-27ad450e322a\") " pod="openshift-ingress/e2e-api-a-3fd8ac-openshift-default-7b9dcc85-6csm2" Jul 06 19:42:16.324012 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:16.323985 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e656ebd-0edb-4769-83f2-27ad450e322a-istio-token\") pod \"e2e-api-a-3fd8ac-openshift-default-7b9dcc85-6csm2\" (UID: \"6e656ebd-0edb-4769-83f2-27ad450e322a\") " pod="openshift-ingress/e2e-api-a-3fd8ac-openshift-default-7b9dcc85-6csm2" Jul 06 19:42:16.324869 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:16.324842 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v6vnl\" (UniqueName: \"kubernetes.io/projected/6e656ebd-0edb-4769-83f2-27ad450e322a-kube-api-access-v6vnl\") pod \"e2e-api-a-3fd8ac-openshift-default-7b9dcc85-6csm2\" (UID: \"6e656ebd-0edb-4769-83f2-27ad450e322a\") " pod="openshift-ingress/e2e-api-a-3fd8ac-openshift-default-7b9dcc85-6csm2" Jul 06 19:42:16.465652 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:16.465553 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-3fd8ac-openshift-default-7b9dcc85-6csm2" Jul 06 19:42:16.623816 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:16.623784 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-3fd8ac-openshift-default-7b9dcc85-6csm2"] Jul 06 19:42:16.624405 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:42:16.624383 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e656ebd_0edb_4769_83f2_27ad450e322a.slice/crio-76a0bbbb4a6d817d9d35863f7ad48d7c3bb5d874e18d71a87533a3db5c369602 WatchSource:0}: Error finding container 76a0bbbb4a6d817d9d35863f7ad48d7c3bb5d874e18d71a87533a3db5c369602: Status 404 returned error can't find the container with id 76a0bbbb4a6d817d9d35863f7ad48d7c3bb5d874e18d71a87533a3db5c369602 Jul 06 19:42:16.626418 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:16.626385 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 19:42:16.626509 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:16.626451 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 19:42:16.626509 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:16.626477 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 19:42:16.682413 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:16.682384 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:42:16.928237 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:16.928203 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-3fd8ac-openshift-default-7b9dcc85-6csm2" event={"ID":"6e656ebd-0edb-4769-83f2-27ad450e322a","Type":"ContainerStarted","Data":"a23648c50649cea06798a5ad2c1f21999bc47534040f8540e9ba0054db96e4fb"} Jul 06 19:42:16.928429 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:16.928258 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-3fd8ac-openshift-default-7b9dcc85-6csm2" event={"ID":"6e656ebd-0edb-4769-83f2-27ad450e322a","Type":"ContainerStarted","Data":"76a0bbbb4a6d817d9d35863f7ad48d7c3bb5d874e18d71a87533a3db5c369602"} Jul 06 19:42:16.968521 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:16.968464 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-3fd8ac-openshift-default-7b9dcc85-6csm2" podStartSLOduration=0.968448769 podStartE2EDuration="968.448769ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:42:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:42:16.966447993 +0000 UTC m=+1962.273806943" watchObservedRunningTime="2026-07-06 19:42:16.968448769 +0000 UTC m=+1962.275807717" Jul 06 19:42:17.466757 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:17.466723 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-3fd8ac-openshift-default-7b9dcc85-6csm2" Jul 06 19:42:17.471542 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:17.471520 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-3fd8ac-openshift-default-7b9dcc85-6csm2" Jul 06 19:42:17.933879 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:17.933846 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-3fd8ac-openshift-default-7b9dcc85-6csm2" Jul 06 19:42:17.934709 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:17.934689 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-3fd8ac-openshift-default-7b9dcc85-6csm2" Jul 06 19:42:18.954246 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:18.954208 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:42:19.051771 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:19.051741 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:42:19.173821 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:19.173786 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:42:19.301149 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:19.301108 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:42:19.676090 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:19.675966 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:42:20.033763 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:20.033684 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:42:20.467347 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:20.467323 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-930663-openshift-default-bb5586749-2tmch" Jul 06 19:42:20.555288 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:20.555253 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/be3f449b-0641-429e-a91c-11c777f5a995-istio-token\") pod \"be3f449b-0641-429e-a91c-11c777f5a995\" (UID: \"be3f449b-0641-429e-a91c-11c777f5a995\") " Jul 06 19:42:20.555489 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:20.555321 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/be3f449b-0641-429e-a91c-11c777f5a995-workload-socket\") pod \"be3f449b-0641-429e-a91c-11c777f5a995\" (UID: \"be3f449b-0641-429e-a91c-11c777f5a995\") " Jul 06 19:42:20.555489 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:20.555342 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/be3f449b-0641-429e-a91c-11c777f5a995-istio-envoy\") pod \"be3f449b-0641-429e-a91c-11c777f5a995\" (UID: \"be3f449b-0641-429e-a91c-11c777f5a995\") " Jul 06 19:42:20.555489 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:20.555368 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-btln2\" (UniqueName: \"kubernetes.io/projected/be3f449b-0641-429e-a91c-11c777f5a995-kube-api-access-btln2\") pod \"be3f449b-0641-429e-a91c-11c777f5a995\" (UID: \"be3f449b-0641-429e-a91c-11c777f5a995\") " Jul 06 19:42:20.555489 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:20.555384 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/be3f449b-0641-429e-a91c-11c777f5a995-istiod-ca-cert\") pod \"be3f449b-0641-429e-a91c-11c777f5a995\" (UID: \"be3f449b-0641-429e-a91c-11c777f5a995\") " Jul 06 19:42:20.555489 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:20.555450 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/be3f449b-0641-429e-a91c-11c777f5a995-istio-podinfo\") pod \"be3f449b-0641-429e-a91c-11c777f5a995\" (UID: \"be3f449b-0641-429e-a91c-11c777f5a995\") " Jul 06 19:42:20.555489 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:20.555472 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/be3f449b-0641-429e-a91c-11c777f5a995-credential-socket\") pod \"be3f449b-0641-429e-a91c-11c777f5a995\" (UID: \"be3f449b-0641-429e-a91c-11c777f5a995\") " Jul 06 19:42:20.555811 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:20.555701 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/be3f449b-0641-429e-a91c-11c777f5a995-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "be3f449b-0641-429e-a91c-11c777f5a995" (UID: "be3f449b-0641-429e-a91c-11c777f5a995"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:42:20.555898 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:20.555879 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/be3f449b-0641-429e-a91c-11c777f5a995-istio-data\") pod \"be3f449b-0641-429e-a91c-11c777f5a995\" (UID: \"be3f449b-0641-429e-a91c-11c777f5a995\") " Jul 06 19:42:20.555959 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:20.555917 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/be3f449b-0641-429e-a91c-11c777f5a995-workload-certs\") pod \"be3f449b-0641-429e-a91c-11c777f5a995\" (UID: \"be3f449b-0641-429e-a91c-11c777f5a995\") " Jul 06 19:42:20.555959 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:20.555923 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/be3f449b-0641-429e-a91c-11c777f5a995-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "be3f449b-0641-429e-a91c-11c777f5a995" (UID: "be3f449b-0641-429e-a91c-11c777f5a995"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:42:20.555959 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:20.555930 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/be3f449b-0641-429e-a91c-11c777f5a995-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "be3f449b-0641-429e-a91c-11c777f5a995" (UID: "be3f449b-0641-429e-a91c-11c777f5a995"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:42:20.556416 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:20.556378 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/be3f449b-0641-429e-a91c-11c777f5a995-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "be3f449b-0641-429e-a91c-11c777f5a995" (UID: "be3f449b-0641-429e-a91c-11c777f5a995"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:42:20.556566 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:20.556547 2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/be3f449b-0641-429e-a91c-11c777f5a995-workload-certs\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:42:20.556650 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:20.556573 2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/be3f449b-0641-429e-a91c-11c777f5a995-workload-socket\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:42:20.556650 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:20.556589 2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/be3f449b-0641-429e-a91c-11c777f5a995-istiod-ca-cert\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:42:20.556650 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:20.556606 2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/be3f449b-0641-429e-a91c-11c777f5a995-credential-socket\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:42:20.556822 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:20.556797 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/be3f449b-0641-429e-a91c-11c777f5a995-istio-data" (OuterVolumeSpecName: "istio-data") pod "be3f449b-0641-429e-a91c-11c777f5a995" (UID: "be3f449b-0641-429e-a91c-11c777f5a995"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:42:20.557831 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:20.557806 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/be3f449b-0641-429e-a91c-11c777f5a995-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "be3f449b-0641-429e-a91c-11c777f5a995" (UID: "be3f449b-0641-429e-a91c-11c777f5a995"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 19:42:20.557915 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:20.557854 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/be3f449b-0641-429e-a91c-11c777f5a995-istio-token" (OuterVolumeSpecName: "istio-token") pod "be3f449b-0641-429e-a91c-11c777f5a995" (UID: "be3f449b-0641-429e-a91c-11c777f5a995"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:42:20.557915 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:20.557873 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/be3f449b-0641-429e-a91c-11c777f5a995-kube-api-access-btln2" (OuterVolumeSpecName: "kube-api-access-btln2") pod "be3f449b-0641-429e-a91c-11c777f5a995" (UID: "be3f449b-0641-429e-a91c-11c777f5a995"). InnerVolumeSpecName "kube-api-access-btln2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:42:20.558002 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:20.557916 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/be3f449b-0641-429e-a91c-11c777f5a995-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "be3f449b-0641-429e-a91c-11c777f5a995" (UID: "be3f449b-0641-429e-a91c-11c777f5a995"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:42:20.657716 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:20.657626 2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/be3f449b-0641-429e-a91c-11c777f5a995-istio-data\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:42:20.657716 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:20.657656 2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/be3f449b-0641-429e-a91c-11c777f5a995-istio-token\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:42:20.657716 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:20.657664 2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/be3f449b-0641-429e-a91c-11c777f5a995-istio-envoy\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:42:20.657716 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:20.657672 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-btln2\" (UniqueName: \"kubernetes.io/projected/be3f449b-0641-429e-a91c-11c777f5a995-kube-api-access-btln2\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:42:20.657716 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:20.657683 2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/be3f449b-0641-429e-a91c-11c777f5a995-istio-podinfo\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:42:20.953711 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:20.953625 2566 generic.go:356] "Generic (PLEG): container finished" podID="be3f449b-0641-429e-a91c-11c777f5a995" containerID="d7fed498453c94a326b00b884062ec4ef15fd2c48cdb5f967ff5b1a627d15daf" exitCode=0 Jul 06 19:42:20.953711 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:20.953671 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-930663-openshift-default-bb5586749-2tmch" Jul 06 19:42:20.953711 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:20.953682 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-930663-openshift-default-bb5586749-2tmch" event={"ID":"be3f449b-0641-429e-a91c-11c777f5a995","Type":"ContainerDied","Data":"d7fed498453c94a326b00b884062ec4ef15fd2c48cdb5f967ff5b1a627d15daf"} Jul 06 19:42:20.953951 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:20.953724 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-930663-openshift-default-bb5586749-2tmch" event={"ID":"be3f449b-0641-429e-a91c-11c777f5a995","Type":"ContainerDied","Data":"c271bb24e60292e483f4911116ec402f163bea6cee387ee9abbd7e704f92c00f"} Jul 06 19:42:20.953951 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:20.953745 2566 scope.go:117] "RemoveContainer" containerID="d7fed498453c94a326b00b884062ec4ef15fd2c48cdb5f967ff5b1a627d15daf" Jul 06 19:42:20.963784 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:20.963764 2566 scope.go:117] "RemoveContainer" containerID="76ca795812c4d6e9bd516812ca46092faaaf42582d86a77141e4b2c49ace7fef" Jul 06 19:42:20.971931 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:20.971912 2566 scope.go:117] "RemoveContainer" containerID="d7fed498453c94a326b00b884062ec4ef15fd2c48cdb5f967ff5b1a627d15daf" Jul 06 19:42:20.972210 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:42:20.972191 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d7fed498453c94a326b00b884062ec4ef15fd2c48cdb5f967ff5b1a627d15daf\": container with ID starting with d7fed498453c94a326b00b884062ec4ef15fd2c48cdb5f967ff5b1a627d15daf not found: ID does not exist" containerID="d7fed498453c94a326b00b884062ec4ef15fd2c48cdb5f967ff5b1a627d15daf" Jul 06 19:42:20.972273 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:20.972222 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7fed498453c94a326b00b884062ec4ef15fd2c48cdb5f967ff5b1a627d15daf"} err="failed to get container status \"d7fed498453c94a326b00b884062ec4ef15fd2c48cdb5f967ff5b1a627d15daf\": rpc error: code = NotFound desc = could not find container \"d7fed498453c94a326b00b884062ec4ef15fd2c48cdb5f967ff5b1a627d15daf\": container with ID starting with d7fed498453c94a326b00b884062ec4ef15fd2c48cdb5f967ff5b1a627d15daf not found: ID does not exist" Jul 06 19:42:20.972273 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:20.972240 2566 scope.go:117] "RemoveContainer" containerID="76ca795812c4d6e9bd516812ca46092faaaf42582d86a77141e4b2c49ace7fef" Jul 06 19:42:20.972508 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:42:20.972483 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"76ca795812c4d6e9bd516812ca46092faaaf42582d86a77141e4b2c49ace7fef\": container with ID starting with 76ca795812c4d6e9bd516812ca46092faaaf42582d86a77141e4b2c49ace7fef not found: ID does not exist" containerID="76ca795812c4d6e9bd516812ca46092faaaf42582d86a77141e4b2c49ace7fef" Jul 06 19:42:20.972562 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:20.972514 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"76ca795812c4d6e9bd516812ca46092faaaf42582d86a77141e4b2c49ace7fef"} err="failed to get container status \"76ca795812c4d6e9bd516812ca46092faaaf42582d86a77141e4b2c49ace7fef\": rpc error: code = NotFound desc = could not find container \"76ca795812c4d6e9bd516812ca46092faaaf42582d86a77141e4b2c49ace7fef\": container with ID starting with 76ca795812c4d6e9bd516812ca46092faaaf42582d86a77141e4b2c49ace7fef not found: ID does not exist" Jul 06 19:42:20.976550 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:20.976526 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-930663-openshift-default-bb5586749-2tmch"] Jul 06 19:42:20.980946 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:20.980923 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-930663-openshift-default-bb5586749-2tmch"] Jul 06 19:42:21.236513 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:21.236429 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="be3f449b-0641-429e-a91c-11c777f5a995" path="/var/lib/kubelet/pods/be3f449b-0641-429e-a91c-11c777f5a995/volumes" Jul 06 19:42:23.444328 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:23.444287 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:42:23.879341 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:23.879300 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:42:24.153475 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:24.153369 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-99607e-openshift-default-75f6fddfcc-5q98x"] Jul 06 19:42:24.153860 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:24.153840 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="be3f449b-0641-429e-a91c-11c777f5a995" containerName="istio-proxy" Jul 06 19:42:24.153930 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:24.153863 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="be3f449b-0641-429e-a91c-11c777f5a995" containerName="istio-proxy" Jul 06 19:42:24.153987 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:24.153930 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="be3f449b-0641-429e-a91c-11c777f5a995" containerName="istio-proxy" Jul 06 19:42:24.153987 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:24.153938 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="30805f05-ecc5-4fe4-a00f-4b3c5c2a066b" containerName="istio-proxy" Jul 06 19:42:24.156831 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:24.156812 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-99607e-openshift-default-75f6fddfcc-5q98x" Jul 06 19:42:24.159121 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:24.159100 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-99607e-openshift-default-dockercfg-p8mp7\"" Jul 06 19:42:24.169026 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:24.168999 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-99607e-openshift-default-75f6fddfcc-5q98x"] Jul 06 19:42:24.181039 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:24.181011 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:42:24.290374 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:24.290334 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/239e7a39-7c76-4f42-8d5d-3d7140be0550-istio-envoy\") pod \"e2e-api-b-99607e-openshift-default-75f6fddfcc-5q98x\" (UID: \"239e7a39-7c76-4f42-8d5d-3d7140be0550\") " pod="openshift-ingress/e2e-api-b-99607e-openshift-default-75f6fddfcc-5q98x" Jul 06 19:42:24.290374 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:24.290375 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/239e7a39-7c76-4f42-8d5d-3d7140be0550-istio-data\") pod \"e2e-api-b-99607e-openshift-default-75f6fddfcc-5q98x\" (UID: \"239e7a39-7c76-4f42-8d5d-3d7140be0550\") " pod="openshift-ingress/e2e-api-b-99607e-openshift-default-75f6fddfcc-5q98x" Jul 06 19:42:24.290617 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:24.290396 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tm7xf\" (UniqueName: \"kubernetes.io/projected/239e7a39-7c76-4f42-8d5d-3d7140be0550-kube-api-access-tm7xf\") pod \"e2e-api-b-99607e-openshift-default-75f6fddfcc-5q98x\" (UID: \"239e7a39-7c76-4f42-8d5d-3d7140be0550\") " pod="openshift-ingress/e2e-api-b-99607e-openshift-default-75f6fddfcc-5q98x" Jul 06 19:42:24.290617 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:24.290504 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/239e7a39-7c76-4f42-8d5d-3d7140be0550-istiod-ca-cert\") pod \"e2e-api-b-99607e-openshift-default-75f6fddfcc-5q98x\" (UID: \"239e7a39-7c76-4f42-8d5d-3d7140be0550\") " pod="openshift-ingress/e2e-api-b-99607e-openshift-default-75f6fddfcc-5q98x" Jul 06 19:42:24.290617 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:24.290549 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/239e7a39-7c76-4f42-8d5d-3d7140be0550-credential-socket\") pod \"e2e-api-b-99607e-openshift-default-75f6fddfcc-5q98x\" (UID: \"239e7a39-7c76-4f42-8d5d-3d7140be0550\") " pod="openshift-ingress/e2e-api-b-99607e-openshift-default-75f6fddfcc-5q98x" Jul 06 19:42:24.290617 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:24.290590 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/239e7a39-7c76-4f42-8d5d-3d7140be0550-workload-certs\") pod \"e2e-api-b-99607e-openshift-default-75f6fddfcc-5q98x\" (UID: \"239e7a39-7c76-4f42-8d5d-3d7140be0550\") " pod="openshift-ingress/e2e-api-b-99607e-openshift-default-75f6fddfcc-5q98x" Jul 06 19:42:24.290814 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:24.290679 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/239e7a39-7c76-4f42-8d5d-3d7140be0550-istio-podinfo\") pod \"e2e-api-b-99607e-openshift-default-75f6fddfcc-5q98x\" (UID: \"239e7a39-7c76-4f42-8d5d-3d7140be0550\") " pod="openshift-ingress/e2e-api-b-99607e-openshift-default-75f6fddfcc-5q98x" Jul 06 19:42:24.290814 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:24.290731 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/239e7a39-7c76-4f42-8d5d-3d7140be0550-workload-socket\") pod \"e2e-api-b-99607e-openshift-default-75f6fddfcc-5q98x\" (UID: \"239e7a39-7c76-4f42-8d5d-3d7140be0550\") " pod="openshift-ingress/e2e-api-b-99607e-openshift-default-75f6fddfcc-5q98x" Jul 06 19:42:24.290814 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:24.290756 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/239e7a39-7c76-4f42-8d5d-3d7140be0550-istio-token\") pod \"e2e-api-b-99607e-openshift-default-75f6fddfcc-5q98x\" (UID: \"239e7a39-7c76-4f42-8d5d-3d7140be0550\") " pod="openshift-ingress/e2e-api-b-99607e-openshift-default-75f6fddfcc-5q98x" Jul 06 19:42:24.391992 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:24.391956 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/239e7a39-7c76-4f42-8d5d-3d7140be0550-istio-podinfo\") pod \"e2e-api-b-99607e-openshift-default-75f6fddfcc-5q98x\" (UID: \"239e7a39-7c76-4f42-8d5d-3d7140be0550\") " pod="openshift-ingress/e2e-api-b-99607e-openshift-default-75f6fddfcc-5q98x" Jul 06 19:42:24.392222 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:24.392001 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/239e7a39-7c76-4f42-8d5d-3d7140be0550-workload-socket\") pod \"e2e-api-b-99607e-openshift-default-75f6fddfcc-5q98x\" (UID: \"239e7a39-7c76-4f42-8d5d-3d7140be0550\") " pod="openshift-ingress/e2e-api-b-99607e-openshift-default-75f6fddfcc-5q98x" Jul 06 19:42:24.392222 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:24.392020 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/239e7a39-7c76-4f42-8d5d-3d7140be0550-istio-token\") pod \"e2e-api-b-99607e-openshift-default-75f6fddfcc-5q98x\" (UID: \"239e7a39-7c76-4f42-8d5d-3d7140be0550\") " pod="openshift-ingress/e2e-api-b-99607e-openshift-default-75f6fddfcc-5q98x" Jul 06 19:42:24.392222 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:24.392047 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/239e7a39-7c76-4f42-8d5d-3d7140be0550-istio-envoy\") pod \"e2e-api-b-99607e-openshift-default-75f6fddfcc-5q98x\" (UID: \"239e7a39-7c76-4f42-8d5d-3d7140be0550\") " pod="openshift-ingress/e2e-api-b-99607e-openshift-default-75f6fddfcc-5q98x" Jul 06 19:42:24.392400 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:24.392260 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/239e7a39-7c76-4f42-8d5d-3d7140be0550-istio-data\") pod \"e2e-api-b-99607e-openshift-default-75f6fddfcc-5q98x\" (UID: \"239e7a39-7c76-4f42-8d5d-3d7140be0550\") " pod="openshift-ingress/e2e-api-b-99607e-openshift-default-75f6fddfcc-5q98x" Jul 06 19:42:24.392400 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:24.392302 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tm7xf\" (UniqueName: \"kubernetes.io/projected/239e7a39-7c76-4f42-8d5d-3d7140be0550-kube-api-access-tm7xf\") pod \"e2e-api-b-99607e-openshift-default-75f6fddfcc-5q98x\" (UID: \"239e7a39-7c76-4f42-8d5d-3d7140be0550\") " pod="openshift-ingress/e2e-api-b-99607e-openshift-default-75f6fddfcc-5q98x" Jul 06 19:42:24.392400 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:24.392352 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/239e7a39-7c76-4f42-8d5d-3d7140be0550-istiod-ca-cert\") pod \"e2e-api-b-99607e-openshift-default-75f6fddfcc-5q98x\" (UID: \"239e7a39-7c76-4f42-8d5d-3d7140be0550\") " pod="openshift-ingress/e2e-api-b-99607e-openshift-default-75f6fddfcc-5q98x" Jul 06 19:42:24.392400 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:24.392384 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/239e7a39-7c76-4f42-8d5d-3d7140be0550-credential-socket\") pod \"e2e-api-b-99607e-openshift-default-75f6fddfcc-5q98x\" (UID: \"239e7a39-7c76-4f42-8d5d-3d7140be0550\") " pod="openshift-ingress/e2e-api-b-99607e-openshift-default-75f6fddfcc-5q98x" Jul 06 19:42:24.392607 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:24.392427 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/239e7a39-7c76-4f42-8d5d-3d7140be0550-workload-certs\") pod \"e2e-api-b-99607e-openshift-default-75f6fddfcc-5q98x\" (UID: \"239e7a39-7c76-4f42-8d5d-3d7140be0550\") " pod="openshift-ingress/e2e-api-b-99607e-openshift-default-75f6fddfcc-5q98x" Jul 06 19:42:24.392607 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:24.392519 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/239e7a39-7c76-4f42-8d5d-3d7140be0550-workload-socket\") pod \"e2e-api-b-99607e-openshift-default-75f6fddfcc-5q98x\" (UID: \"239e7a39-7c76-4f42-8d5d-3d7140be0550\") " pod="openshift-ingress/e2e-api-b-99607e-openshift-default-75f6fddfcc-5q98x" Jul 06 19:42:24.392607 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:24.392584 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/239e7a39-7c76-4f42-8d5d-3d7140be0550-istio-data\") pod \"e2e-api-b-99607e-openshift-default-75f6fddfcc-5q98x\" (UID: \"239e7a39-7c76-4f42-8d5d-3d7140be0550\") " pod="openshift-ingress/e2e-api-b-99607e-openshift-default-75f6fddfcc-5q98x" Jul 06 19:42:24.392779 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:24.392760 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/239e7a39-7c76-4f42-8d5d-3d7140be0550-workload-certs\") pod \"e2e-api-b-99607e-openshift-default-75f6fddfcc-5q98x\" (UID: \"239e7a39-7c76-4f42-8d5d-3d7140be0550\") " pod="openshift-ingress/e2e-api-b-99607e-openshift-default-75f6fddfcc-5q98x" Jul 06 19:42:24.392868 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:24.392845 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/239e7a39-7c76-4f42-8d5d-3d7140be0550-credential-socket\") pod \"e2e-api-b-99607e-openshift-default-75f6fddfcc-5q98x\" (UID: \"239e7a39-7c76-4f42-8d5d-3d7140be0550\") " pod="openshift-ingress/e2e-api-b-99607e-openshift-default-75f6fddfcc-5q98x" Jul 06 19:42:24.393168 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:24.393148 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/239e7a39-7c76-4f42-8d5d-3d7140be0550-istiod-ca-cert\") pod \"e2e-api-b-99607e-openshift-default-75f6fddfcc-5q98x\" (UID: \"239e7a39-7c76-4f42-8d5d-3d7140be0550\") " pod="openshift-ingress/e2e-api-b-99607e-openshift-default-75f6fddfcc-5q98x" Jul 06 19:42:24.394536 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:24.394517 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/239e7a39-7c76-4f42-8d5d-3d7140be0550-istio-envoy\") pod \"e2e-api-b-99607e-openshift-default-75f6fddfcc-5q98x\" (UID: \"239e7a39-7c76-4f42-8d5d-3d7140be0550\") " pod="openshift-ingress/e2e-api-b-99607e-openshift-default-75f6fddfcc-5q98x" Jul 06 19:42:24.394672 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:24.394646 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/239e7a39-7c76-4f42-8d5d-3d7140be0550-istio-podinfo\") pod \"e2e-api-b-99607e-openshift-default-75f6fddfcc-5q98x\" (UID: \"239e7a39-7c76-4f42-8d5d-3d7140be0550\") " pod="openshift-ingress/e2e-api-b-99607e-openshift-default-75f6fddfcc-5q98x" Jul 06 19:42:24.401832 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:24.401812 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/239e7a39-7c76-4f42-8d5d-3d7140be0550-istio-token\") pod \"e2e-api-b-99607e-openshift-default-75f6fddfcc-5q98x\" (UID: \"239e7a39-7c76-4f42-8d5d-3d7140be0550\") " pod="openshift-ingress/e2e-api-b-99607e-openshift-default-75f6fddfcc-5q98x" Jul 06 19:42:24.401981 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:24.401964 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tm7xf\" (UniqueName: \"kubernetes.io/projected/239e7a39-7c76-4f42-8d5d-3d7140be0550-kube-api-access-tm7xf\") pod \"e2e-api-b-99607e-openshift-default-75f6fddfcc-5q98x\" (UID: \"239e7a39-7c76-4f42-8d5d-3d7140be0550\") " pod="openshift-ingress/e2e-api-b-99607e-openshift-default-75f6fddfcc-5q98x" Jul 06 19:42:24.438114 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:24.437979 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:42:24.467907 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:24.467873 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-99607e-openshift-default-75f6fddfcc-5q98x" Jul 06 19:42:24.623062 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:24.623034 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-99607e-openshift-default-75f6fddfcc-5q98x"] Jul 06 19:42:24.625345 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:42:24.625304 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod239e7a39_7c76_4f42_8d5d_3d7140be0550.slice/crio-bccd96bbf8ecc12a00b5824e5f18aa1469d256dfa04811b6be052bf67027c9f7 WatchSource:0}: Error finding container bccd96bbf8ecc12a00b5824e5f18aa1469d256dfa04811b6be052bf67027c9f7: Status 404 returned error can't find the container with id bccd96bbf8ecc12a00b5824e5f18aa1469d256dfa04811b6be052bf67027c9f7 Jul 06 19:42:24.627600 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:24.627559 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 19:42:24.627686 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:24.627627 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 19:42:24.627686 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:24.627655 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 19:42:24.978136 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:24.978044 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-99607e-openshift-default-75f6fddfcc-5q98x" event={"ID":"239e7a39-7c76-4f42-8d5d-3d7140be0550","Type":"ContainerStarted","Data":"d2fccf0c5ebee0d801d5e22ddd2db76d82a55db3d33bb14a2a967c16111c3316"} Jul 06 19:42:24.978322 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:24.978163 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-99607e-openshift-default-75f6fddfcc-5q98x" event={"ID":"239e7a39-7c76-4f42-8d5d-3d7140be0550","Type":"ContainerStarted","Data":"bccd96bbf8ecc12a00b5824e5f18aa1469d256dfa04811b6be052bf67027c9f7"} Jul 06 19:42:25.000921 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:25.000862 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-99607e-openshift-default-75f6fddfcc-5q98x" podStartSLOduration=1.00084333 podStartE2EDuration="1.00084333s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:42:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:42:24.999388918 +0000 UTC m=+1970.306747866" watchObservedRunningTime="2026-07-06 19:42:25.00084333 +0000 UTC m=+1970.308202280" Jul 06 19:42:25.468248 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:25.468206 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-99607e-openshift-default-75f6fddfcc-5q98x" Jul 06 19:42:25.473559 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:25.473530 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-99607e-openshift-default-75f6fddfcc-5q98x" Jul 06 19:42:25.983773 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:25.983733 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-99607e-openshift-default-75f6fddfcc-5q98x" Jul 06 19:42:25.984638 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:25.984612 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-99607e-openshift-default-75f6fddfcc-5q98x" Jul 06 19:42:27.039487 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:27.039454 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:42:27.145331 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:27.145290 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:42:27.253048 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:27.253007 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:42:27.748621 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:27.748586 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:42:27.849770 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:27.849734 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:42:28.499523 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:28.499482 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:42:31.470368 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:31.470287 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:42:33.897737 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:33.897706 2566 patch_prober.go:28] interesting pod/e2e-api-a-3fd8ac-openshift-default-7b9dcc85-6csm2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": read tcp 10.132.0.2:54898->10.132.0.62:15021: read: connection reset by peer" start-of-body= Jul 06 19:42:33.898144 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:33.897791 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-3fd8ac-openshift-default-7b9dcc85-6csm2" podUID="6e656ebd-0edb-4769-83f2-27ad450e322a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": read tcp 10.132.0.2:54898->10.132.0.62:15021: read: connection reset by peer" Jul 06 19:42:34.035162 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:34.035122 2566 generic.go:356] "Generic (PLEG): container finished" podID="6e656ebd-0edb-4769-83f2-27ad450e322a" containerID="a23648c50649cea06798a5ad2c1f21999bc47534040f8540e9ba0054db96e4fb" exitCode=0 Jul 06 19:42:34.035355 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:34.035199 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-3fd8ac-openshift-default-7b9dcc85-6csm2" event={"ID":"6e656ebd-0edb-4769-83f2-27ad450e322a","Type":"ContainerDied","Data":"a23648c50649cea06798a5ad2c1f21999bc47534040f8540e9ba0054db96e4fb"} Jul 06 19:42:34.035643 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:34.035626 2566 scope.go:117] "RemoveContainer" containerID="a23648c50649cea06798a5ad2c1f21999bc47534040f8540e9ba0054db96e4fb" Jul 06 19:42:34.036777 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:34.036743 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 19:42:34.036868 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:34.036841 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 19:42:34.036913 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:34.036875 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 19:42:35.047503 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:35.047465 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-3fd8ac-openshift-default-7b9dcc85-6csm2" event={"ID":"6e656ebd-0edb-4769-83f2-27ad450e322a","Type":"ContainerStarted","Data":"da3c1508dfdba9cd6af92a4455f2f7d87156e8c8a25dcdc6adc72a9ff2abdaad"} Jul 06 19:42:35.467017 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:35.466458 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-3fd8ac-openshift-default-7b9dcc85-6csm2" Jul 06 19:42:35.467017 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:35.466594 2566 patch_prober.go:28] interesting pod/e2e-api-a-3fd8ac-openshift-default-7b9dcc85-6csm2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body= Jul 06 19:42:35.467017 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:35.466657 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-3fd8ac-openshift-default-7b9dcc85-6csm2" podUID="6e656ebd-0edb-4769-83f2-27ad450e322a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" Jul 06 19:42:36.466283 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:36.466241 2566 patch_prober.go:28] interesting pod/e2e-api-a-3fd8ac-openshift-default-7b9dcc85-6csm2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body= Jul 06 19:42:36.466897 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:36.466855 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-3fd8ac-openshift-default-7b9dcc85-6csm2" podUID="6e656ebd-0edb-4769-83f2-27ad450e322a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" Jul 06 19:42:37.466209 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:37.466163 2566 patch_prober.go:28] interesting pod/e2e-api-a-3fd8ac-openshift-default-7b9dcc85-6csm2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body= Jul 06 19:42:37.466430 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:37.466232 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-3fd8ac-openshift-default-7b9dcc85-6csm2" podUID="6e656ebd-0edb-4769-83f2-27ad450e322a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" Jul 06 19:42:38.466289 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:38.466240 2566 patch_prober.go:28] interesting pod/e2e-api-a-3fd8ac-openshift-default-7b9dcc85-6csm2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body= Jul 06 19:42:38.466556 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:38.466328 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-3fd8ac-openshift-default-7b9dcc85-6csm2" podUID="6e656ebd-0edb-4769-83f2-27ad450e322a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" Jul 06 19:42:39.470461 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:39.470426 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-3fd8ac-openshift-default-7b9dcc85-6csm2" Jul 06 19:42:39.470952 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:39.470885 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-3fd8ac-openshift-default-7b9dcc85-6csm2" Jul 06 19:42:39.471634 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:39.471612 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-3fd8ac-openshift-default-7b9dcc85-6csm2" Jul 06 19:42:39.744763 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:39.744664 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:42:40.168481 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:40.168450 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:42:40.290505 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:40.290466 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:42:40.378302 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:40.378271 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:42:40.563522 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:40.563492 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:42:42.026279 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:42.026238 2566 patch_prober.go:28] interesting pod/e2e-api-b-99607e-openshift-default-75f6fddfcc-5q98x container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 06 19:42:42.026666 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:42.026302 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-99607e-openshift-default-75f6fddfcc-5q98x" podUID="239e7a39-7c76-4f42-8d5d-3d7140be0550" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 06 19:42:43.106238 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:43.106204 2566 generic.go:356] "Generic (PLEG): container finished" podID="239e7a39-7c76-4f42-8d5d-3d7140be0550" containerID="d2fccf0c5ebee0d801d5e22ddd2db76d82a55db3d33bb14a2a967c16111c3316" exitCode=0 Jul 06 19:42:43.106644 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:43.106278 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-99607e-openshift-default-75f6fddfcc-5q98x" event={"ID":"239e7a39-7c76-4f42-8d5d-3d7140be0550","Type":"ContainerDied","Data":"d2fccf0c5ebee0d801d5e22ddd2db76d82a55db3d33bb14a2a967c16111c3316"} Jul 06 19:42:43.106692 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:43.106675 2566 scope.go:117] "RemoveContainer" containerID="d2fccf0c5ebee0d801d5e22ddd2db76d82a55db3d33bb14a2a967c16111c3316" Jul 06 19:42:43.107762 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:43.107736 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 19:42:43.107832 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:43.107819 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 19:42:43.107877 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:43.107864 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 19:42:44.113559 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:44.113530 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-99607e-openshift-default-75f6fddfcc-5q98x" event={"ID":"239e7a39-7c76-4f42-8d5d-3d7140be0550","Type":"ContainerStarted","Data":"d021a9cf9d30682203f171d37fc429ea134972305a11f8e152e4a6ef1db59de2"} Jul 06 19:42:44.468657 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:44.468570 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-99607e-openshift-default-75f6fddfcc-5q98x" Jul 06 19:42:44.468810 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:44.468757 2566 patch_prober.go:28] interesting pod/e2e-api-b-99607e-openshift-default-75f6fddfcc-5q98x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body= Jul 06 19:42:44.468848 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:44.468806 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-99607e-openshift-default-75f6fddfcc-5q98x" podUID="239e7a39-7c76-4f42-8d5d-3d7140be0550" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" Jul 06 19:42:45.468701 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:45.468661 2566 patch_prober.go:28] interesting pod/e2e-api-b-99607e-openshift-default-75f6fddfcc-5q98x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body= Jul 06 19:42:45.469122 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:45.468724 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-99607e-openshift-default-75f6fddfcc-5q98x" podUID="239e7a39-7c76-4f42-8d5d-3d7140be0550" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" Jul 06 19:42:46.317056 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:46.317024 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:42:46.418112 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:46.418053 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:42:46.468890 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:46.468851 2566 patch_prober.go:28] interesting pod/e2e-api-b-99607e-openshift-default-75f6fddfcc-5q98x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body= Jul 06 19:42:46.469295 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:46.468928 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-99607e-openshift-default-75f6fddfcc-5q98x" podUID="239e7a39-7c76-4f42-8d5d-3d7140be0550" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" Jul 06 19:42:46.573245 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:46.573147 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-3fd8ac-openshift-default-7b9dcc85-6csm2"] Jul 06 19:42:46.573416 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:46.573355 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-3fd8ac-openshift-default-7b9dcc85-6csm2" podUID="6e656ebd-0edb-4769-83f2-27ad450e322a" containerName="istio-proxy" containerID="cri-o://da3c1508dfdba9cd6af92a4455f2f7d87156e8c8a25dcdc6adc72a9ff2abdaad" gracePeriod=30 Jul 06 19:42:46.585688 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:46.585656 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:42:46.791762 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:46.791727 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:42:46.978924 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:46.978826 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:42:47.376859 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:47.376812 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:42:47.468448 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:47.468416 2566 patch_prober.go:28] interesting pod/e2e-api-b-99607e-openshift-default-75f6fddfcc-5q98x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body= Jul 06 19:42:47.468638 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:47.468469 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-99607e-openshift-default-75f6fddfcc-5q98x" podUID="239e7a39-7c76-4f42-8d5d-3d7140be0550" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" Jul 06 19:42:47.543643 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:47.543601 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:42:47.643184 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:47.642958 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:42:47.818035 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:47.817988 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:42:48.472324 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:48.472295 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-99607e-openshift-default-75f6fddfcc-5q98x" Jul 06 19:42:48.472557 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:48.472540 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-99607e-openshift-default-75f6fddfcc-5q98x" Jul 06 19:42:48.473195 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:48.473177 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-99607e-openshift-default-75f6fddfcc-5q98x" Jul 06 19:42:51.836374 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:51.836303 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-3fd8ac-openshift-default-7b9dcc85-6csm2" Jul 06 19:42:51.877255 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:51.877223 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e656ebd-0edb-4769-83f2-27ad450e322a-workload-socket\") pod \"6e656ebd-0edb-4769-83f2-27ad450e322a\" (UID: \"6e656ebd-0edb-4769-83f2-27ad450e322a\") " Jul 06 19:42:51.877405 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:51.877265 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e656ebd-0edb-4769-83f2-27ad450e322a-istio-token\") pod \"6e656ebd-0edb-4769-83f2-27ad450e322a\" (UID: \"6e656ebd-0edb-4769-83f2-27ad450e322a\") " Jul 06 19:42:51.877405 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:51.877293 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v6vnl\" (UniqueName: \"kubernetes.io/projected/6e656ebd-0edb-4769-83f2-27ad450e322a-kube-api-access-v6vnl\") pod \"6e656ebd-0edb-4769-83f2-27ad450e322a\" (UID: \"6e656ebd-0edb-4769-83f2-27ad450e322a\") " Jul 06 19:42:51.877405 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:51.877321 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6e656ebd-0edb-4769-83f2-27ad450e322a-istiod-ca-cert\") pod \"6e656ebd-0edb-4769-83f2-27ad450e322a\" (UID: \"6e656ebd-0edb-4769-83f2-27ad450e322a\") " Jul 06 19:42:51.877405 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:51.877352 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6e656ebd-0edb-4769-83f2-27ad450e322a-istio-envoy\") pod \"6e656ebd-0edb-4769-83f2-27ad450e322a\" (UID: \"6e656ebd-0edb-4769-83f2-27ad450e322a\") " Jul 06 19:42:51.877633 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:51.877412 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6e656ebd-0edb-4769-83f2-27ad450e322a-istio-podinfo\") pod \"6e656ebd-0edb-4769-83f2-27ad450e322a\" (UID: \"6e656ebd-0edb-4769-83f2-27ad450e322a\") " Jul 06 19:42:51.877633 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:51.877482 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6e656ebd-0edb-4769-83f2-27ad450e322a-istio-data\") pod \"6e656ebd-0edb-4769-83f2-27ad450e322a\" (UID: \"6e656ebd-0edb-4769-83f2-27ad450e322a\") " Jul 06 19:42:51.877633 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:51.877492 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e656ebd-0edb-4769-83f2-27ad450e322a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6e656ebd-0edb-4769-83f2-27ad450e322a" (UID: "6e656ebd-0edb-4769-83f2-27ad450e322a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:42:51.877633 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:51.877532 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e656ebd-0edb-4769-83f2-27ad450e322a-credential-socket\") pod \"6e656ebd-0edb-4769-83f2-27ad450e322a\" (UID: \"6e656ebd-0edb-4769-83f2-27ad450e322a\") " Jul 06 19:42:51.877633 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:51.877572 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e656ebd-0edb-4769-83f2-27ad450e322a-workload-certs\") pod \"6e656ebd-0edb-4769-83f2-27ad450e322a\" (UID: \"6e656ebd-0edb-4769-83f2-27ad450e322a\") " Jul 06 19:42:51.877888 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:51.877819 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6e656ebd-0edb-4769-83f2-27ad450e322a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6e656ebd-0edb-4769-83f2-27ad450e322a" (UID: "6e656ebd-0edb-4769-83f2-27ad450e322a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:42:51.877946 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:51.877934 2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e656ebd-0edb-4769-83f2-27ad450e322a-workload-socket\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:42:51.878005 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:51.877936 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e656ebd-0edb-4769-83f2-27ad450e322a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6e656ebd-0edb-4769-83f2-27ad450e322a" (UID: "6e656ebd-0edb-4769-83f2-27ad450e322a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:42:51.878005 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:51.877952 2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6e656ebd-0edb-4769-83f2-27ad450e322a-istiod-ca-cert\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:42:51.878298 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:51.878042 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e656ebd-0edb-4769-83f2-27ad450e322a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6e656ebd-0edb-4769-83f2-27ad450e322a" (UID: "6e656ebd-0edb-4769-83f2-27ad450e322a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:42:51.878722 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:51.878696 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e656ebd-0edb-4769-83f2-27ad450e322a-istio-data" (OuterVolumeSpecName: "istio-data") pod "6e656ebd-0edb-4769-83f2-27ad450e322a" (UID: "6e656ebd-0edb-4769-83f2-27ad450e322a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:42:51.879738 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:51.879708 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e656ebd-0edb-4769-83f2-27ad450e322a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6e656ebd-0edb-4769-83f2-27ad450e322a" (UID: "6e656ebd-0edb-4769-83f2-27ad450e322a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:42:51.879842 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:51.879764 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e656ebd-0edb-4769-83f2-27ad450e322a-kube-api-access-v6vnl" (OuterVolumeSpecName: "kube-api-access-v6vnl") pod "6e656ebd-0edb-4769-83f2-27ad450e322a" (UID: "6e656ebd-0edb-4769-83f2-27ad450e322a"). InnerVolumeSpecName "kube-api-access-v6vnl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:42:51.879842 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:51.879817 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e656ebd-0edb-4769-83f2-27ad450e322a-istio-token" (OuterVolumeSpecName: "istio-token") pod "6e656ebd-0edb-4769-83f2-27ad450e322a" (UID: "6e656ebd-0edb-4769-83f2-27ad450e322a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:42:51.880223 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:51.880203 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6e656ebd-0edb-4769-83f2-27ad450e322a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6e656ebd-0edb-4769-83f2-27ad450e322a" (UID: "6e656ebd-0edb-4769-83f2-27ad450e322a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 19:42:51.978857 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:51.978819 2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e656ebd-0edb-4769-83f2-27ad450e322a-istio-token\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:42:51.978857 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:51.978852 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v6vnl\" (UniqueName: \"kubernetes.io/projected/6e656ebd-0edb-4769-83f2-27ad450e322a-kube-api-access-v6vnl\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:42:51.978857 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:51.978862 2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6e656ebd-0edb-4769-83f2-27ad450e322a-istio-envoy\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:42:51.979140 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:51.978872 2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6e656ebd-0edb-4769-83f2-27ad450e322a-istio-podinfo\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:42:51.979140 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:51.978880 2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6e656ebd-0edb-4769-83f2-27ad450e322a-istio-data\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:42:51.979140 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:51.978889 2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e656ebd-0edb-4769-83f2-27ad450e322a-credential-socket\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:42:51.979140 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:51.978896 2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e656ebd-0edb-4769-83f2-27ad450e322a-workload-certs\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:42:52.159379 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:52.159273 2566 generic.go:356] "Generic (PLEG): container finished" podID="6e656ebd-0edb-4769-83f2-27ad450e322a" containerID="da3c1508dfdba9cd6af92a4455f2f7d87156e8c8a25dcdc6adc72a9ff2abdaad" exitCode=0 Jul 06 19:42:52.159379 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:52.159334 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-3fd8ac-openshift-default-7b9dcc85-6csm2" Jul 06 19:42:52.159379 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:52.159339 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-3fd8ac-openshift-default-7b9dcc85-6csm2" event={"ID":"6e656ebd-0edb-4769-83f2-27ad450e322a","Type":"ContainerDied","Data":"da3c1508dfdba9cd6af92a4455f2f7d87156e8c8a25dcdc6adc72a9ff2abdaad"} Jul 06 19:42:52.159659 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:52.159399 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-3fd8ac-openshift-default-7b9dcc85-6csm2" event={"ID":"6e656ebd-0edb-4769-83f2-27ad450e322a","Type":"ContainerDied","Data":"76a0bbbb4a6d817d9d35863f7ad48d7c3bb5d874e18d71a87533a3db5c369602"} Jul 06 19:42:52.159659 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:52.159427 2566 scope.go:117] "RemoveContainer" containerID="da3c1508dfdba9cd6af92a4455f2f7d87156e8c8a25dcdc6adc72a9ff2abdaad" Jul 06 19:42:52.172841 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:52.172821 2566 scope.go:117] "RemoveContainer" containerID="a23648c50649cea06798a5ad2c1f21999bc47534040f8540e9ba0054db96e4fb" Jul 06 19:42:52.182967 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:52.182934 2566 scope.go:117] "RemoveContainer" containerID="da3c1508dfdba9cd6af92a4455f2f7d87156e8c8a25dcdc6adc72a9ff2abdaad" Jul 06 19:42:52.183325 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:42:52.183295 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"da3c1508dfdba9cd6af92a4455f2f7d87156e8c8a25dcdc6adc72a9ff2abdaad\": container with ID starting with da3c1508dfdba9cd6af92a4455f2f7d87156e8c8a25dcdc6adc72a9ff2abdaad not found: ID does not exist" containerID="da3c1508dfdba9cd6af92a4455f2f7d87156e8c8a25dcdc6adc72a9ff2abdaad" Jul 06 19:42:52.183440 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:52.183336 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da3c1508dfdba9cd6af92a4455f2f7d87156e8c8a25dcdc6adc72a9ff2abdaad"} err="failed to get container status \"da3c1508dfdba9cd6af92a4455f2f7d87156e8c8a25dcdc6adc72a9ff2abdaad\": rpc error: code = NotFound desc = could not find container \"da3c1508dfdba9cd6af92a4455f2f7d87156e8c8a25dcdc6adc72a9ff2abdaad\": container with ID starting with da3c1508dfdba9cd6af92a4455f2f7d87156e8c8a25dcdc6adc72a9ff2abdaad not found: ID does not exist" Jul 06 19:42:52.183440 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:52.183361 2566 scope.go:117] "RemoveContainer" containerID="a23648c50649cea06798a5ad2c1f21999bc47534040f8540e9ba0054db96e4fb" Jul 06 19:42:52.183663 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:42:52.183639 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a23648c50649cea06798a5ad2c1f21999bc47534040f8540e9ba0054db96e4fb\": container with ID starting with a23648c50649cea06798a5ad2c1f21999bc47534040f8540e9ba0054db96e4fb not found: ID does not exist" containerID="a23648c50649cea06798a5ad2c1f21999bc47534040f8540e9ba0054db96e4fb" Jul 06 19:42:52.183762 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:52.183668 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a23648c50649cea06798a5ad2c1f21999bc47534040f8540e9ba0054db96e4fb"} err="failed to get container status \"a23648c50649cea06798a5ad2c1f21999bc47534040f8540e9ba0054db96e4fb\": rpc error: code = NotFound desc = could not find container \"a23648c50649cea06798a5ad2c1f21999bc47534040f8540e9ba0054db96e4fb\": container with ID starting with a23648c50649cea06798a5ad2c1f21999bc47534040f8540e9ba0054db96e4fb not found: ID does not exist" Jul 06 19:42:52.186330 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:52.186309 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-3fd8ac-openshift-default-7b9dcc85-6csm2"] Jul 06 19:42:52.190540 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:52.190520 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-3fd8ac-openshift-default-7b9dcc85-6csm2"] Jul 06 19:42:53.237030 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:53.236998 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6e656ebd-0edb-4769-83f2-27ad450e322a" path="/var/lib/kubelet/pods/6e656ebd-0edb-4769-83f2-27ad450e322a/volumes" Jul 06 19:42:53.438947 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:53.438912 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:42:53.538800 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:53.538769 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:42:53.703641 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:53.703611 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-99607e-openshift-default-75f6fddfcc-5q98x"] Jul 06 19:42:53.703837 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:53.703817 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-99607e-openshift-default-75f6fddfcc-5q98x" podUID="239e7a39-7c76-4f42-8d5d-3d7140be0550" containerName="istio-proxy" containerID="cri-o://d021a9cf9d30682203f171d37fc429ea134972305a11f8e152e4a6ef1db59de2" gracePeriod=30 Jul 06 19:42:53.727498 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:53.727466 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:42:53.914841 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:53.914808 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:42:54.252548 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:54.252454 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:42:54.533640 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:54.533534 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-26761c-openshift-default-7695b94987-wxtmc"] Jul 06 19:42:54.534010 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:54.533991 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e656ebd-0edb-4769-83f2-27ad450e322a" containerName="istio-proxy" Jul 06 19:42:54.534010 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:54.534009 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e656ebd-0edb-4769-83f2-27ad450e322a" containerName="istio-proxy" Jul 06 19:42:54.534197 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:54.534025 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e656ebd-0edb-4769-83f2-27ad450e322a" containerName="istio-proxy" Jul 06 19:42:54.534197 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:54.534031 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e656ebd-0edb-4769-83f2-27ad450e322a" containerName="istio-proxy" Jul 06 19:42:54.534197 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:54.534040 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="be3f449b-0641-429e-a91c-11c777f5a995" containerName="istio-proxy" Jul 06 19:42:54.534197 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:54.534046 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="be3f449b-0641-429e-a91c-11c777f5a995" containerName="istio-proxy" Jul 06 19:42:54.534197 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:54.534139 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="be3f449b-0641-429e-a91c-11c777f5a995" containerName="istio-proxy" Jul 06 19:42:54.534197 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:54.534154 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="6e656ebd-0edb-4769-83f2-27ad450e322a" containerName="istio-proxy" Jul 06 19:42:54.534197 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:54.534163 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="6e656ebd-0edb-4769-83f2-27ad450e322a" containerName="istio-proxy" Jul 06 19:42:54.537442 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:54.537418 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-26761c-openshift-default-7695b94987-wxtmc" Jul 06 19:42:54.539768 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:54.539750 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-26761c-openshift-default-dockercfg-bfrsh\"" Jul 06 19:42:54.550518 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:54.550493 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-26761c-openshift-default-7695b94987-wxtmc"] Jul 06 19:42:54.569615 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:54.569585 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:42:54.600760 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:54.600722 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d3091ded-001f-4349-bb50-6adec8921337-istio-data\") pod \"e2e-shared-a-26761c-openshift-default-7695b94987-wxtmc\" (UID: \"d3091ded-001f-4349-bb50-6adec8921337\") " pod="openshift-ingress/e2e-shared-a-26761c-openshift-default-7695b94987-wxtmc" Jul 06 19:42:54.600919 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:54.600761 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3091ded-001f-4349-bb50-6adec8921337-credential-socket\") pod \"e2e-shared-a-26761c-openshift-default-7695b94987-wxtmc\" (UID: \"d3091ded-001f-4349-bb50-6adec8921337\") " pod="openshift-ingress/e2e-shared-a-26761c-openshift-default-7695b94987-wxtmc" Jul 06 19:42:54.600919 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:54.600800 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d3091ded-001f-4349-bb50-6adec8921337-workload-certs\") pod \"e2e-shared-a-26761c-openshift-default-7695b94987-wxtmc\" (UID: \"d3091ded-001f-4349-bb50-6adec8921337\") " pod="openshift-ingress/e2e-shared-a-26761c-openshift-default-7695b94987-wxtmc" Jul 06 19:42:54.600919 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:54.600846 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d3091ded-001f-4349-bb50-6adec8921337-istio-podinfo\") pod \"e2e-shared-a-26761c-openshift-default-7695b94987-wxtmc\" (UID: \"d3091ded-001f-4349-bb50-6adec8921337\") " pod="openshift-ingress/e2e-shared-a-26761c-openshift-default-7695b94987-wxtmc" Jul 06 19:42:54.600919 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:54.600892 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d3091ded-001f-4349-bb50-6adec8921337-istiod-ca-cert\") pod \"e2e-shared-a-26761c-openshift-default-7695b94987-wxtmc\" (UID: \"d3091ded-001f-4349-bb50-6adec8921337\") " pod="openshift-ingress/e2e-shared-a-26761c-openshift-default-7695b94987-wxtmc" Jul 06 19:42:54.600919 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:54.600910 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3091ded-001f-4349-bb50-6adec8921337-workload-socket\") pod \"e2e-shared-a-26761c-openshift-default-7695b94987-wxtmc\" (UID: \"d3091ded-001f-4349-bb50-6adec8921337\") " pod="openshift-ingress/e2e-shared-a-26761c-openshift-default-7695b94987-wxtmc" Jul 06 19:42:54.601103 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:54.600963 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d3091ded-001f-4349-bb50-6adec8921337-istio-envoy\") pod \"e2e-shared-a-26761c-openshift-default-7695b94987-wxtmc\" (UID: \"d3091ded-001f-4349-bb50-6adec8921337\") " pod="openshift-ingress/e2e-shared-a-26761c-openshift-default-7695b94987-wxtmc" Jul 06 19:42:54.601103 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:54.601006 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d3091ded-001f-4349-bb50-6adec8921337-istio-token\") pod \"e2e-shared-a-26761c-openshift-default-7695b94987-wxtmc\" (UID: \"d3091ded-001f-4349-bb50-6adec8921337\") " pod="openshift-ingress/e2e-shared-a-26761c-openshift-default-7695b94987-wxtmc" Jul 06 19:42:54.601103 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:54.601026 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hlb9r\" (UniqueName: \"kubernetes.io/projected/d3091ded-001f-4349-bb50-6adec8921337-kube-api-access-hlb9r\") pod \"e2e-shared-a-26761c-openshift-default-7695b94987-wxtmc\" (UID: \"d3091ded-001f-4349-bb50-6adec8921337\") " pod="openshift-ingress/e2e-shared-a-26761c-openshift-default-7695b94987-wxtmc" Jul 06 19:42:54.702244 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:54.702208 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d3091ded-001f-4349-bb50-6adec8921337-istiod-ca-cert\") pod \"e2e-shared-a-26761c-openshift-default-7695b94987-wxtmc\" (UID: \"d3091ded-001f-4349-bb50-6adec8921337\") " pod="openshift-ingress/e2e-shared-a-26761c-openshift-default-7695b94987-wxtmc" Jul 06 19:42:54.702244 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:54.702248 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3091ded-001f-4349-bb50-6adec8921337-workload-socket\") pod \"e2e-shared-a-26761c-openshift-default-7695b94987-wxtmc\" (UID: \"d3091ded-001f-4349-bb50-6adec8921337\") " pod="openshift-ingress/e2e-shared-a-26761c-openshift-default-7695b94987-wxtmc" Jul 06 19:42:54.702535 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:54.702311 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d3091ded-001f-4349-bb50-6adec8921337-istio-envoy\") pod \"e2e-shared-a-26761c-openshift-default-7695b94987-wxtmc\" (UID: \"d3091ded-001f-4349-bb50-6adec8921337\") " pod="openshift-ingress/e2e-shared-a-26761c-openshift-default-7695b94987-wxtmc" Jul 06 19:42:54.702535 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:54.702349 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d3091ded-001f-4349-bb50-6adec8921337-istio-token\") pod \"e2e-shared-a-26761c-openshift-default-7695b94987-wxtmc\" (UID: \"d3091ded-001f-4349-bb50-6adec8921337\") " pod="openshift-ingress/e2e-shared-a-26761c-openshift-default-7695b94987-wxtmc" Jul 06 19:42:54.702535 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:54.702374 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hlb9r\" (UniqueName: \"kubernetes.io/projected/d3091ded-001f-4349-bb50-6adec8921337-kube-api-access-hlb9r\") pod \"e2e-shared-a-26761c-openshift-default-7695b94987-wxtmc\" (UID: \"d3091ded-001f-4349-bb50-6adec8921337\") " pod="openshift-ingress/e2e-shared-a-26761c-openshift-default-7695b94987-wxtmc" Jul 06 19:42:54.702535 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:54.702401 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d3091ded-001f-4349-bb50-6adec8921337-istio-data\") pod \"e2e-shared-a-26761c-openshift-default-7695b94987-wxtmc\" (UID: \"d3091ded-001f-4349-bb50-6adec8921337\") " pod="openshift-ingress/e2e-shared-a-26761c-openshift-default-7695b94987-wxtmc" Jul 06 19:42:54.702535 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:54.702442 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3091ded-001f-4349-bb50-6adec8921337-credential-socket\") pod \"e2e-shared-a-26761c-openshift-default-7695b94987-wxtmc\" (UID: \"d3091ded-001f-4349-bb50-6adec8921337\") " pod="openshift-ingress/e2e-shared-a-26761c-openshift-default-7695b94987-wxtmc" Jul 06 19:42:54.702535 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:54.702466 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d3091ded-001f-4349-bb50-6adec8921337-workload-certs\") pod \"e2e-shared-a-26761c-openshift-default-7695b94987-wxtmc\" (UID: \"d3091ded-001f-4349-bb50-6adec8921337\") " pod="openshift-ingress/e2e-shared-a-26761c-openshift-default-7695b94987-wxtmc" Jul 06 19:42:54.702535 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:54.702501 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d3091ded-001f-4349-bb50-6adec8921337-istio-podinfo\") pod \"e2e-shared-a-26761c-openshift-default-7695b94987-wxtmc\" (UID: \"d3091ded-001f-4349-bb50-6adec8921337\") " pod="openshift-ingress/e2e-shared-a-26761c-openshift-default-7695b94987-wxtmc" Jul 06 19:42:54.702960 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:54.702930 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d3091ded-001f-4349-bb50-6adec8921337-istio-data\") pod \"e2e-shared-a-26761c-openshift-default-7695b94987-wxtmc\" (UID: \"d3091ded-001f-4349-bb50-6adec8921337\") " pod="openshift-ingress/e2e-shared-a-26761c-openshift-default-7695b94987-wxtmc" Jul 06 19:42:54.703030 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:54.702954 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3091ded-001f-4349-bb50-6adec8921337-workload-socket\") pod \"e2e-shared-a-26761c-openshift-default-7695b94987-wxtmc\" (UID: \"d3091ded-001f-4349-bb50-6adec8921337\") " pod="openshift-ingress/e2e-shared-a-26761c-openshift-default-7695b94987-wxtmc" Jul 06 19:42:54.703030 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:54.703010 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3091ded-001f-4349-bb50-6adec8921337-credential-socket\") pod \"e2e-shared-a-26761c-openshift-default-7695b94987-wxtmc\" (UID: \"d3091ded-001f-4349-bb50-6adec8921337\") " pod="openshift-ingress/e2e-shared-a-26761c-openshift-default-7695b94987-wxtmc" Jul 06 19:42:54.703166 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:54.703060 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d3091ded-001f-4349-bb50-6adec8921337-istiod-ca-cert\") pod \"e2e-shared-a-26761c-openshift-default-7695b94987-wxtmc\" (UID: \"d3091ded-001f-4349-bb50-6adec8921337\") " pod="openshift-ingress/e2e-shared-a-26761c-openshift-default-7695b94987-wxtmc" Jul 06 19:42:54.703229 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:54.703183 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d3091ded-001f-4349-bb50-6adec8921337-workload-certs\") pod \"e2e-shared-a-26761c-openshift-default-7695b94987-wxtmc\" (UID: \"d3091ded-001f-4349-bb50-6adec8921337\") " pod="openshift-ingress/e2e-shared-a-26761c-openshift-default-7695b94987-wxtmc" Jul 06 19:42:54.705402 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:54.705383 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d3091ded-001f-4349-bb50-6adec8921337-istio-envoy\") pod \"e2e-shared-a-26761c-openshift-default-7695b94987-wxtmc\" (UID: \"d3091ded-001f-4349-bb50-6adec8921337\") " pod="openshift-ingress/e2e-shared-a-26761c-openshift-default-7695b94987-wxtmc" Jul 06 19:42:54.705597 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:54.705578 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d3091ded-001f-4349-bb50-6adec8921337-istio-podinfo\") pod \"e2e-shared-a-26761c-openshift-default-7695b94987-wxtmc\" (UID: \"d3091ded-001f-4349-bb50-6adec8921337\") " pod="openshift-ingress/e2e-shared-a-26761c-openshift-default-7695b94987-wxtmc" Jul 06 19:42:54.714851 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:54.714827 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hlb9r\" (UniqueName: \"kubernetes.io/projected/d3091ded-001f-4349-bb50-6adec8921337-kube-api-access-hlb9r\") pod \"e2e-shared-a-26761c-openshift-default-7695b94987-wxtmc\" (UID: \"d3091ded-001f-4349-bb50-6adec8921337\") " pod="openshift-ingress/e2e-shared-a-26761c-openshift-default-7695b94987-wxtmc" Jul 06 19:42:54.714968 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:54.714862 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d3091ded-001f-4349-bb50-6adec8921337-istio-token\") pod \"e2e-shared-a-26761c-openshift-default-7695b94987-wxtmc\" (UID: \"d3091ded-001f-4349-bb50-6adec8921337\") " pod="openshift-ingress/e2e-shared-a-26761c-openshift-default-7695b94987-wxtmc" Jul 06 19:42:54.787447 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:54.787411 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:42:54.847826 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:54.847794 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-26761c-openshift-default-7695b94987-wxtmc" Jul 06 19:42:54.992043 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:54.992018 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-26761c-openshift-default-7695b94987-wxtmc"] Jul 06 19:42:54.994103 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:42:54.994053 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3091ded_001f_4349_bb50_6adec8921337.slice/crio-c226ca63ff98cc4c3ad8c6ee9a28a7f87070f70d057f2097cc4ff54e9d56def5 WatchSource:0}: Error finding container c226ca63ff98cc4c3ad8c6ee9a28a7f87070f70d057f2097cc4ff54e9d56def5: Status 404 returned error can't find the container with id c226ca63ff98cc4c3ad8c6ee9a28a7f87070f70d057f2097cc4ff54e9d56def5 Jul 06 19:42:54.996353 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:54.996323 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 19:42:55.000799 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:54.996814 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 19:42:55.000799 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:54.996942 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 19:42:55.180049 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:55.180005 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-26761c-openshift-default-7695b94987-wxtmc" event={"ID":"d3091ded-001f-4349-bb50-6adec8921337","Type":"ContainerStarted","Data":"da8f3bf9ac99b8157b40ade53036649f8f9792b09328c375149e23000f5d6eec"} Jul 06 19:42:55.180230 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:55.180128 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-26761c-openshift-default-7695b94987-wxtmc" event={"ID":"d3091ded-001f-4349-bb50-6adec8921337","Type":"ContainerStarted","Data":"c226ca63ff98cc4c3ad8c6ee9a28a7f87070f70d057f2097cc4ff54e9d56def5"} Jul 06 19:42:55.210420 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:55.210368 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-26761c-openshift-default-7695b94987-wxtmc" podStartSLOduration=1.210352684 podStartE2EDuration="1.210352684s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:42:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:42:55.207955527 +0000 UTC m=+2000.515314476" watchObservedRunningTime="2026-07-06 19:42:55.210352684 +0000 UTC m=+2000.517711631" Jul 06 19:42:55.849039 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:55.849002 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-26761c-openshift-default-7695b94987-wxtmc" Jul 06 19:42:56.853754 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:56.853723 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-26761c-openshift-default-7695b94987-wxtmc" Jul 06 19:42:57.143439 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:57.143350 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:42:57.191476 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:57.191445 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-26761c-openshift-default-7695b94987-wxtmc" Jul 06 19:42:57.192519 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:57.192498 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-26761c-openshift-default-7695b94987-wxtmc" Jul 06 19:42:57.247230 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:57.247185 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:42:57.345077 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:57.345040 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:42:57.542670 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:57.542638 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:42:57.861276 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:57.861204 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:42:57.961247 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:57.961212 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:42:58.961511 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:58.961481 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-99607e-openshift-default-75f6fddfcc-5q98x" Jul 06 19:42:59.045253 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:59.045225 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/239e7a39-7c76-4f42-8d5d-3d7140be0550-istio-podinfo\") pod \"239e7a39-7c76-4f42-8d5d-3d7140be0550\" (UID: \"239e7a39-7c76-4f42-8d5d-3d7140be0550\") " Jul 06 19:42:59.045454 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:59.045281 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/239e7a39-7c76-4f42-8d5d-3d7140be0550-workload-socket\") pod \"239e7a39-7c76-4f42-8d5d-3d7140be0550\" (UID: \"239e7a39-7c76-4f42-8d5d-3d7140be0550\") " Jul 06 19:42:59.045454 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:59.045301 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/239e7a39-7c76-4f42-8d5d-3d7140be0550-istio-envoy\") pod \"239e7a39-7c76-4f42-8d5d-3d7140be0550\" (UID: \"239e7a39-7c76-4f42-8d5d-3d7140be0550\") " Jul 06 19:42:59.045454 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:59.045326 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/239e7a39-7c76-4f42-8d5d-3d7140be0550-istio-token\") pod \"239e7a39-7c76-4f42-8d5d-3d7140be0550\" (UID: \"239e7a39-7c76-4f42-8d5d-3d7140be0550\") " Jul 06 19:42:59.045454 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:59.045360 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/239e7a39-7c76-4f42-8d5d-3d7140be0550-workload-certs\") pod \"239e7a39-7c76-4f42-8d5d-3d7140be0550\" (UID: \"239e7a39-7c76-4f42-8d5d-3d7140be0550\") " Jul 06 19:42:59.045454 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:59.045430 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/239e7a39-7c76-4f42-8d5d-3d7140be0550-istio-data\") pod \"239e7a39-7c76-4f42-8d5d-3d7140be0550\" (UID: \"239e7a39-7c76-4f42-8d5d-3d7140be0550\") " Jul 06 19:42:59.045714 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:59.045475 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tm7xf\" (UniqueName: \"kubernetes.io/projected/239e7a39-7c76-4f42-8d5d-3d7140be0550-kube-api-access-tm7xf\") pod \"239e7a39-7c76-4f42-8d5d-3d7140be0550\" (UID: \"239e7a39-7c76-4f42-8d5d-3d7140be0550\") " Jul 06 19:42:59.045714 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:59.045503 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/239e7a39-7c76-4f42-8d5d-3d7140be0550-istiod-ca-cert\") pod \"239e7a39-7c76-4f42-8d5d-3d7140be0550\" (UID: \"239e7a39-7c76-4f42-8d5d-3d7140be0550\") " Jul 06 19:42:59.045714 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:59.045526 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/239e7a39-7c76-4f42-8d5d-3d7140be0550-credential-socket\") pod \"239e7a39-7c76-4f42-8d5d-3d7140be0550\" (UID: \"239e7a39-7c76-4f42-8d5d-3d7140be0550\") " Jul 06 19:42:59.045714 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:59.045547 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/239e7a39-7c76-4f42-8d5d-3d7140be0550-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "239e7a39-7c76-4f42-8d5d-3d7140be0550" (UID: "239e7a39-7c76-4f42-8d5d-3d7140be0550"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:42:59.045914 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:59.045863 2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/239e7a39-7c76-4f42-8d5d-3d7140be0550-workload-socket\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:42:59.045914 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:59.045895 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/239e7a39-7c76-4f42-8d5d-3d7140be0550-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "239e7a39-7c76-4f42-8d5d-3d7140be0550" (UID: "239e7a39-7c76-4f42-8d5d-3d7140be0550"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:42:59.046280 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:59.046235 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/239e7a39-7c76-4f42-8d5d-3d7140be0550-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "239e7a39-7c76-4f42-8d5d-3d7140be0550" (UID: "239e7a39-7c76-4f42-8d5d-3d7140be0550"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:42:59.046448 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:59.046425 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/239e7a39-7c76-4f42-8d5d-3d7140be0550-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "239e7a39-7c76-4f42-8d5d-3d7140be0550" (UID: "239e7a39-7c76-4f42-8d5d-3d7140be0550"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:42:59.046706 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:59.046684 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/239e7a39-7c76-4f42-8d5d-3d7140be0550-istio-data" (OuterVolumeSpecName: "istio-data") pod "239e7a39-7c76-4f42-8d5d-3d7140be0550" (UID: "239e7a39-7c76-4f42-8d5d-3d7140be0550"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:42:59.047739 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:59.047718 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/239e7a39-7c76-4f42-8d5d-3d7140be0550-kube-api-access-tm7xf" (OuterVolumeSpecName: "kube-api-access-tm7xf") pod "239e7a39-7c76-4f42-8d5d-3d7140be0550" (UID: "239e7a39-7c76-4f42-8d5d-3d7140be0550"). InnerVolumeSpecName "kube-api-access-tm7xf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:42:59.048022 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:59.047996 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/239e7a39-7c76-4f42-8d5d-3d7140be0550-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "239e7a39-7c76-4f42-8d5d-3d7140be0550" (UID: "239e7a39-7c76-4f42-8d5d-3d7140be0550"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:42:59.048022 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:59.048004 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/239e7a39-7c76-4f42-8d5d-3d7140be0550-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "239e7a39-7c76-4f42-8d5d-3d7140be0550" (UID: "239e7a39-7c76-4f42-8d5d-3d7140be0550"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 19:42:59.048178 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:59.048061 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/239e7a39-7c76-4f42-8d5d-3d7140be0550-istio-token" (OuterVolumeSpecName: "istio-token") pod "239e7a39-7c76-4f42-8d5d-3d7140be0550" (UID: "239e7a39-7c76-4f42-8d5d-3d7140be0550"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:42:59.147350 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:59.147313 2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/239e7a39-7c76-4f42-8d5d-3d7140be0550-istiod-ca-cert\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:42:59.147350 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:59.147345 2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/239e7a39-7c76-4f42-8d5d-3d7140be0550-credential-socket\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:42:59.147350 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:59.147355 2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/239e7a39-7c76-4f42-8d5d-3d7140be0550-istio-podinfo\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:42:59.147578 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:59.147363 2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/239e7a39-7c76-4f42-8d5d-3d7140be0550-istio-envoy\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:42:59.147578 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:59.147375 2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/239e7a39-7c76-4f42-8d5d-3d7140be0550-istio-token\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:42:59.147578 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:59.147383 2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/239e7a39-7c76-4f42-8d5d-3d7140be0550-workload-certs\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:42:59.147578 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:59.147390 2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/239e7a39-7c76-4f42-8d5d-3d7140be0550-istio-data\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:42:59.147578 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:59.147400 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tm7xf\" (UniqueName: \"kubernetes.io/projected/239e7a39-7c76-4f42-8d5d-3d7140be0550-kube-api-access-tm7xf\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:42:59.203749 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:59.203706 2566 generic.go:356] "Generic (PLEG): container finished" podID="239e7a39-7c76-4f42-8d5d-3d7140be0550" containerID="d021a9cf9d30682203f171d37fc429ea134972305a11f8e152e4a6ef1db59de2" exitCode=0 Jul 06 19:42:59.203749 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:59.203752 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-99607e-openshift-default-75f6fddfcc-5q98x" Jul 06 19:42:59.203994 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:59.203796 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-99607e-openshift-default-75f6fddfcc-5q98x" event={"ID":"239e7a39-7c76-4f42-8d5d-3d7140be0550","Type":"ContainerDied","Data":"d021a9cf9d30682203f171d37fc429ea134972305a11f8e152e4a6ef1db59de2"} Jul 06 19:42:59.203994 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:59.203854 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-99607e-openshift-default-75f6fddfcc-5q98x" event={"ID":"239e7a39-7c76-4f42-8d5d-3d7140be0550","Type":"ContainerDied","Data":"bccd96bbf8ecc12a00b5824e5f18aa1469d256dfa04811b6be052bf67027c9f7"} Jul 06 19:42:59.203994 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:59.203881 2566 scope.go:117] "RemoveContainer" containerID="d021a9cf9d30682203f171d37fc429ea134972305a11f8e152e4a6ef1db59de2" Jul 06 19:42:59.215726 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:59.215703 2566 scope.go:117] "RemoveContainer" containerID="d2fccf0c5ebee0d801d5e22ddd2db76d82a55db3d33bb14a2a967c16111c3316" Jul 06 19:42:59.224623 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:59.224476 2566 scope.go:117] "RemoveContainer" containerID="d021a9cf9d30682203f171d37fc429ea134972305a11f8e152e4a6ef1db59de2" Jul 06 19:42:59.224873 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:42:59.224840 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d021a9cf9d30682203f171d37fc429ea134972305a11f8e152e4a6ef1db59de2\": container with ID starting with d021a9cf9d30682203f171d37fc429ea134972305a11f8e152e4a6ef1db59de2 not found: ID does not exist" containerID="d021a9cf9d30682203f171d37fc429ea134972305a11f8e152e4a6ef1db59de2" Jul 06 19:42:59.224999 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:59.224880 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d021a9cf9d30682203f171d37fc429ea134972305a11f8e152e4a6ef1db59de2"} err="failed to get container status \"d021a9cf9d30682203f171d37fc429ea134972305a11f8e152e4a6ef1db59de2\": rpc error: code = NotFound desc = could not find container \"d021a9cf9d30682203f171d37fc429ea134972305a11f8e152e4a6ef1db59de2\": container with ID starting with d021a9cf9d30682203f171d37fc429ea134972305a11f8e152e4a6ef1db59de2 not found: ID does not exist" Jul 06 19:42:59.224999 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:59.224905 2566 scope.go:117] "RemoveContainer" containerID="d2fccf0c5ebee0d801d5e22ddd2db76d82a55db3d33bb14a2a967c16111c3316" Jul 06 19:42:59.225299 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:42:59.225277 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d2fccf0c5ebee0d801d5e22ddd2db76d82a55db3d33bb14a2a967c16111c3316\": container with ID starting with d2fccf0c5ebee0d801d5e22ddd2db76d82a55db3d33bb14a2a967c16111c3316 not found: ID does not exist" containerID="d2fccf0c5ebee0d801d5e22ddd2db76d82a55db3d33bb14a2a967c16111c3316" Jul 06 19:42:59.225364 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:59.225308 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d2fccf0c5ebee0d801d5e22ddd2db76d82a55db3d33bb14a2a967c16111c3316"} err="failed to get container status \"d2fccf0c5ebee0d801d5e22ddd2db76d82a55db3d33bb14a2a967c16111c3316\": rpc error: code = NotFound desc = could not find container \"d2fccf0c5ebee0d801d5e22ddd2db76d82a55db3d33bb14a2a967c16111c3316\": container with ID starting with d2fccf0c5ebee0d801d5e22ddd2db76d82a55db3d33bb14a2a967c16111c3316 not found: ID does not exist" Jul 06 19:42:59.227434 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:59.227413 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-99607e-openshift-default-75f6fddfcc-5q98x"] Jul 06 19:42:59.238009 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:42:59.237979 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-99607e-openshift-default-75f6fddfcc-5q98x"] Jul 06 19:43:01.237702 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:43:01.237670 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="239e7a39-7c76-4f42-8d5d-3d7140be0550" path="/var/lib/kubelet/pods/239e7a39-7c76-4f42-8d5d-3d7140be0550/volumes" Jul 06 19:43:01.666415 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:43:01.666384 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:43:02.278582 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:43:02.278548 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-bb8c15-openshift-default-6b66779f5-h2lmx"] Jul 06 19:43:02.279030 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:43:02.278948 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="239e7a39-7c76-4f42-8d5d-3d7140be0550" containerName="istio-proxy" Jul 06 19:43:02.279030 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:43:02.278962 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="239e7a39-7c76-4f42-8d5d-3d7140be0550" containerName="istio-proxy" Jul 06 19:43:02.279030 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:43:02.278986 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="239e7a39-7c76-4f42-8d5d-3d7140be0550" containerName="istio-proxy" Jul 06 19:43:02.279030 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:43:02.278991 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="239e7a39-7c76-4f42-8d5d-3d7140be0550" containerName="istio-proxy" Jul 06 19:43:02.279262 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:43:02.279049 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="239e7a39-7c76-4f42-8d5d-3d7140be0550" containerName="istio-proxy" Jul 06 19:43:02.283552 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:43:02.283531 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-bb8c15-openshift-default-6b66779f5-h2lmx" Jul 06 19:43:02.285709 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:43:02.285687 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-bb8c15-openshift-default-dockercfg-8wrjc\"" Jul 06 19:43:02.295660 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:43:02.295588 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-bb8c15-openshift-default-6b66779f5-h2lmx"] Jul 06 19:43:02.348242 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:43:02.348204 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:43:02.376562 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:43:02.376507 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6da034d-1bae-46ab-9bd1-3a6fe5809201-credential-socket\") pod \"e2e-shared-b-bb8c15-openshift-default-6b66779f5-h2lmx\" (UID: \"b6da034d-1bae-46ab-9bd1-3a6fe5809201\") " pod="openshift-ingress/e2e-shared-b-bb8c15-openshift-default-6b66779f5-h2lmx" Jul 06 19:43:02.376562 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:43:02.376550 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b6da034d-1bae-46ab-9bd1-3a6fe5809201-istio-data\") pod \"e2e-shared-b-bb8c15-openshift-default-6b66779f5-h2lmx\" (UID: \"b6da034d-1bae-46ab-9bd1-3a6fe5809201\") " pod="openshift-ingress/e2e-shared-b-bb8c15-openshift-default-6b66779f5-h2lmx" Jul 06 19:43:02.376834 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:43:02.376641 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b6da034d-1bae-46ab-9bd1-3a6fe5809201-workload-certs\") pod \"e2e-shared-b-bb8c15-openshift-default-6b66779f5-h2lmx\" (UID: \"b6da034d-1bae-46ab-9bd1-3a6fe5809201\") " pod="openshift-ingress/e2e-shared-b-bb8c15-openshift-default-6b66779f5-h2lmx" Jul 06 19:43:02.376834 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:43:02.376665 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b6da034d-1bae-46ab-9bd1-3a6fe5809201-istiod-ca-cert\") pod \"e2e-shared-b-bb8c15-openshift-default-6b66779f5-h2lmx\" (UID: \"b6da034d-1bae-46ab-9bd1-3a6fe5809201\") " pod="openshift-ingress/e2e-shared-b-bb8c15-openshift-default-6b66779f5-h2lmx" Jul 06 19:43:02.376834 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:43:02.376709 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gq4rs\" (UniqueName: \"kubernetes.io/projected/b6da034d-1bae-46ab-9bd1-3a6fe5809201-kube-api-access-gq4rs\") pod \"e2e-shared-b-bb8c15-openshift-default-6b66779f5-h2lmx\" (UID: \"b6da034d-1bae-46ab-9bd1-3a6fe5809201\") " pod="openshift-ingress/e2e-shared-b-bb8c15-openshift-default-6b66779f5-h2lmx" Jul 06 19:43:02.376834 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:43:02.376750 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b6da034d-1bae-46ab-9bd1-3a6fe5809201-istio-envoy\") pod \"e2e-shared-b-bb8c15-openshift-default-6b66779f5-h2lmx\" (UID: \"b6da034d-1bae-46ab-9bd1-3a6fe5809201\") " pod="openshift-ingress/e2e-shared-b-bb8c15-openshift-default-6b66779f5-h2lmx" Jul 06 19:43:02.376834 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:43:02.376778 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b6da034d-1bae-46ab-9bd1-3a6fe5809201-istio-token\") pod \"e2e-shared-b-bb8c15-openshift-default-6b66779f5-h2lmx\" (UID: \"b6da034d-1bae-46ab-9bd1-3a6fe5809201\") " pod="openshift-ingress/e2e-shared-b-bb8c15-openshift-default-6b66779f5-h2lmx" Jul 06 19:43:02.376834 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:43:02.376814 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b6da034d-1bae-46ab-9bd1-3a6fe5809201-istio-podinfo\") pod \"e2e-shared-b-bb8c15-openshift-default-6b66779f5-h2lmx\" (UID: \"b6da034d-1bae-46ab-9bd1-3a6fe5809201\") " pod="openshift-ingress/e2e-shared-b-bb8c15-openshift-default-6b66779f5-h2lmx" Jul 06 19:43:02.377208 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:43:02.376925 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6da034d-1bae-46ab-9bd1-3a6fe5809201-workload-socket\") pod \"e2e-shared-b-bb8c15-openshift-default-6b66779f5-h2lmx\" (UID: \"b6da034d-1bae-46ab-9bd1-3a6fe5809201\") " pod="openshift-ingress/e2e-shared-b-bb8c15-openshift-default-6b66779f5-h2lmx" Jul 06 19:43:02.448390 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:43:02.448353 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:43:02.477696 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:43:02.477662 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b6da034d-1bae-46ab-9bd1-3a6fe5809201-workload-certs\") pod \"e2e-shared-b-bb8c15-openshift-default-6b66779f5-h2lmx\" (UID: \"b6da034d-1bae-46ab-9bd1-3a6fe5809201\") " pod="openshift-ingress/e2e-shared-b-bb8c15-openshift-default-6b66779f5-h2lmx" Jul 06 19:43:02.477898 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:43:02.477700 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b6da034d-1bae-46ab-9bd1-3a6fe5809201-istiod-ca-cert\") pod \"e2e-shared-b-bb8c15-openshift-default-6b66779f5-h2lmx\" (UID: \"b6da034d-1bae-46ab-9bd1-3a6fe5809201\") " pod="openshift-ingress/e2e-shared-b-bb8c15-openshift-default-6b66779f5-h2lmx" Jul 06 19:43:02.477898 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:43:02.477738 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gq4rs\" (UniqueName: \"kubernetes.io/projected/b6da034d-1bae-46ab-9bd1-3a6fe5809201-kube-api-access-gq4rs\") pod \"e2e-shared-b-bb8c15-openshift-default-6b66779f5-h2lmx\" (UID: \"b6da034d-1bae-46ab-9bd1-3a6fe5809201\") " pod="openshift-ingress/e2e-shared-b-bb8c15-openshift-default-6b66779f5-h2lmx" Jul 06 19:43:02.477898 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:43:02.477773 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b6da034d-1bae-46ab-9bd1-3a6fe5809201-istio-envoy\") pod \"e2e-shared-b-bb8c15-openshift-default-6b66779f5-h2lmx\" (UID: \"b6da034d-1bae-46ab-9bd1-3a6fe5809201\") " pod="openshift-ingress/e2e-shared-b-bb8c15-openshift-default-6b66779f5-h2lmx" Jul 06 19:43:02.477898 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:43:02.477799 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b6da034d-1bae-46ab-9bd1-3a6fe5809201-istio-token\") pod \"e2e-shared-b-bb8c15-openshift-default-6b66779f5-h2lmx\" (UID: \"b6da034d-1bae-46ab-9bd1-3a6fe5809201\") " pod="openshift-ingress/e2e-shared-b-bb8c15-openshift-default-6b66779f5-h2lmx" Jul 06 19:43:02.477898 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:43:02.477833 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b6da034d-1bae-46ab-9bd1-3a6fe5809201-istio-podinfo\") pod \"e2e-shared-b-bb8c15-openshift-default-6b66779f5-h2lmx\" (UID: \"b6da034d-1bae-46ab-9bd1-3a6fe5809201\") " pod="openshift-ingress/e2e-shared-b-bb8c15-openshift-default-6b66779f5-h2lmx" Jul 06 19:43:02.478189 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:43:02.477906 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6da034d-1bae-46ab-9bd1-3a6fe5809201-workload-socket\") pod \"e2e-shared-b-bb8c15-openshift-default-6b66779f5-h2lmx\" (UID: \"b6da034d-1bae-46ab-9bd1-3a6fe5809201\") " pod="openshift-ingress/e2e-shared-b-bb8c15-openshift-default-6b66779f5-h2lmx" Jul 06 19:43:02.478189 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:43:02.477962 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6da034d-1bae-46ab-9bd1-3a6fe5809201-credential-socket\") pod \"e2e-shared-b-bb8c15-openshift-default-6b66779f5-h2lmx\" (UID: \"b6da034d-1bae-46ab-9bd1-3a6fe5809201\") " pod="openshift-ingress/e2e-shared-b-bb8c15-openshift-default-6b66779f5-h2lmx" Jul 06 19:43:02.478189 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:43:02.477986 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b6da034d-1bae-46ab-9bd1-3a6fe5809201-istio-data\") pod \"e2e-shared-b-bb8c15-openshift-default-6b66779f5-h2lmx\" (UID: \"b6da034d-1bae-46ab-9bd1-3a6fe5809201\") " pod="openshift-ingress/e2e-shared-b-bb8c15-openshift-default-6b66779f5-h2lmx" Jul 06 19:43:02.478189 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:43:02.478142 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b6da034d-1bae-46ab-9bd1-3a6fe5809201-workload-certs\") pod \"e2e-shared-b-bb8c15-openshift-default-6b66779f5-h2lmx\" (UID: \"b6da034d-1bae-46ab-9bd1-3a6fe5809201\") " pod="openshift-ingress/e2e-shared-b-bb8c15-openshift-default-6b66779f5-h2lmx" Jul 06 19:43:02.478425 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:43:02.478400 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6da034d-1bae-46ab-9bd1-3a6fe5809201-credential-socket\") pod \"e2e-shared-b-bb8c15-openshift-default-6b66779f5-h2lmx\" (UID: \"b6da034d-1bae-46ab-9bd1-3a6fe5809201\") " pod="openshift-ingress/e2e-shared-b-bb8c15-openshift-default-6b66779f5-h2lmx" Jul 06 19:43:02.478474 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:43:02.478416 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b6da034d-1bae-46ab-9bd1-3a6fe5809201-istio-data\") pod \"e2e-shared-b-bb8c15-openshift-default-6b66779f5-h2lmx\" (UID: \"b6da034d-1bae-46ab-9bd1-3a6fe5809201\") " pod="openshift-ingress/e2e-shared-b-bb8c15-openshift-default-6b66779f5-h2lmx" Jul 06 19:43:02.478474 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:43:02.478440 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6da034d-1bae-46ab-9bd1-3a6fe5809201-workload-socket\") pod \"e2e-shared-b-bb8c15-openshift-default-6b66779f5-h2lmx\" (UID: \"b6da034d-1bae-46ab-9bd1-3a6fe5809201\") " pod="openshift-ingress/e2e-shared-b-bb8c15-openshift-default-6b66779f5-h2lmx" Jul 06 19:43:02.478972 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:43:02.478912 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b6da034d-1bae-46ab-9bd1-3a6fe5809201-istiod-ca-cert\") pod \"e2e-shared-b-bb8c15-openshift-default-6b66779f5-h2lmx\" (UID: \"b6da034d-1bae-46ab-9bd1-3a6fe5809201\") " pod="openshift-ingress/e2e-shared-b-bb8c15-openshift-default-6b66779f5-h2lmx" Jul 06 19:43:02.480213 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:43:02.480193 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b6da034d-1bae-46ab-9bd1-3a6fe5809201-istio-envoy\") pod \"e2e-shared-b-bb8c15-openshift-default-6b66779f5-h2lmx\" (UID: \"b6da034d-1bae-46ab-9bd1-3a6fe5809201\") " pod="openshift-ingress/e2e-shared-b-bb8c15-openshift-default-6b66779f5-h2lmx" Jul 06 19:43:02.480686 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:43:02.480659 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b6da034d-1bae-46ab-9bd1-3a6fe5809201-istio-podinfo\") pod \"e2e-shared-b-bb8c15-openshift-default-6b66779f5-h2lmx\" (UID: \"b6da034d-1bae-46ab-9bd1-3a6fe5809201\") " pod="openshift-ingress/e2e-shared-b-bb8c15-openshift-default-6b66779f5-h2lmx" Jul 06 19:43:02.485570 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:43:02.485545 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b6da034d-1bae-46ab-9bd1-3a6fe5809201-istio-token\") pod \"e2e-shared-b-bb8c15-openshift-default-6b66779f5-h2lmx\" (UID: \"b6da034d-1bae-46ab-9bd1-3a6fe5809201\") " pod="openshift-ingress/e2e-shared-b-bb8c15-openshift-default-6b66779f5-h2lmx" Jul 06 19:43:02.485869 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:43:02.485849 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gq4rs\" (UniqueName: \"kubernetes.io/projected/b6da034d-1bae-46ab-9bd1-3a6fe5809201-kube-api-access-gq4rs\") pod \"e2e-shared-b-bb8c15-openshift-default-6b66779f5-h2lmx\" (UID: \"b6da034d-1bae-46ab-9bd1-3a6fe5809201\") " pod="openshift-ingress/e2e-shared-b-bb8c15-openshift-default-6b66779f5-h2lmx" Jul 06 19:43:02.577957 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:43:02.577917 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:43:02.595036 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:43:02.595003 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-bb8c15-openshift-default-6b66779f5-h2lmx" Jul 06 19:43:02.743079 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:43:02.743035 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-bb8c15-openshift-default-6b66779f5-h2lmx"] Jul 06 19:43:02.744766 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:43:02.744740 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6da034d_1bae_46ab_9bd1_3a6fe5809201.slice/crio-48ed4676aa0dc0ce6ee41e0775dac639651347427f17d6648df3561169e44337 WatchSource:0}: Error finding container 48ed4676aa0dc0ce6ee41e0775dac639651347427f17d6648df3561169e44337: Status 404 returned error can't find the container with id 48ed4676aa0dc0ce6ee41e0775dac639651347427f17d6648df3561169e44337 Jul 06 19:43:02.747033 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:43:02.747002 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 19:43:02.747130 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:43:02.747090 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 19:43:02.747130 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:43:02.747126 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 19:43:02.854293 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:43:02.854201 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:43:02.956912 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:43:02.956883 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:43:03.229929 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:43:03.229829 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-bb8c15-openshift-default-6b66779f5-h2lmx" event={"ID":"b6da034d-1bae-46ab-9bd1-3a6fe5809201","Type":"ContainerStarted","Data":"2ffde3564793e675e4bfb6a9004bc0bd7b88bcc468d4e75c64e20bc3e903f5fb"} Jul 06 19:43:03.229929 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:43:03.229879 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-bb8c15-openshift-default-6b66779f5-h2lmx" event={"ID":"b6da034d-1bae-46ab-9bd1-3a6fe5809201","Type":"ContainerStarted","Data":"48ed4676aa0dc0ce6ee41e0775dac639651347427f17d6648df3561169e44337"} Jul 06 19:43:03.256497 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:43:03.256436 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-bb8c15-openshift-default-6b66779f5-h2lmx" podStartSLOduration=1.256419613 podStartE2EDuration="1.256419613s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:43:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:43:03.254804238 +0000 UTC m=+2008.562163186" watchObservedRunningTime="2026-07-06 19:43:03.256419613 +0000 UTC m=+2008.563778560" Jul 06 19:43:03.595556 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:43:03.595517 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-bb8c15-openshift-default-6b66779f5-h2lmx" Jul 06 19:43:03.600757 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:43:03.600733 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-bb8c15-openshift-default-6b66779f5-h2lmx" Jul 06 19:43:04.236442 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:43:04.236408 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-bb8c15-openshift-default-6b66779f5-h2lmx" Jul 06 19:43:04.242649 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:43:04.242613 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-bb8c15-openshift-default-6b66779f5-h2lmx" Jul 06 19:43:05.211807 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:43:05.211748 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:43:05.340910 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:43:05.340877 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:43:05.435569 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:43:05.435534 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:43:06.032809 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:43:06.032771 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:43:06.161249 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:43:06.161212 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:43:06.346662 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:43:06.346626 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:43:09.633056 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:43:09.633003 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:43:11.292248 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:43:11.292211 2566 generic.go:356] "Generic (PLEG): container finished" podID="d3091ded-001f-4349-bb50-6adec8921337" containerID="da8f3bf9ac99b8157b40ade53036649f8f9792b09328c375149e23000f5d6eec" exitCode=0 Jul 06 19:43:11.292694 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:43:11.292280 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-26761c-openshift-default-7695b94987-wxtmc" event={"ID":"d3091ded-001f-4349-bb50-6adec8921337","Type":"ContainerDied","Data":"da8f3bf9ac99b8157b40ade53036649f8f9792b09328c375149e23000f5d6eec"} Jul 06 19:43:11.292737 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:43:11.292709 2566 scope.go:117] "RemoveContainer" containerID="da8f3bf9ac99b8157b40ade53036649f8f9792b09328c375149e23000f5d6eec" Jul 06 19:43:11.293720 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:43:11.293692 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 19:43:11.293817 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:43:11.293758 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 19:43:11.293817 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:43:11.293788 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 19:43:12.191872 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:43:12.191822 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-26761c-openshift-default-7695b94987-wxtmc" Jul 06 19:43:12.303289 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:43:12.303257 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-26761c-openshift-default-7695b94987-wxtmc" event={"ID":"d3091ded-001f-4349-bb50-6adec8921337","Type":"ContainerStarted","Data":"0ec4e9b69eb5bceedb2abe2a73bc937a447c60c55979f054edbdfd7844e9d2c4"} Jul 06 19:43:12.848640 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:43:12.848603 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-26761c-openshift-default-7695b94987-wxtmc" Jul 06 19:43:12.849794 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:43:12.849763 2566 patch_prober.go:28] interesting pod/e2e-shared-a-26761c-openshift-default-7695b94987-wxtmc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body= Jul 06 19:43:12.850022 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:43:12.849994 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-26761c-openshift-default-7695b94987-wxtmc" podUID="d3091ded-001f-4349-bb50-6adec8921337" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" Jul 06 19:43:13.848381 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:43:13.848340 2566 patch_prober.go:28] interesting pod/e2e-shared-a-26761c-openshift-default-7695b94987-wxtmc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body= Jul 06 19:43:13.849411 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:43:13.848975 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-26761c-openshift-default-7695b94987-wxtmc" podUID="d3091ded-001f-4349-bb50-6adec8921337" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" Jul 06 19:43:14.625862 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:43:14.625829 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:43:14.686635 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:43:14.686596 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:43:14.849158 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:43:14.849114 2566 patch_prober.go:28] interesting pod/e2e-shared-a-26761c-openshift-default-7695b94987-wxtmc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body= Jul 06 19:43:14.849660 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:43:14.849180 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-26761c-openshift-default-7695b94987-wxtmc" podUID="d3091ded-001f-4349-bb50-6adec8921337" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" Jul 06 19:43:15.848586 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:43:15.848542 2566 patch_prober.go:28] interesting pod/e2e-shared-a-26761c-openshift-default-7695b94987-wxtmc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" start-of-body= Jul 06 19:43:15.848889 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:43:15.848859 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-26761c-openshift-default-7695b94987-wxtmc" podUID="d3091ded-001f-4349-bb50-6adec8921337" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.64:15021/healthz/ready\": dial tcp 10.132.0.64:15021: connect: connection refused" Jul 06 19:43:16.853473 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:43:16.853442 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-26761c-openshift-default-7695b94987-wxtmc" Jul 06 19:43:16.853958 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:43:16.853782 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-26761c-openshift-default-7695b94987-wxtmc" Jul 06 19:43:16.854584 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:43:16.854565 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-26761c-openshift-default-7695b94987-wxtmc" Jul 06 19:43:18.345217 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:43:18.345184 2566 generic.go:356] "Generic (PLEG): container finished" podID="b6da034d-1bae-46ab-9bd1-3a6fe5809201" containerID="2ffde3564793e675e4bfb6a9004bc0bd7b88bcc468d4e75c64e20bc3e903f5fb" exitCode=0 Jul 06 19:43:18.345713 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:43:18.345259 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-bb8c15-openshift-default-6b66779f5-h2lmx" event={"ID":"b6da034d-1bae-46ab-9bd1-3a6fe5809201","Type":"ContainerDied","Data":"2ffde3564793e675e4bfb6a9004bc0bd7b88bcc468d4e75c64e20bc3e903f5fb"} Jul 06 19:43:18.345853 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:43:18.345834 2566 scope.go:117] "RemoveContainer" containerID="2ffde3564793e675e4bfb6a9004bc0bd7b88bcc468d4e75c64e20bc3e903f5fb" Jul 06 19:43:18.346908 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:43:18.346874 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 19:43:18.347002 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:43:18.346960 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 19:43:18.347046 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:43:18.347006 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 06 19:43:18.795550 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:43:18.795523 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:43:18.899481 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:43:18.899450 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:43:19.240321 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:43:19.239926 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-bb8c15-openshift-default-6b66779f5-h2lmx" Jul 06 19:43:19.358671 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:43:19.358617 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-bb8c15-openshift-default-6b66779f5-h2lmx" event={"ID":"b6da034d-1bae-46ab-9bd1-3a6fe5809201","Type":"ContainerStarted","Data":"aa254f64560b2d773344891570fa660841ca956e2ac98b2e9b3b98297a92609e"} Jul 06 19:43:19.595717 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:43:19.595680 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-bb8c15-openshift-default-6b66779f5-h2lmx" Jul 06 19:43:19.596936 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:43:19.596906 2566 patch_prober.go:28] interesting pod/e2e-shared-b-bb8c15-openshift-default-6b66779f5-h2lmx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body= Jul 06 19:43:19.597087 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:43:19.596965 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-bb8c15-openshift-default-6b66779f5-h2lmx" podUID="b6da034d-1bae-46ab-9bd1-3a6fe5809201" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" Jul 06 19:43:20.596240 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:43:20.596201 2566 patch_prober.go:28] interesting pod/e2e-shared-b-bb8c15-openshift-default-6b66779f5-h2lmx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body= Jul 06 19:43:20.596638 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:43:20.596271 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-bb8c15-openshift-default-6b66779f5-h2lmx" podUID="b6da034d-1bae-46ab-9bd1-3a6fe5809201" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" Jul 06 19:43:21.596031 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:43:21.595994 2566 patch_prober.go:28] interesting pod/e2e-shared-b-bb8c15-openshift-default-6b66779f5-h2lmx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body= Jul 06 19:43:21.596273 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:43:21.596061 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-bb8c15-openshift-default-6b66779f5-h2lmx" podUID="b6da034d-1bae-46ab-9bd1-3a6fe5809201" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" Jul 06 19:43:21.861618 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:43:21.861511 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:43:22.596344 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:43:22.596308 2566 patch_prober.go:28] interesting pod/e2e-shared-b-bb8c15-openshift-default-6b66779f5-h2lmx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body= Jul 06 19:43:22.596749 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:43:22.596377 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-bb8c15-openshift-default-6b66779f5-h2lmx" podUID="b6da034d-1bae-46ab-9bd1-3a6fe5809201" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" Jul 06 19:43:23.599542 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:43:23.599500 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-bb8c15-openshift-default-6b66779f5-h2lmx" Jul 06 19:43:23.599948 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:43:23.599804 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-bb8c15-openshift-default-6b66779f5-h2lmx" Jul 06 19:43:23.600500 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:43:23.600467 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-bb8c15-openshift-default-6b66779f5-h2lmx" Jul 06 19:43:28.107949 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:43:28.107919 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:43:28.206388 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:43:28.206357 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:44:01.510420 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:44:01.510331 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:44:28.503397 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:44:28.503359 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:45:56.939618 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:45:56.939570 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:45:58.281675 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:45:58.281639 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:46:04.801372 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:46:04.801339 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-26761c/sub-test-model-26761c-kserve-7f465bc6c7-8fpmk"] Jul 06 19:46:04.801823 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:46:04.801810 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="239e7a39-7c76-4f42-8d5d-3d7140be0550" containerName="istio-proxy" Jul 06 19:46:04.805035 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:46:04.805015 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-26761c/sub-test-model-26761c-kserve-7f465bc6c7-8fpmk" Jul 06 19:46:04.807754 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:46:04.807735 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-26761c\"/\"default-dockercfg-t4rpq\"" Jul 06 19:46:04.808800 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:46:04.808783 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-26761c\"/\"openshift-service-ca.crt\"" Jul 06 19:46:04.808899 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:46:04.808813 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-26761c\"/\"kube-root-ca.crt\"" Jul 06 19:46:04.808899 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:46:04.808851 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-26761c\"/\"sub-test-model-26761c-kserve-self-signed-certs\"" Jul 06 19:46:04.824041 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:46:04.824012 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-26761c/sub-test-model-26761c-kserve-7f465bc6c7-8fpmk"] Jul 06 19:46:04.842312 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:46:04.842271 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:46:04.950706 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:46:04.950668 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4ld6g\" (UniqueName: \"kubernetes.io/projected/990afe83-30ab-478f-84a8-158a7e55bbeb-kube-api-access-4ld6g\") pod \"sub-test-model-26761c-kserve-7f465bc6c7-8fpmk\" (UID: \"990afe83-30ab-478f-84a8-158a7e55bbeb\") " pod="ai-tenant-e2e-shared-a-26761c/sub-test-model-26761c-kserve-7f465bc6c7-8fpmk" Jul 06 19:46:04.951001 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:46:04.950971 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/990afe83-30ab-478f-84a8-158a7e55bbeb-dshm\") pod \"sub-test-model-26761c-kserve-7f465bc6c7-8fpmk\" (UID: \"990afe83-30ab-478f-84a8-158a7e55bbeb\") " pod="ai-tenant-e2e-shared-a-26761c/sub-test-model-26761c-kserve-7f465bc6c7-8fpmk" Jul 06 19:46:04.951220 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:46:04.951173 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/990afe83-30ab-478f-84a8-158a7e55bbeb-model-cache\") pod \"sub-test-model-26761c-kserve-7f465bc6c7-8fpmk\" (UID: \"990afe83-30ab-478f-84a8-158a7e55bbeb\") " pod="ai-tenant-e2e-shared-a-26761c/sub-test-model-26761c-kserve-7f465bc6c7-8fpmk" Jul 06 19:46:04.951313 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:46:04.951217 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/990afe83-30ab-478f-84a8-158a7e55bbeb-tls-certs\") pod \"sub-test-model-26761c-kserve-7f465bc6c7-8fpmk\" (UID: \"990afe83-30ab-478f-84a8-158a7e55bbeb\") " pod="ai-tenant-e2e-shared-a-26761c/sub-test-model-26761c-kserve-7f465bc6c7-8fpmk" Jul 06 19:46:04.951313 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:46:04.951245 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/990afe83-30ab-478f-84a8-158a7e55bbeb-tmp-dir\") pod \"sub-test-model-26761c-kserve-7f465bc6c7-8fpmk\" (UID: \"990afe83-30ab-478f-84a8-158a7e55bbeb\") " pod="ai-tenant-e2e-shared-a-26761c/sub-test-model-26761c-kserve-7f465bc6c7-8fpmk" Jul 06 19:46:04.951313 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:46:04.951275 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/990afe83-30ab-478f-84a8-158a7e55bbeb-home\") pod \"sub-test-model-26761c-kserve-7f465bc6c7-8fpmk\" (UID: \"990afe83-30ab-478f-84a8-158a7e55bbeb\") " pod="ai-tenant-e2e-shared-a-26761c/sub-test-model-26761c-kserve-7f465bc6c7-8fpmk" Jul 06 19:46:05.052626 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:46:05.052538 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/990afe83-30ab-478f-84a8-158a7e55bbeb-model-cache\") pod \"sub-test-model-26761c-kserve-7f465bc6c7-8fpmk\" (UID: \"990afe83-30ab-478f-84a8-158a7e55bbeb\") " pod="ai-tenant-e2e-shared-a-26761c/sub-test-model-26761c-kserve-7f465bc6c7-8fpmk" Jul 06 19:46:05.052626 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:46:05.052575 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/990afe83-30ab-478f-84a8-158a7e55bbeb-tls-certs\") pod \"sub-test-model-26761c-kserve-7f465bc6c7-8fpmk\" (UID: \"990afe83-30ab-478f-84a8-158a7e55bbeb\") " pod="ai-tenant-e2e-shared-a-26761c/sub-test-model-26761c-kserve-7f465bc6c7-8fpmk" Jul 06 19:46:05.052626 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:46:05.052595 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/990afe83-30ab-478f-84a8-158a7e55bbeb-tmp-dir\") pod \"sub-test-model-26761c-kserve-7f465bc6c7-8fpmk\" (UID: \"990afe83-30ab-478f-84a8-158a7e55bbeb\") " pod="ai-tenant-e2e-shared-a-26761c/sub-test-model-26761c-kserve-7f465bc6c7-8fpmk" Jul 06 19:46:05.052626 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:46:05.052613 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/990afe83-30ab-478f-84a8-158a7e55bbeb-home\") pod \"sub-test-model-26761c-kserve-7f465bc6c7-8fpmk\" (UID: \"990afe83-30ab-478f-84a8-158a7e55bbeb\") " pod="ai-tenant-e2e-shared-a-26761c/sub-test-model-26761c-kserve-7f465bc6c7-8fpmk" Jul 06 19:46:05.052946 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:46:05.052652 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4ld6g\" (UniqueName: \"kubernetes.io/projected/990afe83-30ab-478f-84a8-158a7e55bbeb-kube-api-access-4ld6g\") pod \"sub-test-model-26761c-kserve-7f465bc6c7-8fpmk\" (UID: \"990afe83-30ab-478f-84a8-158a7e55bbeb\") " pod="ai-tenant-e2e-shared-a-26761c/sub-test-model-26761c-kserve-7f465bc6c7-8fpmk" Jul 06 19:46:05.052946 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:46:05.052672 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/990afe83-30ab-478f-84a8-158a7e55bbeb-dshm\") pod \"sub-test-model-26761c-kserve-7f465bc6c7-8fpmk\" (UID: \"990afe83-30ab-478f-84a8-158a7e55bbeb\") " pod="ai-tenant-e2e-shared-a-26761c/sub-test-model-26761c-kserve-7f465bc6c7-8fpmk" Jul 06 19:46:05.053057 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:46:05.052945 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/990afe83-30ab-478f-84a8-158a7e55bbeb-model-cache\") pod \"sub-test-model-26761c-kserve-7f465bc6c7-8fpmk\" (UID: \"990afe83-30ab-478f-84a8-158a7e55bbeb\") " pod="ai-tenant-e2e-shared-a-26761c/sub-test-model-26761c-kserve-7f465bc6c7-8fpmk" Jul 06 19:46:05.053057 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:46:05.053003 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/990afe83-30ab-478f-84a8-158a7e55bbeb-tmp-dir\") pod \"sub-test-model-26761c-kserve-7f465bc6c7-8fpmk\" (UID: \"990afe83-30ab-478f-84a8-158a7e55bbeb\") " pod="ai-tenant-e2e-shared-a-26761c/sub-test-model-26761c-kserve-7f465bc6c7-8fpmk" Jul 06 19:46:05.053192 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:46:05.053112 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/990afe83-30ab-478f-84a8-158a7e55bbeb-home\") pod \"sub-test-model-26761c-kserve-7f465bc6c7-8fpmk\" (UID: \"990afe83-30ab-478f-84a8-158a7e55bbeb\") " pod="ai-tenant-e2e-shared-a-26761c/sub-test-model-26761c-kserve-7f465bc6c7-8fpmk" Jul 06 19:46:05.054908 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:46:05.054892 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/990afe83-30ab-478f-84a8-158a7e55bbeb-dshm\") pod \"sub-test-model-26761c-kserve-7f465bc6c7-8fpmk\" (UID: \"990afe83-30ab-478f-84a8-158a7e55bbeb\") " pod="ai-tenant-e2e-shared-a-26761c/sub-test-model-26761c-kserve-7f465bc6c7-8fpmk" Jul 06 19:46:05.055276 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:46:05.055259 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/990afe83-30ab-478f-84a8-158a7e55bbeb-tls-certs\") pod \"sub-test-model-26761c-kserve-7f465bc6c7-8fpmk\" (UID: \"990afe83-30ab-478f-84a8-158a7e55bbeb\") " pod="ai-tenant-e2e-shared-a-26761c/sub-test-model-26761c-kserve-7f465bc6c7-8fpmk" Jul 06 19:46:05.064882 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:46:05.064860 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4ld6g\" (UniqueName: \"kubernetes.io/projected/990afe83-30ab-478f-84a8-158a7e55bbeb-kube-api-access-4ld6g\") pod \"sub-test-model-26761c-kserve-7f465bc6c7-8fpmk\" (UID: \"990afe83-30ab-478f-84a8-158a7e55bbeb\") " pod="ai-tenant-e2e-shared-a-26761c/sub-test-model-26761c-kserve-7f465bc6c7-8fpmk" Jul 06 19:46:05.114161 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:46:05.114126 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-26761c/sub-test-model-26761c-kserve-7f465bc6c7-8fpmk" Jul 06 19:46:05.261832 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:46:05.261809 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-26761c/sub-test-model-26761c-kserve-7f465bc6c7-8fpmk"] Jul 06 19:46:05.263813 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:46:05.263790 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod990afe83_30ab_478f_84a8_158a7e55bbeb.slice/crio-f34efedff44aca4556c58b76d8a6ba615a1110530561e6bbbb574662eadb9d9a WatchSource:0}: Error finding container f34efedff44aca4556c58b76d8a6ba615a1110530561e6bbbb574662eadb9d9a: Status 404 returned error can't find the container with id f34efedff44aca4556c58b76d8a6ba615a1110530561e6bbbb574662eadb9d9a Jul 06 19:46:05.307717 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:46:05.307693 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-26761c/sub-test-model-26761c-kserve-7f465bc6c7-8fpmk" event={"ID":"990afe83-30ab-478f-84a8-158a7e55bbeb","Type":"ContainerStarted","Data":"f34efedff44aca4556c58b76d8a6ba615a1110530561e6bbbb574662eadb9d9a"} Jul 06 19:46:06.315023 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:46:06.314984 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-26761c/sub-test-model-26761c-kserve-7f465bc6c7-8fpmk" event={"ID":"990afe83-30ab-478f-84a8-158a7e55bbeb","Type":"ContainerStarted","Data":"89cd23fcc20838839572f3be1761858af30c0094498d96bc5f184d830071d6c0"} Jul 06 19:46:06.333686 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:46:06.333634 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-26761c/sub-test-model-26761c-kserve-7f465bc6c7-8fpmk" podStartSLOduration=2.333614178 podStartE2EDuration="2.333614178s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:46:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:46:06.333496302 +0000 UTC m=+2191.640855249" watchObservedRunningTime="2026-07-06 19:46:06.333614178 +0000 UTC m=+2191.640973127" Jul 06 19:46:15.115193 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:46:15.115150 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-26761c/sub-test-model-26761c-kserve-7f465bc6c7-8fpmk" Jul 06 19:46:15.115685 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:46:15.115208 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-26761c/sub-test-model-26761c-kserve-7f465bc6c7-8fpmk" Jul 06 19:46:15.127787 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:46:15.127763 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-26761c/sub-test-model-26761c-kserve-7f465bc6c7-8fpmk" Jul 06 19:46:15.376873 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:46:15.376786 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-26761c/sub-test-model-26761c-kserve-7f465bc6c7-8fpmk" Jul 06 19:46:38.029750 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:46:38.029709 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:47:20.345391 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:47:20.345309 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-26761c/sub-test-model-26761c-kserve-7f465bc6c7-8fpmk"] Jul 06 19:47:20.365223 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:47:20.365189 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:47:21.209340 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:47:21.209297 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:47:34.521521 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:47:34.521487 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:47:35.365826 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:47:35.365780 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-26761c/sub-test-model-26761c-kserve-7f465bc6c7-8fpmk" podUID="990afe83-30ab-478f-84a8-158a7e55bbeb" containerName="main" containerID="cri-o://89cd23fcc20838839572f3be1761858af30c0094498d96bc5f184d830071d6c0" gracePeriod=45 Jul 06 19:47:35.370208 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:47:35.370176 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-shared-a-26761c/sub-test-model-26761c-kserve-7f465bc6c7-8fpmk" podUID="990afe83-30ab-478f-84a8-158a7e55bbeb" containerName="main" probeResult="failure" output="Get \"https://10.132.0.66:8000/ready\": EOF" Jul 06 19:47:35.615489 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:47:35.615464 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-26761c/sub-test-model-26761c-kserve-7f465bc6c7-8fpmk" Jul 06 19:47:35.655419 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:47:35.655336 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4ld6g\" (UniqueName: \"kubernetes.io/projected/990afe83-30ab-478f-84a8-158a7e55bbeb-kube-api-access-4ld6g\") pod \"990afe83-30ab-478f-84a8-158a7e55bbeb\" (UID: \"990afe83-30ab-478f-84a8-158a7e55bbeb\") " Jul 06 19:47:35.655589 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:47:35.655422 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/990afe83-30ab-478f-84a8-158a7e55bbeb-home\") pod \"990afe83-30ab-478f-84a8-158a7e55bbeb\" (UID: \"990afe83-30ab-478f-84a8-158a7e55bbeb\") " Jul 06 19:47:35.655589 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:47:35.655474 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/990afe83-30ab-478f-84a8-158a7e55bbeb-tls-certs\") pod \"990afe83-30ab-478f-84a8-158a7e55bbeb\" (UID: \"990afe83-30ab-478f-84a8-158a7e55bbeb\") " Jul 06 19:47:35.655589 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:47:35.655508 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/990afe83-30ab-478f-84a8-158a7e55bbeb-tmp-dir\") pod \"990afe83-30ab-478f-84a8-158a7e55bbeb\" (UID: \"990afe83-30ab-478f-84a8-158a7e55bbeb\") " Jul 06 19:47:35.655589 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:47:35.655555 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/990afe83-30ab-478f-84a8-158a7e55bbeb-dshm\") pod \"990afe83-30ab-478f-84a8-158a7e55bbeb\" (UID: \"990afe83-30ab-478f-84a8-158a7e55bbeb\") " Jul 06 19:47:35.655854 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:47:35.655596 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/990afe83-30ab-478f-84a8-158a7e55bbeb-model-cache\") pod \"990afe83-30ab-478f-84a8-158a7e55bbeb\" (UID: \"990afe83-30ab-478f-84a8-158a7e55bbeb\") " Jul 06 19:47:35.655854 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:47:35.655710 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/990afe83-30ab-478f-84a8-158a7e55bbeb-home" (OuterVolumeSpecName: "home") pod "990afe83-30ab-478f-84a8-158a7e55bbeb" (UID: "990afe83-30ab-478f-84a8-158a7e55bbeb"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:47:35.655957 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:47:35.655890 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/990afe83-30ab-478f-84a8-158a7e55bbeb-model-cache" (OuterVolumeSpecName: "model-cache") pod "990afe83-30ab-478f-84a8-158a7e55bbeb" (UID: "990afe83-30ab-478f-84a8-158a7e55bbeb"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:47:35.656009 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:47:35.655962 2566 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/990afe83-30ab-478f-84a8-158a7e55bbeb-model-cache\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:47:35.656009 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:47:35.655981 2566 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/990afe83-30ab-478f-84a8-158a7e55bbeb-home\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:47:35.656371 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:47:35.656181 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/990afe83-30ab-478f-84a8-158a7e55bbeb-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "990afe83-30ab-478f-84a8-158a7e55bbeb" (UID: "990afe83-30ab-478f-84a8-158a7e55bbeb"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:47:35.657983 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:47:35.657956 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/990afe83-30ab-478f-84a8-158a7e55bbeb-kube-api-access-4ld6g" (OuterVolumeSpecName: "kube-api-access-4ld6g") pod "990afe83-30ab-478f-84a8-158a7e55bbeb" (UID: "990afe83-30ab-478f-84a8-158a7e55bbeb"). InnerVolumeSpecName "kube-api-access-4ld6g". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:47:35.658126 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:47:35.658085 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/990afe83-30ab-478f-84a8-158a7e55bbeb-dshm" (OuterVolumeSpecName: "dshm") pod "990afe83-30ab-478f-84a8-158a7e55bbeb" (UID: "990afe83-30ab-478f-84a8-158a7e55bbeb"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:47:35.658243 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:47:35.658219 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/990afe83-30ab-478f-84a8-158a7e55bbeb-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "990afe83-30ab-478f-84a8-158a7e55bbeb" (UID: "990afe83-30ab-478f-84a8-158a7e55bbeb"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 06 19:47:35.759833 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:47:35.759792 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4ld6g\" (UniqueName: \"kubernetes.io/projected/990afe83-30ab-478f-84a8-158a7e55bbeb-kube-api-access-4ld6g\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:47:35.759833 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:47:35.759833 2566 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/990afe83-30ab-478f-84a8-158a7e55bbeb-tls-certs\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:47:35.760037 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:47:35.759850 2566 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/990afe83-30ab-478f-84a8-158a7e55bbeb-tmp-dir\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:47:35.760037 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:47:35.759861 2566 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/990afe83-30ab-478f-84a8-158a7e55bbeb-dshm\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:47:35.840525 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:47:35.840481 2566 generic.go:356] "Generic (PLEG): container finished" podID="990afe83-30ab-478f-84a8-158a7e55bbeb" containerID="89cd23fcc20838839572f3be1761858af30c0094498d96bc5f184d830071d6c0" exitCode=0 Jul 06 19:47:35.840712 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:47:35.840537 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-26761c/sub-test-model-26761c-kserve-7f465bc6c7-8fpmk" Jul 06 19:47:35.840712 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:47:35.840559 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-26761c/sub-test-model-26761c-kserve-7f465bc6c7-8fpmk" event={"ID":"990afe83-30ab-478f-84a8-158a7e55bbeb","Type":"ContainerDied","Data":"89cd23fcc20838839572f3be1761858af30c0094498d96bc5f184d830071d6c0"} Jul 06 19:47:35.840712 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:47:35.840624 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-26761c/sub-test-model-26761c-kserve-7f465bc6c7-8fpmk" event={"ID":"990afe83-30ab-478f-84a8-158a7e55bbeb","Type":"ContainerDied","Data":"f34efedff44aca4556c58b76d8a6ba615a1110530561e6bbbb574662eadb9d9a"} Jul 06 19:47:35.840712 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:47:35.840648 2566 scope.go:117] "RemoveContainer" containerID="89cd23fcc20838839572f3be1761858af30c0094498d96bc5f184d830071d6c0" Jul 06 19:47:35.852459 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:47:35.852431 2566 scope.go:117] "RemoveContainer" containerID="89cd23fcc20838839572f3be1761858af30c0094498d96bc5f184d830071d6c0" Jul 06 19:47:35.852762 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:47:35.852741 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"89cd23fcc20838839572f3be1761858af30c0094498d96bc5f184d830071d6c0\": container with ID starting with 89cd23fcc20838839572f3be1761858af30c0094498d96bc5f184d830071d6c0 not found: ID does not exist" containerID="89cd23fcc20838839572f3be1761858af30c0094498d96bc5f184d830071d6c0" Jul 06 19:47:35.852848 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:47:35.852771 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"89cd23fcc20838839572f3be1761858af30c0094498d96bc5f184d830071d6c0"} err="failed to get container status \"89cd23fcc20838839572f3be1761858af30c0094498d96bc5f184d830071d6c0\": rpc error: code = NotFound desc = could not find container \"89cd23fcc20838839572f3be1761858af30c0094498d96bc5f184d830071d6c0\": container with ID starting with 89cd23fcc20838839572f3be1761858af30c0094498d96bc5f184d830071d6c0 not found: ID does not exist" Jul 06 19:47:35.865898 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:47:35.865859 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-26761c/sub-test-model-26761c-kserve-7f465bc6c7-8fpmk"] Jul 06 19:47:35.869667 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:47:35.869641 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-26761c/sub-test-model-26761c-kserve-7f465bc6c7-8fpmk"] Jul 06 19:47:37.237555 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:47:37.237524 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="990afe83-30ab-478f-84a8-158a7e55bbeb" path="/var/lib/kubelet/pods/990afe83-30ab-478f-84a8-158a7e55bbeb/volumes" Jul 06 19:47:57.986356 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:47:57.986308 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:48:03.724178 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:48:03.724143 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:48:03.827286 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:48:03.827252 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:48:19.571589 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:48:19.571553 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:48:19.668657 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:48:19.668620 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:48:28.860061 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:48:28.859969 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:48:28.957954 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:48:28.957922 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:48:35.200671 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:48:35.200621 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:48:36.098458 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:48:36.098427 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:48:37.137052 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:48:37.137017 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:48:37.535317 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:48:37.535287 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:48:37.632949 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:48:37.632914 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:48:37.734014 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:48:37.733984 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:48:37.823060 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:48:37.822970 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:48:42.612596 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:48:42.612562 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:48:51.415853 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:48:51.415817 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:49:00.804537 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:00.804499 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:49:19.813900 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:19.813867 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:49:20.051872 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:20.051831 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-26761c-openshift-default-7695b94987-wxtmc"] Jul 06 19:49:20.052163 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:20.052117 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-26761c-openshift-default-7695b94987-wxtmc" podUID="d3091ded-001f-4349-bb50-6adec8921337" containerName="istio-proxy" containerID="cri-o://0ec4e9b69eb5bceedb2abe2a73bc937a447c60c55979f054edbdfd7844e9d2c4" gracePeriod=30 Jul 06 19:49:20.067240 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:20.067166 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:49:20.301129 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:20.301094 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:49:20.459099 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:20.458992 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:49:20.854018 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:20.853988 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:49:21.002057 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:21.002023 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:49:21.103587 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:21.103554 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:49:21.264450 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:21.264361 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:49:25.309811 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:25.309782 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-26761c-openshift-default-7695b94987-wxtmc" Jul 06 19:49:25.382919 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:25.382829 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3091ded-001f-4349-bb50-6adec8921337-credential-socket\") pod \"d3091ded-001f-4349-bb50-6adec8921337\" (UID: \"d3091ded-001f-4349-bb50-6adec8921337\") " Jul 06 19:49:25.382919 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:25.382869 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d3091ded-001f-4349-bb50-6adec8921337-istio-token\") pod \"d3091ded-001f-4349-bb50-6adec8921337\" (UID: \"d3091ded-001f-4349-bb50-6adec8921337\") " Jul 06 19:49:25.382919 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:25.382895 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d3091ded-001f-4349-bb50-6adec8921337-workload-certs\") pod \"d3091ded-001f-4349-bb50-6adec8921337\" (UID: \"d3091ded-001f-4349-bb50-6adec8921337\") " Jul 06 19:49:25.382919 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:25.382920 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d3091ded-001f-4349-bb50-6adec8921337-istio-podinfo\") pod \"d3091ded-001f-4349-bb50-6adec8921337\" (UID: \"d3091ded-001f-4349-bb50-6adec8921337\") " Jul 06 19:49:25.383260 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:25.383063 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hlb9r\" (UniqueName: \"kubernetes.io/projected/d3091ded-001f-4349-bb50-6adec8921337-kube-api-access-hlb9r\") pod \"d3091ded-001f-4349-bb50-6adec8921337\" (UID: \"d3091ded-001f-4349-bb50-6adec8921337\") " Jul 06 19:49:25.383260 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:25.383138 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d3091ded-001f-4349-bb50-6adec8921337-istio-envoy\") pod \"d3091ded-001f-4349-bb50-6adec8921337\" (UID: \"d3091ded-001f-4349-bb50-6adec8921337\") " Jul 06 19:49:25.383260 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:25.383194 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d3091ded-001f-4349-bb50-6adec8921337-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d3091ded-001f-4349-bb50-6adec8921337" (UID: "d3091ded-001f-4349-bb50-6adec8921337"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:49:25.383260 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:25.383200 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d3091ded-001f-4349-bb50-6adec8921337-istio-data\") pod \"d3091ded-001f-4349-bb50-6adec8921337\" (UID: \"d3091ded-001f-4349-bb50-6adec8921337\") " Jul 06 19:49:25.383260 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:25.383213 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d3091ded-001f-4349-bb50-6adec8921337-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d3091ded-001f-4349-bb50-6adec8921337" (UID: "d3091ded-001f-4349-bb50-6adec8921337"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:49:25.383512 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:25.383291 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d3091ded-001f-4349-bb50-6adec8921337-istiod-ca-cert\") pod \"d3091ded-001f-4349-bb50-6adec8921337\" (UID: \"d3091ded-001f-4349-bb50-6adec8921337\") " Jul 06 19:49:25.383512 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:25.383370 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3091ded-001f-4349-bb50-6adec8921337-workload-socket\") pod \"d3091ded-001f-4349-bb50-6adec8921337\" (UID: \"d3091ded-001f-4349-bb50-6adec8921337\") " Jul 06 19:49:25.383683 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:25.383643 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d3091ded-001f-4349-bb50-6adec8921337-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d3091ded-001f-4349-bb50-6adec8921337" (UID: "d3091ded-001f-4349-bb50-6adec8921337"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:49:25.383750 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:25.383703 2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3091ded-001f-4349-bb50-6adec8921337-credential-socket\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:49:25.383750 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:25.383719 2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d3091ded-001f-4349-bb50-6adec8921337-workload-certs\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:49:25.383824 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:25.383704 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d3091ded-001f-4349-bb50-6adec8921337-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d3091ded-001f-4349-bb50-6adec8921337" (UID: "d3091ded-001f-4349-bb50-6adec8921337"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:49:25.383888 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:25.383870 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d3091ded-001f-4349-bb50-6adec8921337-istio-data" (OuterVolumeSpecName: "istio-data") pod "d3091ded-001f-4349-bb50-6adec8921337" (UID: "d3091ded-001f-4349-bb50-6adec8921337"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:49:25.385871 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:25.385843 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d3091ded-001f-4349-bb50-6adec8921337-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d3091ded-001f-4349-bb50-6adec8921337" (UID: "d3091ded-001f-4349-bb50-6adec8921337"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:49:25.385871 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:25.385863 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d3091ded-001f-4349-bb50-6adec8921337-kube-api-access-hlb9r" (OuterVolumeSpecName: "kube-api-access-hlb9r") pod "d3091ded-001f-4349-bb50-6adec8921337" (UID: "d3091ded-001f-4349-bb50-6adec8921337"). InnerVolumeSpecName "kube-api-access-hlb9r". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:49:25.385995 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:25.385884 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d3091ded-001f-4349-bb50-6adec8921337-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d3091ded-001f-4349-bb50-6adec8921337" (UID: "d3091ded-001f-4349-bb50-6adec8921337"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 19:49:25.385995 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:25.385892 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d3091ded-001f-4349-bb50-6adec8921337-istio-token" (OuterVolumeSpecName: "istio-token") pod "d3091ded-001f-4349-bb50-6adec8921337" (UID: "d3091ded-001f-4349-bb50-6adec8921337"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:49:25.467547 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:25.467513 2566 generic.go:356] "Generic (PLEG): container finished" podID="d3091ded-001f-4349-bb50-6adec8921337" containerID="0ec4e9b69eb5bceedb2abe2a73bc937a447c60c55979f054edbdfd7844e9d2c4" exitCode=0 Jul 06 19:49:25.467714 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:25.467564 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-26761c-openshift-default-7695b94987-wxtmc" Jul 06 19:49:25.467714 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:25.467603 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-26761c-openshift-default-7695b94987-wxtmc" event={"ID":"d3091ded-001f-4349-bb50-6adec8921337","Type":"ContainerDied","Data":"0ec4e9b69eb5bceedb2abe2a73bc937a447c60c55979f054edbdfd7844e9d2c4"} Jul 06 19:49:25.467714 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:25.467659 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-26761c-openshift-default-7695b94987-wxtmc" event={"ID":"d3091ded-001f-4349-bb50-6adec8921337","Type":"ContainerDied","Data":"c226ca63ff98cc4c3ad8c6ee9a28a7f87070f70d057f2097cc4ff54e9d56def5"} Jul 06 19:49:25.467714 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:25.467684 2566 scope.go:117] "RemoveContainer" containerID="0ec4e9b69eb5bceedb2abe2a73bc937a447c60c55979f054edbdfd7844e9d2c4" Jul 06 19:49:25.479887 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:25.479869 2566 scope.go:117] "RemoveContainer" containerID="da8f3bf9ac99b8157b40ade53036649f8f9792b09328c375149e23000f5d6eec" Jul 06 19:49:25.485225 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:25.485201 2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d3091ded-001f-4349-bb50-6adec8921337-istio-token\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:49:25.485312 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:25.485231 2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d3091ded-001f-4349-bb50-6adec8921337-istio-podinfo\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:49:25.485312 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:25.485244 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hlb9r\" (UniqueName: \"kubernetes.io/projected/d3091ded-001f-4349-bb50-6adec8921337-kube-api-access-hlb9r\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:49:25.485312 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:25.485267 2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d3091ded-001f-4349-bb50-6adec8921337-istio-envoy\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:49:25.485312 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:25.485281 2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d3091ded-001f-4349-bb50-6adec8921337-istio-data\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:49:25.485312 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:25.485293 2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d3091ded-001f-4349-bb50-6adec8921337-istiod-ca-cert\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:49:25.485312 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:25.485307 2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3091ded-001f-4349-bb50-6adec8921337-workload-socket\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:49:25.488576 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:25.488559 2566 scope.go:117] "RemoveContainer" containerID="0ec4e9b69eb5bceedb2abe2a73bc937a447c60c55979f054edbdfd7844e9d2c4" Jul 06 19:49:25.488845 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:49:25.488819 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0ec4e9b69eb5bceedb2abe2a73bc937a447c60c55979f054edbdfd7844e9d2c4\": container with ID starting with 0ec4e9b69eb5bceedb2abe2a73bc937a447c60c55979f054edbdfd7844e9d2c4 not found: ID does not exist" containerID="0ec4e9b69eb5bceedb2abe2a73bc937a447c60c55979f054edbdfd7844e9d2c4" Jul 06 19:49:25.488933 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:25.488851 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ec4e9b69eb5bceedb2abe2a73bc937a447c60c55979f054edbdfd7844e9d2c4"} err="failed to get container status \"0ec4e9b69eb5bceedb2abe2a73bc937a447c60c55979f054edbdfd7844e9d2c4\": rpc error: code = NotFound desc = could not find container \"0ec4e9b69eb5bceedb2abe2a73bc937a447c60c55979f054edbdfd7844e9d2c4\": container with ID starting with 0ec4e9b69eb5bceedb2abe2a73bc937a447c60c55979f054edbdfd7844e9d2c4 not found: ID does not exist" Jul 06 19:49:25.488933 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:25.488868 2566 scope.go:117] "RemoveContainer" containerID="da8f3bf9ac99b8157b40ade53036649f8f9792b09328c375149e23000f5d6eec" Jul 06 19:49:25.489165 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:49:25.489139 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"da8f3bf9ac99b8157b40ade53036649f8f9792b09328c375149e23000f5d6eec\": container with ID starting with da8f3bf9ac99b8157b40ade53036649f8f9792b09328c375149e23000f5d6eec not found: ID does not exist" containerID="da8f3bf9ac99b8157b40ade53036649f8f9792b09328c375149e23000f5d6eec" Jul 06 19:49:25.489237 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:25.489173 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da8f3bf9ac99b8157b40ade53036649f8f9792b09328c375149e23000f5d6eec"} err="failed to get container status \"da8f3bf9ac99b8157b40ade53036649f8f9792b09328c375149e23000f5d6eec\": rpc error: code = NotFound desc = could not find container \"da8f3bf9ac99b8157b40ade53036649f8f9792b09328c375149e23000f5d6eec\": container with ID starting with da8f3bf9ac99b8157b40ade53036649f8f9792b09328c375149e23000f5d6eec not found: ID does not exist" Jul 06 19:49:25.492140 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:25.492116 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-26761c-openshift-default-7695b94987-wxtmc"] Jul 06 19:49:25.496461 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:25.496432 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-26761c-openshift-default-7695b94987-wxtmc"] Jul 06 19:49:26.729938 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:26.729902 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:49:26.793413 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:26.793376 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:49:26.967023 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:26.966987 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-bb8c15-openshift-default-6b66779f5-h2lmx"] Jul 06 19:49:26.967289 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:26.967257 2566 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-bb8c15-openshift-default-6b66779f5-h2lmx" podUID="b6da034d-1bae-46ab-9bd1-3a6fe5809201" containerName="istio-proxy" containerID="cri-o://aa254f64560b2d773344891570fa660841ca956e2ac98b2e9b3b98297a92609e" gracePeriod=30 Jul 06 19:49:26.988137 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:26.988046 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:49:27.219101 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:27.219049 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:49:27.237877 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:27.237846 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d3091ded-001f-4349-bb50-6adec8921337" path="/var/lib/kubelet/pods/d3091ded-001f-4349-bb50-6adec8921337/volumes" Jul 06 19:49:32.223781 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:32.223758 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-bb8c15-openshift-default-6b66779f5-h2lmx" Jul 06 19:49:32.300219 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:32.300185 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-wkh6n_abe694ae-dad6-43a4-a8e0-767db8e50929/manager/0.log" Jul 06 19:49:32.347519 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:32.347482 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b6da034d-1bae-46ab-9bd1-3a6fe5809201-istio-podinfo\") pod \"b6da034d-1bae-46ab-9bd1-3a6fe5809201\" (UID: \"b6da034d-1bae-46ab-9bd1-3a6fe5809201\") " Jul 06 19:49:32.347697 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:32.347544 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b6da034d-1bae-46ab-9bd1-3a6fe5809201-istio-envoy\") pod \"b6da034d-1bae-46ab-9bd1-3a6fe5809201\" (UID: \"b6da034d-1bae-46ab-9bd1-3a6fe5809201\") " Jul 06 19:49:32.347697 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:32.347566 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gq4rs\" (UniqueName: \"kubernetes.io/projected/b6da034d-1bae-46ab-9bd1-3a6fe5809201-kube-api-access-gq4rs\") pod \"b6da034d-1bae-46ab-9bd1-3a6fe5809201\" (UID: \"b6da034d-1bae-46ab-9bd1-3a6fe5809201\") " Jul 06 19:49:32.347697 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:32.347611 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6da034d-1bae-46ab-9bd1-3a6fe5809201-credential-socket\") pod \"b6da034d-1bae-46ab-9bd1-3a6fe5809201\" (UID: \"b6da034d-1bae-46ab-9bd1-3a6fe5809201\") " Jul 06 19:49:32.347697 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:32.347636 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b6da034d-1bae-46ab-9bd1-3a6fe5809201-istio-data\") pod \"b6da034d-1bae-46ab-9bd1-3a6fe5809201\" (UID: \"b6da034d-1bae-46ab-9bd1-3a6fe5809201\") " Jul 06 19:49:32.347697 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:32.347684 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b6da034d-1bae-46ab-9bd1-3a6fe5809201-istiod-ca-cert\") pod \"b6da034d-1bae-46ab-9bd1-3a6fe5809201\" (UID: \"b6da034d-1bae-46ab-9bd1-3a6fe5809201\") " Jul 06 19:49:32.348023 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:32.347702 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b6da034d-1bae-46ab-9bd1-3a6fe5809201-workload-certs\") pod \"b6da034d-1bae-46ab-9bd1-3a6fe5809201\" (UID: \"b6da034d-1bae-46ab-9bd1-3a6fe5809201\") " Jul 06 19:49:32.348023 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:32.347747 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b6da034d-1bae-46ab-9bd1-3a6fe5809201-istio-token\") pod \"b6da034d-1bae-46ab-9bd1-3a6fe5809201\" (UID: \"b6da034d-1bae-46ab-9bd1-3a6fe5809201\") " Jul 06 19:49:32.348023 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:32.347776 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6da034d-1bae-46ab-9bd1-3a6fe5809201-workload-socket\") pod \"b6da034d-1bae-46ab-9bd1-3a6fe5809201\" (UID: \"b6da034d-1bae-46ab-9bd1-3a6fe5809201\") " Jul 06 19:49:32.348023 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:32.347941 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6da034d-1bae-46ab-9bd1-3a6fe5809201-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b6da034d-1bae-46ab-9bd1-3a6fe5809201" (UID: "b6da034d-1bae-46ab-9bd1-3a6fe5809201"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:49:32.348259 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:32.348155 2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6da034d-1bae-46ab-9bd1-3a6fe5809201-credential-socket\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:49:32.348259 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:32.348160 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6da034d-1bae-46ab-9bd1-3a6fe5809201-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b6da034d-1bae-46ab-9bd1-3a6fe5809201" (UID: "b6da034d-1bae-46ab-9bd1-3a6fe5809201"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:49:32.348259 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:32.348171 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6da034d-1bae-46ab-9bd1-3a6fe5809201-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b6da034d-1bae-46ab-9bd1-3a6fe5809201" (UID: "b6da034d-1bae-46ab-9bd1-3a6fe5809201"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:49:32.348259 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:32.348209 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b6da034d-1bae-46ab-9bd1-3a6fe5809201-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b6da034d-1bae-46ab-9bd1-3a6fe5809201" (UID: "b6da034d-1bae-46ab-9bd1-3a6fe5809201"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 06 19:49:32.348516 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:32.348494 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6da034d-1bae-46ab-9bd1-3a6fe5809201-istio-data" (OuterVolumeSpecName: "istio-data") pod "b6da034d-1bae-46ab-9bd1-3a6fe5809201" (UID: "b6da034d-1bae-46ab-9bd1-3a6fe5809201"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:49:32.350012 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:32.349989 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b6da034d-1bae-46ab-9bd1-3a6fe5809201-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b6da034d-1bae-46ab-9bd1-3a6fe5809201" (UID: "b6da034d-1bae-46ab-9bd1-3a6fe5809201"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 06 19:49:32.350325 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:32.350293 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6da034d-1bae-46ab-9bd1-3a6fe5809201-istio-token" (OuterVolumeSpecName: "istio-token") pod "b6da034d-1bae-46ab-9bd1-3a6fe5809201" (UID: "b6da034d-1bae-46ab-9bd1-3a6fe5809201"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:49:32.350429 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:32.350382 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b6da034d-1bae-46ab-9bd1-3a6fe5809201-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b6da034d-1bae-46ab-9bd1-3a6fe5809201" (UID: "b6da034d-1bae-46ab-9bd1-3a6fe5809201"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 06 19:49:32.350503 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:32.350479 2566 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6da034d-1bae-46ab-9bd1-3a6fe5809201-kube-api-access-gq4rs" (OuterVolumeSpecName: "kube-api-access-gq4rs") pod "b6da034d-1bae-46ab-9bd1-3a6fe5809201" (UID: "b6da034d-1bae-46ab-9bd1-3a6fe5809201"). InnerVolumeSpecName "kube-api-access-gq4rs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 06 19:49:32.449562 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:32.449525 2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b6da034d-1bae-46ab-9bd1-3a6fe5809201-istio-podinfo\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:49:32.449562 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:32.449561 2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b6da034d-1bae-46ab-9bd1-3a6fe5809201-istio-envoy\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:49:32.449767 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:32.449574 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gq4rs\" (UniqueName: \"kubernetes.io/projected/b6da034d-1bae-46ab-9bd1-3a6fe5809201-kube-api-access-gq4rs\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:49:32.449767 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:32.449588 2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b6da034d-1bae-46ab-9bd1-3a6fe5809201-istio-data\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:49:32.449767 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:32.449599 2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b6da034d-1bae-46ab-9bd1-3a6fe5809201-istiod-ca-cert\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:49:32.449767 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:32.449607 2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b6da034d-1bae-46ab-9bd1-3a6fe5809201-workload-certs\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:49:32.449767 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:32.449614 2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b6da034d-1bae-46ab-9bd1-3a6fe5809201-istio-token\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:49:32.449767 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:32.449623 2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b6da034d-1bae-46ab-9bd1-3a6fe5809201-workload-socket\") on node \"ip-10-0-138-35.ec2.internal\" DevicePath \"\"" Jul 06 19:49:32.509503 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:32.509466 2566 generic.go:356] "Generic (PLEG): container finished" podID="b6da034d-1bae-46ab-9bd1-3a6fe5809201" containerID="aa254f64560b2d773344891570fa660841ca956e2ac98b2e9b3b98297a92609e" exitCode=0 Jul 06 19:49:32.509718 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:32.509515 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-bb8c15-openshift-default-6b66779f5-h2lmx" event={"ID":"b6da034d-1bae-46ab-9bd1-3a6fe5809201","Type":"ContainerDied","Data":"aa254f64560b2d773344891570fa660841ca956e2ac98b2e9b3b98297a92609e"} Jul 06 19:49:32.509718 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:32.509523 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-bb8c15-openshift-default-6b66779f5-h2lmx" Jul 06 19:49:32.509718 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:32.509568 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-bb8c15-openshift-default-6b66779f5-h2lmx" event={"ID":"b6da034d-1bae-46ab-9bd1-3a6fe5809201","Type":"ContainerDied","Data":"48ed4676aa0dc0ce6ee41e0775dac639651347427f17d6648df3561169e44337"} Jul 06 19:49:32.509718 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:32.509595 2566 scope.go:117] "RemoveContainer" containerID="aa254f64560b2d773344891570fa660841ca956e2ac98b2e9b3b98297a92609e" Jul 06 19:49:32.519901 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:32.519883 2566 scope.go:117] "RemoveContainer" containerID="2ffde3564793e675e4bfb6a9004bc0bd7b88bcc468d4e75c64e20bc3e903f5fb" Jul 06 19:49:32.529668 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:32.529642 2566 scope.go:117] "RemoveContainer" containerID="aa254f64560b2d773344891570fa660841ca956e2ac98b2e9b3b98297a92609e" Jul 06 19:49:32.529966 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:49:32.529941 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aa254f64560b2d773344891570fa660841ca956e2ac98b2e9b3b98297a92609e\": container with ID starting with aa254f64560b2d773344891570fa660841ca956e2ac98b2e9b3b98297a92609e not found: ID does not exist" containerID="aa254f64560b2d773344891570fa660841ca956e2ac98b2e9b3b98297a92609e" Jul 06 19:49:32.530062 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:32.529973 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aa254f64560b2d773344891570fa660841ca956e2ac98b2e9b3b98297a92609e"} err="failed to get container status \"aa254f64560b2d773344891570fa660841ca956e2ac98b2e9b3b98297a92609e\": rpc error: code = NotFound desc = could not find container \"aa254f64560b2d773344891570fa660841ca956e2ac98b2e9b3b98297a92609e\": container with ID starting with aa254f64560b2d773344891570fa660841ca956e2ac98b2e9b3b98297a92609e not found: ID does not exist" Jul 06 19:49:32.530062 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:32.529999 2566 scope.go:117] "RemoveContainer" containerID="2ffde3564793e675e4bfb6a9004bc0bd7b88bcc468d4e75c64e20bc3e903f5fb" Jul 06 19:49:32.530315 ip-10-0-138-35 kubenswrapper[2566]: E0706 19:49:32.530289 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2ffde3564793e675e4bfb6a9004bc0bd7b88bcc468d4e75c64e20bc3e903f5fb\": container with ID starting with 2ffde3564793e675e4bfb6a9004bc0bd7b88bcc468d4e75c64e20bc3e903f5fb not found: ID does not exist" containerID="2ffde3564793e675e4bfb6a9004bc0bd7b88bcc468d4e75c64e20bc3e903f5fb" Jul 06 19:49:32.530420 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:32.530321 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2ffde3564793e675e4bfb6a9004bc0bd7b88bcc468d4e75c64e20bc3e903f5fb"} err="failed to get container status \"2ffde3564793e675e4bfb6a9004bc0bd7b88bcc468d4e75c64e20bc3e903f5fb\": rpc error: code = NotFound desc = could not find container \"2ffde3564793e675e4bfb6a9004bc0bd7b88bcc468d4e75c64e20bc3e903f5fb\": container with ID starting with 2ffde3564793e675e4bfb6a9004bc0bd7b88bcc468d4e75c64e20bc3e903f5fb not found: ID does not exist" Jul 06 19:49:32.531135 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:32.531117 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-5d68c7bd7c-4ltxv_f6e6c625-f78a-4051-bf1f-10953d59dba1/maas-api/0.log" Jul 06 19:49:32.532370 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:32.532347 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-bb8c15-openshift-default-6b66779f5-h2lmx"] Jul 06 19:49:32.534630 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:32.534610 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-bb8c15-openshift-default-6b66779f5-h2lmx"] Jul 06 19:49:33.006461 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:33.006431 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-bl6x9_b2160904-d8c6-4fd7-a41f-a10ff617640f/server/0.log" Jul 06 19:49:33.120823 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:33.120769 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-sdhp4_c10a5b93-4175-4ac7-a141-6c248f02bd8c/manager/0.log" Jul 06 19:49:33.228096 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:33.228033 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-55c47dfb7f-tbk57_f4693d1d-13dd-4b20-a725-f722f97cee41/copy-manifests/0.log" Jul 06 19:49:33.233692 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:33.233664 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-55c47dfb7f-tbk57_f4693d1d-13dd-4b20-a725-f722f97cee41/manager/0.log" Jul 06 19:49:33.236898 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:33.236873 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b6da034d-1bae-46ab-9bd1-3a6fe5809201" path="/var/lib/kubelet/pods/b6da034d-1bae-46ab-9bd1-3a6fe5809201/volumes" Jul 06 19:49:34.382025 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:34.381999 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594zlb4_403eaac6-75e0-459d-a251-a28e7e46c434/util/0.log" Jul 06 19:49:34.388748 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:34.388727 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594zlb4_403eaac6-75e0-459d-a251-a28e7e46c434/pull/0.log" Jul 06 19:49:34.394756 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:34.394737 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594zlb4_403eaac6-75e0-459d-a251-a28e7e46c434/extract/0.log" Jul 06 19:49:34.503484 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:34.503453 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zcf4q_0d0869d7-b5a9-4689-93d2-90ed8ad52d52/extract/0.log" Jul 06 19:49:34.509379 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:34.509355 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zcf4q_0d0869d7-b5a9-4689-93d2-90ed8ad52d52/util/0.log" Jul 06 19:49:34.515010 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:34.514990 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zcf4q_0d0869d7-b5a9-4689-93d2-90ed8ad52d52/pull/0.log" Jul 06 19:49:34.618660 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:34.618636 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732l4dr_101e034c-ad87-4f3c-af45-de74d04a1c21/util/0.log" Jul 06 19:49:34.624439 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:34.624420 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732l4dr_101e034c-ad87-4f3c-af45-de74d04a1c21/pull/0.log" Jul 06 19:49:34.630222 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:34.630201 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732l4dr_101e034c-ad87-4f3c-af45-de74d04a1c21/extract/0.log" Jul 06 19:49:34.738661 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:34.738589 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mkghr_b8af6bcb-5bcc-4465-85e0-778e48921c4c/util/0.log" Jul 06 19:49:34.744656 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:34.744610 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mkghr_b8af6bcb-5bcc-4465-85e0-778e48921c4c/pull/0.log" Jul 06 19:49:34.750343 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:34.750323 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mkghr_b8af6bcb-5bcc-4465-85e0-778e48921c4c/extract/0.log" Jul 06 19:49:34.986006 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:34.985975 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-rddw8_bf9d0112-eb11-4996-8fca-e6a781f22e14/manager/0.log" Jul 06 19:49:35.096206 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:35.096180 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-5j2ll_f2e760ff-630e-4095-ae84-8377ff92111c/manager/0.log" Jul 06 19:49:35.203168 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:35.203141 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-q2m67_5ef34edc-9e6f-42ed-a3f1-ac7b914b2921/kuadrant-console-plugin/0.log" Jul 06 19:49:35.319615 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:35.319584 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-zfdvv_43e2d40f-c63e-4db9-b0f0-25cf4189f93d/registry-server/0.log" Jul 06 19:49:35.442292 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:35.442214 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-f78vh_b129a63b-6e6f-4ca8-bd52-05d8638059fd/manager/0.log" Jul 06 19:49:35.679889 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:35.679857 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-lxjln_75cfa14e-1d0b-490c-ac55-15c3715e8adb/manager/0.log" Jul 06 19:49:36.026044 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:36.026014 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4kzvjv_fc73ba95-f338-4688-a38a-ee10bbce473d/istio-proxy/0.log" Jul 06 19:49:36.374116 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:36.374062 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-754576df6f-jrtrg_e1be9243-b0bd-493c-8e06-e3f6c2c8a9a7/kube-auth-proxy/0.log" Jul 06 19:49:36.488206 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:36.488172 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-t9fgz_0af75ee7-3c24-4d81-9764-64b52e4f3623/istio-proxy/1.log" Jul 06 19:49:36.719292 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:36.719182 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-vt6xf_1e8eaff3-cc54-4648-b0c6-cb184ba29ff8/payload-processing/0.log" Jul 06 19:49:37.193027 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:37.192989 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-bqfjf_8eb3f370-ef60-4437-884e-967f9f8b27f9/main/0.log" Jul 06 19:49:37.753496 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:37.753473 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-sl787_7f7af955-921e-4eb0-99f5-60af3cc477ec/main/0.log" Jul 06 19:49:42.327243 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:42.327205 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-p6v6j/must-gather-nkmwz"] Jul 06 19:49:42.327775 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:42.327753 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="990afe83-30ab-478f-84a8-158a7e55bbeb" containerName="main" Jul 06 19:49:42.327858 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:42.327779 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="990afe83-30ab-478f-84a8-158a7e55bbeb" containerName="main" Jul 06 19:49:42.327858 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:42.327792 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d3091ded-001f-4349-bb50-6adec8921337" containerName="istio-proxy" Jul 06 19:49:42.327858 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:42.327801 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="d3091ded-001f-4349-bb50-6adec8921337" containerName="istio-proxy" Jul 06 19:49:42.327858 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:42.327818 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d3091ded-001f-4349-bb50-6adec8921337" containerName="istio-proxy" Jul 06 19:49:42.327858 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:42.327826 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="d3091ded-001f-4349-bb50-6adec8921337" containerName="istio-proxy" Jul 06 19:49:42.327858 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:42.327837 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b6da034d-1bae-46ab-9bd1-3a6fe5809201" containerName="istio-proxy" Jul 06 19:49:42.327858 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:42.327844 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="b6da034d-1bae-46ab-9bd1-3a6fe5809201" containerName="istio-proxy" Jul 06 19:49:42.328243 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:42.327870 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b6da034d-1bae-46ab-9bd1-3a6fe5809201" containerName="istio-proxy" Jul 06 19:49:42.328243 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:42.327878 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="b6da034d-1bae-46ab-9bd1-3a6fe5809201" containerName="istio-proxy" Jul 06 19:49:42.328243 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:42.327978 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="990afe83-30ab-478f-84a8-158a7e55bbeb" containerName="main" Jul 06 19:49:42.328243 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:42.327992 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="b6da034d-1bae-46ab-9bd1-3a6fe5809201" containerName="istio-proxy" Jul 06 19:49:42.328243 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:42.328002 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="d3091ded-001f-4349-bb50-6adec8921337" containerName="istio-proxy" Jul 06 19:49:42.328243 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:42.328012 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="b6da034d-1bae-46ab-9bd1-3a6fe5809201" containerName="istio-proxy" Jul 06 19:49:42.328243 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:42.328022 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="d3091ded-001f-4349-bb50-6adec8921337" containerName="istio-proxy" Jul 06 19:49:42.335881 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:42.335849 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-p6v6j/must-gather-nkmwz" Jul 06 19:49:42.338828 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:42.338802 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-p6v6j\"/\"kube-root-ca.crt\"" Jul 06 19:49:42.338987 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:42.338865 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-p6v6j\"/\"default-dockercfg-f9kr7\"" Jul 06 19:49:42.338987 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:42.338968 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:49:42.339426 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:42.339405 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-p6v6j\"/\"openshift-service-ca.crt\"" Jul 06 19:49:42.340222 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:42.340198 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-p6v6j/must-gather-nkmwz"] Jul 06 19:49:42.436986 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:42.436948 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" limit=1024 Jul 06 19:49:42.441014 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:42.440994 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hq5np\" (UniqueName: \"kubernetes.io/projected/729f01f3-d05b-41d1-9134-3ea7f41bfc9f-kube-api-access-hq5np\") pod \"must-gather-nkmwz\" (UID: \"729f01f3-d05b-41d1-9134-3ea7f41bfc9f\") " pod="openshift-must-gather-p6v6j/must-gather-nkmwz" Jul 06 19:49:42.441122 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:42.441023 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/729f01f3-d05b-41d1-9134-3ea7f41bfc9f-must-gather-output\") pod \"must-gather-nkmwz\" (UID: \"729f01f3-d05b-41d1-9134-3ea7f41bfc9f\") " pod="openshift-must-gather-p6v6j/must-gather-nkmwz" Jul 06 19:49:42.541925 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:42.541874 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hq5np\" (UniqueName: \"kubernetes.io/projected/729f01f3-d05b-41d1-9134-3ea7f41bfc9f-kube-api-access-hq5np\") pod \"must-gather-nkmwz\" (UID: \"729f01f3-d05b-41d1-9134-3ea7f41bfc9f\") " pod="openshift-must-gather-p6v6j/must-gather-nkmwz" Jul 06 19:49:42.541925 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:42.541928 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/729f01f3-d05b-41d1-9134-3ea7f41bfc9f-must-gather-output\") pod \"must-gather-nkmwz\" (UID: \"729f01f3-d05b-41d1-9134-3ea7f41bfc9f\") " pod="openshift-must-gather-p6v6j/must-gather-nkmwz" Jul 06 19:49:42.542244 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:42.542227 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/729f01f3-d05b-41d1-9134-3ea7f41bfc9f-must-gather-output\") pod \"must-gather-nkmwz\" (UID: \"729f01f3-d05b-41d1-9134-3ea7f41bfc9f\") " pod="openshift-must-gather-p6v6j/must-gather-nkmwz" Jul 06 19:49:42.553523 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:42.553499 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hq5np\" (UniqueName: \"kubernetes.io/projected/729f01f3-d05b-41d1-9134-3ea7f41bfc9f-kube-api-access-hq5np\") pod \"must-gather-nkmwz\" (UID: \"729f01f3-d05b-41d1-9134-3ea7f41bfc9f\") " pod="openshift-must-gather-p6v6j/must-gather-nkmwz" Jul 06 19:49:42.647768 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:42.647672 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-p6v6j/must-gather-nkmwz" Jul 06 19:49:42.784780 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:42.784750 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-p6v6j/must-gather-nkmwz"] Jul 06 19:49:42.788083 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:49:42.788015 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod729f01f3_d05b_41d1_9134_3ea7f41bfc9f.slice/crio-f6dbf935d03a574054024f1bb945291dfa18615a6b645b9ca8fc04c9f9c4ab36 WatchSource:0}: Error finding container f6dbf935d03a574054024f1bb945291dfa18615a6b645b9ca8fc04c9f9c4ab36: Status 404 returned error can't find the container with id f6dbf935d03a574054024f1bb945291dfa18615a6b645b9ca8fc04c9f9c4ab36 Jul 06 19:49:42.789982 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:42.789965 2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 06 19:49:43.571496 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:43.571461 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-p6v6j/must-gather-nkmwz" event={"ID":"729f01f3-d05b-41d1-9134-3ea7f41bfc9f","Type":"ContainerStarted","Data":"f6dbf935d03a574054024f1bb945291dfa18615a6b645b9ca8fc04c9f9c4ab36"} Jul 06 19:49:44.582021 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:44.581987 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-p6v6j/must-gather-nkmwz" event={"ID":"729f01f3-d05b-41d1-9134-3ea7f41bfc9f","Type":"ContainerStarted","Data":"944f3f876c07a96773de868f69e2a66f270e3806500691d7d027f5c602483d6b"} Jul 06 19:49:44.582523 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:44.582032 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-p6v6j/must-gather-nkmwz" event={"ID":"729f01f3-d05b-41d1-9134-3ea7f41bfc9f","Type":"ContainerStarted","Data":"7692e97d8f7b7c1f62cc7bd4342102c7fb3281e3cd6fd9c37465415e14ce51de"} Jul 06 19:49:44.598007 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:44.597930 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-p6v6j/must-gather-nkmwz" podStartSLOduration=1.683371357 podStartE2EDuration="2.597909423s" totalImagesPullingTime="914.538066ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:49:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:49:44.595673368 +0000 UTC m=+2409.903032316" watchObservedRunningTime="2026-07-06 19:49:44.597909423 +0000 UTC m=+2409.905268372" Jul 06 19:49:45.288117 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:45.288092 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-svgl2_e8c2f347-1744-49ff-b46e-f036d27a5e46/global-pull-secret-syncer/0.log" Jul 06 19:49:45.338678 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:45.338647 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-fjdvf_fa566ec3-6b97-4393-9744-5a9f7e8f0614/konnectivity-agent/0.log" Jul 06 19:49:45.452085 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:45.452036 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-138-35.ec2.internal_9fd512c517b497732795138b0a0b8677/haproxy/0.log" Jul 06 19:49:49.419605 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:49.419251 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594zlb4_403eaac6-75e0-459d-a251-a28e7e46c434/extract/0.log" Jul 06 19:49:49.463738 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:49.462956 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594zlb4_403eaac6-75e0-459d-a251-a28e7e46c434/util/0.log" Jul 06 19:49:49.492104 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:49.491611 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b7594zlb4_403eaac6-75e0-459d-a251-a28e7e46c434/pull/0.log" Jul 06 19:49:49.528279 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:49.527689 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zcf4q_0d0869d7-b5a9-4689-93d2-90ed8ad52d52/extract/0.log" Jul 06 19:49:49.562112 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:49.561147 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zcf4q_0d0869d7-b5a9-4689-93d2-90ed8ad52d52/util/0.log" Jul 06 19:49:49.590895 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:49.590846 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0zcf4q_0d0869d7-b5a9-4689-93d2-90ed8ad52d52/pull/0.log" Jul 06 19:49:49.640361 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:49.640247 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732l4dr_101e034c-ad87-4f3c-af45-de74d04a1c21/extract/0.log" Jul 06 19:49:49.687641 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:49.687418 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732l4dr_101e034c-ad87-4f3c-af45-de74d04a1c21/util/0.log" Jul 06 19:49:49.715035 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:49.715001 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed732l4dr_101e034c-ad87-4f3c-af45-de74d04a1c21/pull/0.log" Jul 06 19:49:49.761915 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:49.761835 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mkghr_b8af6bcb-5bcc-4465-85e0-778e48921c4c/extract/0.log" Jul 06 19:49:49.790036 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:49.789996 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mkghr_b8af6bcb-5bcc-4465-85e0-778e48921c4c/util/0.log" Jul 06 19:49:49.816043 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:49.815984 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1mkghr_b8af6bcb-5bcc-4465-85e0-778e48921c4c/pull/0.log" Jul 06 19:49:50.109737 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:50.109700 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-rddw8_bf9d0112-eb11-4996-8fca-e6a781f22e14/manager/0.log" Jul 06 19:49:50.132374 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:50.132327 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-5j2ll_f2e760ff-630e-4095-ae84-8377ff92111c/manager/0.log" Jul 06 19:49:50.159739 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:50.159706 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-q2m67_5ef34edc-9e6f-42ed-a3f1-ac7b914b2921/kuadrant-console-plugin/0.log" Jul 06 19:49:50.221612 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:50.221548 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-zfdvv_43e2d40f-c63e-4db9-b0f0-25cf4189f93d/registry-server/0.log" Jul 06 19:49:50.401364 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:50.399908 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-f78vh_b129a63b-6e6f-4ca8-bd52-05d8638059fd/manager/0.log" Jul 06 19:49:50.510629 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:50.509251 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-lxjln_75cfa14e-1d0b-490c-ac55-15c3715e8adb/manager/0.log" Jul 06 19:49:52.194638 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:52.194599 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-64786d9cbb-z9c9q_c4564790-651c-4af2-a036-c285bb57f10a/cluster-monitoring-operator/0.log" Jul 06 19:49:52.508315 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:52.508167 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-qbxt4_cc77fe73-70aa-4c81-8d65-fcf55a4b5f9c/node-exporter/0.log" Jul 06 19:49:52.536474 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:52.536437 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-qbxt4_cc77fe73-70aa-4c81-8d65-fcf55a4b5f9c/kube-rbac-proxy/0.log" Jul 06 19:49:52.561701 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:52.561666 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-qbxt4_cc77fe73-70aa-4c81-8d65-fcf55a4b5f9c/init-textfile/0.log" Jul 06 19:49:52.858598 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:52.858559 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-bkxlw_414f6ad7-4bba-4bff-99ec-9c29b9cb2c85/prometheus-operator/0.log" Jul 06 19:49:52.886413 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:52.886358 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-bkxlw_414f6ad7-4bba-4bff-99ec-9c29b9cb2c85/kube-rbac-proxy/0.log" Jul 06 19:49:53.056295 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:53.056273 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-65c786fdd5-6rpm8_61470d37-f33e-46ee-9282-76d61d3a2b0e/thanos-query/0.log" Jul 06 19:49:53.081842 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:53.081816 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-65c786fdd5-6rpm8_61470d37-f33e-46ee-9282-76d61d3a2b0e/kube-rbac-proxy-web/0.log" Jul 06 19:49:53.108680 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:53.108609 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-65c786fdd5-6rpm8_61470d37-f33e-46ee-9282-76d61d3a2b0e/kube-rbac-proxy/0.log" Jul 06 19:49:53.137609 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:53.137577 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-65c786fdd5-6rpm8_61470d37-f33e-46ee-9282-76d61d3a2b0e/prom-label-proxy/0.log" Jul 06 19:49:53.158986 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:53.158960 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-65c786fdd5-6rpm8_61470d37-f33e-46ee-9282-76d61d3a2b0e/kube-rbac-proxy-rules/0.log" Jul 06 19:49:53.179723 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:53.179693 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-65c786fdd5-6rpm8_61470d37-f33e-46ee-9282-76d61d3a2b0e/kube-rbac-proxy-metrics/0.log" Jul 06 19:49:55.619696 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:55.619598 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-p6v6j/perf-node-gather-daemonset-t28v4"] Jul 06 19:49:55.625823 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:55.625796 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-p6v6j/perf-node-gather-daemonset-t28v4" Jul 06 19:49:55.632406 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:55.632371 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-p6v6j/perf-node-gather-daemonset-t28v4"] Jul 06 19:49:55.692842 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:55.692386 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a069e0e7-1173-4fdd-8a8f-fe9adb66ca58-proc\") pod \"perf-node-gather-daemonset-t28v4\" (UID: \"a069e0e7-1173-4fdd-8a8f-fe9adb66ca58\") " pod="openshift-must-gather-p6v6j/perf-node-gather-daemonset-t28v4" Jul 06 19:49:55.692842 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:55.692440 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a069e0e7-1173-4fdd-8a8f-fe9adb66ca58-lib-modules\") pod \"perf-node-gather-daemonset-t28v4\" (UID: \"a069e0e7-1173-4fdd-8a8f-fe9adb66ca58\") " pod="openshift-must-gather-p6v6j/perf-node-gather-daemonset-t28v4" Jul 06 19:49:55.692842 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:55.692594 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ktn74\" (UniqueName: \"kubernetes.io/projected/a069e0e7-1173-4fdd-8a8f-fe9adb66ca58-kube-api-access-ktn74\") pod \"perf-node-gather-daemonset-t28v4\" (UID: \"a069e0e7-1173-4fdd-8a8f-fe9adb66ca58\") " pod="openshift-must-gather-p6v6j/perf-node-gather-daemonset-t28v4" Jul 06 19:49:55.692842 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:55.692756 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a069e0e7-1173-4fdd-8a8f-fe9adb66ca58-podres\") pod \"perf-node-gather-daemonset-t28v4\" (UID: \"a069e0e7-1173-4fdd-8a8f-fe9adb66ca58\") " pod="openshift-must-gather-p6v6j/perf-node-gather-daemonset-t28v4" Jul 06 19:49:55.692842 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:55.692787 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a069e0e7-1173-4fdd-8a8f-fe9adb66ca58-sys\") pod \"perf-node-gather-daemonset-t28v4\" (UID: \"a069e0e7-1173-4fdd-8a8f-fe9adb66ca58\") " pod="openshift-must-gather-p6v6j/perf-node-gather-daemonset-t28v4" Jul 06 19:49:55.794255 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:55.794215 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a069e0e7-1173-4fdd-8a8f-fe9adb66ca58-sys\") pod \"perf-node-gather-daemonset-t28v4\" (UID: \"a069e0e7-1173-4fdd-8a8f-fe9adb66ca58\") " pod="openshift-must-gather-p6v6j/perf-node-gather-daemonset-t28v4" Jul 06 19:49:55.794514 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:55.794478 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a069e0e7-1173-4fdd-8a8f-fe9adb66ca58-sys\") pod \"perf-node-gather-daemonset-t28v4\" (UID: \"a069e0e7-1173-4fdd-8a8f-fe9adb66ca58\") " pod="openshift-must-gather-p6v6j/perf-node-gather-daemonset-t28v4" Jul 06 19:49:55.794638 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:55.794574 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a069e0e7-1173-4fdd-8a8f-fe9adb66ca58-proc\") pod \"perf-node-gather-daemonset-t28v4\" (UID: \"a069e0e7-1173-4fdd-8a8f-fe9adb66ca58\") " pod="openshift-must-gather-p6v6j/perf-node-gather-daemonset-t28v4" Jul 06 19:49:55.794638 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:55.794607 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a069e0e7-1173-4fdd-8a8f-fe9adb66ca58-lib-modules\") pod \"perf-node-gather-daemonset-t28v4\" (UID: \"a069e0e7-1173-4fdd-8a8f-fe9adb66ca58\") " pod="openshift-must-gather-p6v6j/perf-node-gather-daemonset-t28v4" Jul 06 19:49:55.794750 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:55.794657 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ktn74\" (UniqueName: \"kubernetes.io/projected/a069e0e7-1173-4fdd-8a8f-fe9adb66ca58-kube-api-access-ktn74\") pod \"perf-node-gather-daemonset-t28v4\" (UID: \"a069e0e7-1173-4fdd-8a8f-fe9adb66ca58\") " pod="openshift-must-gather-p6v6j/perf-node-gather-daemonset-t28v4" Jul 06 19:49:55.794750 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:55.794696 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a069e0e7-1173-4fdd-8a8f-fe9adb66ca58-podres\") pod \"perf-node-gather-daemonset-t28v4\" (UID: \"a069e0e7-1173-4fdd-8a8f-fe9adb66ca58\") " pod="openshift-must-gather-p6v6j/perf-node-gather-daemonset-t28v4" Jul 06 19:49:55.794861 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:55.794815 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a069e0e7-1173-4fdd-8a8f-fe9adb66ca58-podres\") pod \"perf-node-gather-daemonset-t28v4\" (UID: \"a069e0e7-1173-4fdd-8a8f-fe9adb66ca58\") " pod="openshift-must-gather-p6v6j/perf-node-gather-daemonset-t28v4" Jul 06 19:49:55.794861 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:55.794852 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a069e0e7-1173-4fdd-8a8f-fe9adb66ca58-proc\") pod \"perf-node-gather-daemonset-t28v4\" (UID: \"a069e0e7-1173-4fdd-8a8f-fe9adb66ca58\") " pod="openshift-must-gather-p6v6j/perf-node-gather-daemonset-t28v4" Jul 06 19:49:55.794967 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:55.794912 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a069e0e7-1173-4fdd-8a8f-fe9adb66ca58-lib-modules\") pod \"perf-node-gather-daemonset-t28v4\" (UID: \"a069e0e7-1173-4fdd-8a8f-fe9adb66ca58\") " pod="openshift-must-gather-p6v6j/perf-node-gather-daemonset-t28v4" Jul 06 19:49:55.804289 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:55.804253 2566 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ktn74\" (UniqueName: \"kubernetes.io/projected/a069e0e7-1173-4fdd-8a8f-fe9adb66ca58-kube-api-access-ktn74\") pod \"perf-node-gather-daemonset-t28v4\" (UID: \"a069e0e7-1173-4fdd-8a8f-fe9adb66ca58\") " pod="openshift-must-gather-p6v6j/perf-node-gather-daemonset-t28v4" Jul 06 19:49:55.939588 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:55.939493 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-p6v6j/perf-node-gather-daemonset-t28v4" Jul 06 19:49:56.112869 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:56.112840 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-p6v6j/perf-node-gather-daemonset-t28v4"] Jul 06 19:49:56.116965 ip-10-0-138-35 kubenswrapper[2566]: W0706 19:49:56.116901 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-poda069e0e7_1173_4fdd_8a8f_fe9adb66ca58.slice/crio-6b71ed8b644cdc978733c5c3c9442161c81d4ac2d863d757b75b92f96d61a873 WatchSource:0}: Error finding container 6b71ed8b644cdc978733c5c3c9442161c81d4ac2d863d757b75b92f96d61a873: Status 404 returned error can't find the container with id 6b71ed8b644cdc978733c5c3c9442161c81d4ac2d863d757b75b92f96d61a873 Jul 06 19:49:56.694298 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:56.694256 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-p6v6j/perf-node-gather-daemonset-t28v4" event={"ID":"a069e0e7-1173-4fdd-8a8f-fe9adb66ca58","Type":"ContainerStarted","Data":"f046a70aa7ed055278d4d114299ce40803adb4287bf423223706552b2ae0d5e5"} Jul 06 19:49:56.694681 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:56.694328 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-p6v6j/perf-node-gather-daemonset-t28v4" Jul 06 19:49:56.694681 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:56.694351 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-p6v6j/perf-node-gather-daemonset-t28v4" event={"ID":"a069e0e7-1173-4fdd-8a8f-fe9adb66ca58","Type":"ContainerStarted","Data":"6b71ed8b644cdc978733c5c3c9442161c81d4ac2d863d757b75b92f96d61a873"} Jul 06 19:49:56.711686 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:56.711633 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-p6v6j/perf-node-gather-daemonset-t28v4" podStartSLOduration=1.711608824 podStartE2EDuration="1.711608824s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 19:49:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:49:56.709163687 +0000 UTC m=+2422.016522636" watchObservedRunningTime="2026-07-06 19:49:56.711608824 +0000 UTC m=+2422.018967771" Jul 06 19:49:57.377896 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:57.377870 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-jg5kg_38cfd0e8-69ad-4b63-bd2d-74e5757ca860/dns/0.log" Jul 06 19:49:57.397197 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:57.397171 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-jg5kg_38cfd0e8-69ad-4b63-bd2d-74e5757ca860/kube-rbac-proxy/0.log" Jul 06 19:49:57.506014 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:57.505982 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-8btbx_c75820ce-6423-4c3f-8e13-0ab0e5b00914/dns-node-resolver/0.log" Jul 06 19:49:58.012894 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:58.012859 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-788dd79d58-8qj85_d7905780-4864-4b14-8c62-b56030fbbe81/registry/0.log" Jul 06 19:49:58.055388 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:58.055358 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-qbwhx_e37e30a5-e12f-4c79-a1ff-28c9460d44b3/node-ca/0.log" Jul 06 19:49:58.952757 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:58.952708 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4kzvjv_fc73ba95-f338-4688-a38a-ee10bbce473d/istio-proxy/0.log" Jul 06 19:49:59.191112 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:59.191087 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-754576df6f-jrtrg_e1be9243-b0bd-493c-8e06-e3f6c2c8a9a7/kube-auth-proxy/0.log" Jul 06 19:49:59.222792 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:59.222678 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-t9fgz_0af75ee7-3c24-4d81-9764-64b52e4f3623/istio-proxy/0.log" Jul 06 19:49:59.261117 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:59.261077 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-t9fgz_0af75ee7-3c24-4d81-9764-64b52e4f3623/istio-proxy/1.log" Jul 06 19:49:59.353813 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:49:59.353780 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-vt6xf_1e8eaff3-cc54-4648-b0c6-cb184ba29ff8/payload-processing/0.log" Jul 06 19:50:00.064032 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:50:00.064002 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-lcp9g_0315bc99-2d7d-4988-a643-58c780793906/serve-healthcheck-canary/0.log" Jul 06 19:50:00.596627 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:50:00.596602 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-hqnrn_1a6ef785-86f6-412f-8ef3-300db1b24c2c/kube-rbac-proxy/0.log" Jul 06 19:50:00.616501 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:50:00.616466 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-hqnrn_1a6ef785-86f6-412f-8ef3-300db1b24c2c/exporter/0.log" Jul 06 19:50:00.637286 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:50:00.637255 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-hqnrn_1a6ef785-86f6-412f-8ef3-300db1b24c2c/extractor/0.log" Jul 06 19:50:02.718473 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:50:02.718440 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-p6v6j/perf-node-gather-daemonset-t28v4" Jul 06 19:50:02.748302 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:50:02.748274 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-wkh6n_abe694ae-dad6-43a4-a8e0-767db8e50929/manager/0.log" Jul 06 19:50:02.924923 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:50:02.924894 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-5d68c7bd7c-4ltxv_f6e6c625-f78a-4051-bf1f-10953d59dba1/maas-api/0.log" Jul 06 19:50:03.413191 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:50:03.413166 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-bl6x9_b2160904-d8c6-4fd7-a41f-a10ff617640f/server/0.log" Jul 06 19:50:03.513939 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:50:03.513905 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-sdhp4_c10a5b93-4175-4ac7-a141-6c248f02bd8c/manager/0.log" Jul 06 19:50:03.542263 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:50:03.542221 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-55c47dfb7f-tbk57_f4693d1d-13dd-4b20-a725-f722f97cee41/manager/0.log" Jul 06 19:50:03.572456 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:50:03.572430 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-55c47dfb7f-tbk57_f4693d1d-13dd-4b20-a725-f722f97cee41/copy-manifests/0.log" Jul 06 19:50:05.413819 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:50:05.413786 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4kzvjv_fc73ba95-f338-4688-a38a-ee10bbce473d/istio-proxy/0.log" Jul 06 19:50:05.423464 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:50:05.423428 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-zkrxk_cfefa92a-56b5-4754-8269-4e840af96bc5/openshift-lws-operator/0.log" Jul 06 19:50:05.630699 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:50:05.630610 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-754576df6f-jrtrg_e1be9243-b0bd-493c-8e06-e3f6c2c8a9a7/kube-auth-proxy/0.log" Jul 06 19:50:05.660804 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:50:05.660771 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-t9fgz_0af75ee7-3c24-4d81-9764-64b52e4f3623/istio-proxy/0.log" Jul 06 19:50:05.700289 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:50:05.700255 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-t9fgz_0af75ee7-3c24-4d81-9764-64b52e4f3623/istio-proxy/1.log" Jul 06 19:50:05.784820 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:50:05.784783 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-vt6xf_1e8eaff3-cc54-4648-b0c6-cb184ba29ff8/payload-processing/0.log" Jul 06 19:50:06.996851 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:50:06.996763 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4kzvjv_fc73ba95-f338-4688-a38a-ee10bbce473d/istio-proxy/0.log" Jul 06 19:50:07.218488 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:50:07.218442 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-754576df6f-jrtrg_e1be9243-b0bd-493c-8e06-e3f6c2c8a9a7/kube-auth-proxy/0.log" Jul 06 19:50:07.241966 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:50:07.241925 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-t9fgz_0af75ee7-3c24-4d81-9764-64b52e4f3623/istio-proxy/0.log" Jul 06 19:50:07.282347 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:50:07.282262 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-t9fgz_0af75ee7-3c24-4d81-9764-64b52e4f3623/istio-proxy/1.log" Jul 06 19:50:07.359897 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:50:07.359865 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-vt6xf_1e8eaff3-cc54-4648-b0c6-cb184ba29ff8/payload-processing/0.log" Jul 06 19:50:10.351161 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:50:10.351130 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-kgvj2_d4495041-9b50-48f1-93fc-631d2979b534/migrator/0.log" Jul 06 19:50:10.373774 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:50:10.373743 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-d57d58cc9-kgvj2_d4495041-9b50-48f1-93fc-631d2979b534/graceful-termination/0.log" Jul 06 19:50:11.770993 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:50:11.770965 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-65k2z_e9a3a524-5e1b-48db-ba03-750883088470/kube-multus-additional-cni-plugins/0.log" Jul 06 19:50:11.794579 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:50:11.794553 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-65k2z_e9a3a524-5e1b-48db-ba03-750883088470/egress-router-binary-copy/0.log" Jul 06 19:50:11.815663 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:50:11.815638 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-65k2z_e9a3a524-5e1b-48db-ba03-750883088470/cni-plugins/0.log" Jul 06 19:50:11.835721 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:50:11.835697 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-65k2z_e9a3a524-5e1b-48db-ba03-750883088470/bond-cni-plugin/0.log" Jul 06 19:50:11.858010 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:50:11.857985 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-65k2z_e9a3a524-5e1b-48db-ba03-750883088470/routeoverride-cni/0.log" Jul 06 19:50:11.878491 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:50:11.878467 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-65k2z_e9a3a524-5e1b-48db-ba03-750883088470/whereabouts-cni-bincopy/0.log" Jul 06 19:50:11.898565 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:50:11.898540 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-65k2z_e9a3a524-5e1b-48db-ba03-750883088470/whereabouts-cni/0.log" Jul 06 19:50:12.315256 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:50:12.315223 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-s84ln_51b731db-0d7f-4a6d-bd3b-30d452687c30/kube-multus/0.log" Jul 06 19:50:12.337908 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:50:12.337881 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-8mnph_6b76700e-1870-40a7-a1f2-ab3c1332058c/network-metrics-daemon/0.log" Jul 06 19:50:12.356427 ip-10-0-138-35 kubenswrapper[2566]: I0706 19:50:12.356400 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-8mnph_6b76700e-1870-40a7-a1f2-ab3c1332058c/kube-rbac-proxy/0.log"